
<Sponsorships>

[**API Collaboration**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/)
has never been easier with comments on collections, folders, and requests using Postman!
Launch into hyper-efficiency using comments to discuss your work with your teammates in Postman.
[**Learn more here**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/).

Build automations with [**UiPath Autopilot™**](https://rebrand.ly/dev-dives-webinar).
Join this free webinar to learn how to build automation workflows, generate code, and create ready-to-use apps by simply describing your needs to the AI-powered Autopilot.
[Register today](https://rebrand.ly/dev-dives-webinar)!

</Sponsorships>

Redis is a popular choice for [**caching data**](caching-in-aspnetcore-improving-application-performance), but its capabilities go far beyond that.
One of its lesser-known features is Pub/Sub support.
Redis channels offer an interesting approach for implementing real-time messaging in your .NET applications.
However, as you'll soon see, channels also have some drawbacks.

In this week's newsletter, we'll explore:

- Basics of Redis channels
- Practical use cases for channels
- Implementing a Pub/Sub example in .NET
- Cache invalidation in distributed systems

Let's dive in.

## Redis Channels

Redis channels are named communication channels that implement the [Publish/Subscribe messaging paradigm](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern).
Each channel is identified by a unique name (e.g., `notifications`, `updates`).
Channels facilitate message delivery from publishers to subscribers.

Publishers use the `PUBLISH` command to send messages to a specific channel.
Subscribers use the `SUBSCRIBE` command to register interest in receiving messages from a channel.

<Image
  src="blogs/mnw_100/redis_channel.png"
  width={2094}
  height={1294}
  alt="Redis channel with publisher and three subscribers."
/>

Redis channels follow a topic-based publish-subscribe model.
Multiple publishers can send messages to a channel, and multiple subscribers can receive messages from that channel.

However, it's crucial to note that Redis channels do not store messages.
If there are no subscribers for a channel when a message is published, that message is immediately discarded.

Redis channels have an **at-most-once delivery** semantics.

## Practical Use Cases

Given that Redis channels operate with **at-most-once delivery** (messages might be lost if there are no subscribers),
they are well-suited for scenarios where occasional message loss is acceptable and real-time or near-real-time communication is desired.

Here are a few possible use cases:

- **Social media feeds**: Broadcasting new posts or updates to users.
- **Live score updates**: Sending live game scores or sports updates to subscribers.
- **Chat applications**: Delivering chat messages in real-time to active participants.
- **Collaborative editing**: Propagating changes in collaborative editing environments.
- **Distributed cache updates**: Invalidating cache entries across multiple servers when data changes. We'll cover this in detail later in the article.

Redis channels aren't the best choice for critical data where message loss is unacceptable.
In such cases, you should consider a more reliable messaging system.

Let's see how we can use Redis channels in .NET.

## Pub/Sub With Redis Channels

We will use the `StackExchange.Redis` library to send messages with Redis channels.

Let's start by installing it:

```powershell
Install-Package StackExchange.Redis
```

You can run [Redis](https://redis.io/) locally in a Docker container.
The default port is `6379`.

```
docker run -it -p 6379:6379 redis
```

Here's a simple background service that'll act as our message `Producer`.

We're creating a `ConnectionMultiplexer` by connecting to our Redis instance.
This allows us to obtain an `ISubscriber` that we can use for pub/sub messaging.
The `ISubscriber` will enable us to publish a message to a channel by specifying the channel name.

```csharp {11,19}
public class Producer(ILogger<Producer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        while (!stoppingToken.IsCancellationRequested)
        {
            var message = new Message(Guid.NewGuid(), DateTime.UtcNow);

            var json = JsonSerializer.Serialize(message);

            await subscriber.PublishAsync(Channel, json);

            logger.LogInformation(
                "Sending message: {Channel} - {@Message}",
                message);

            await Task.Delay(5000, stoppingToken);
        }
    }
}
```

Let's also introduce a separate background service for consuming messages.

The `Consumer` connects to the same Redis instance and obtains an `ISubscriber`.
The `ISubscriber` exposes a `SubscribeAsync` method that we can use to subscribe to messages from a given channel.
This method accepts a callback delegate that we can use to handle the message.

```csharp
public class Consumer(ILogger<Consumer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        await subscriber.SubscribeAsync(Channel, (channel, message) =>
        {
            var message = JsonSerializer.Deserialize<Message>(message);

            logger.LogInformation(
                "Received message: {Channel} - {@Message}",
                channel,
                message);
        });
    }
}
```

Finally, here's what we get when we run both the `Producer` and `Consumer` services:

<Image
  src="blogs/mnw_100/redis_pub_sub.gif"
  width={1920}
  height={500}
  alt="Redis channels publish/subscribe demo."
/>

## Cache Invalidation in Distributed Systems

In a recent project, I tackled a common challenge in distributed systems: keeping the caches in sync.
We were using a two-level caching approach.
First, we had an in-memory cache on each web server for super-fast access.
Second, we had a shared Redis cache to avoid hitting our database too often.

The problem was that when data changed in the database, we needed a way to quickly tell all the web servers to clear their in-memory caches.
This is where Redis Pub/Sub came to the rescue.
We set up a Redis channel specifically for cache invalidation messages.

Each application would run a `CacheInvalidationBackgroundService` that subscribes to messages from the cache invalidation channel.

```csharp
public class CacheInvalidationBackgroundService(
    IServiceProvider serviceProvider)
    : BackgroundService
{
    public const string Channel = "cache-invalidation";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await subscriber.SubscribeAsync(Channel, (channel, key) =>
        {
            var cache = serviceProvider.GetRequiredService<IMemoryCache>();

            cache.Remove(key);

            return Task.CompletedTask;
        });
    }
}
```

Whenever data changes in the database, we publish a message on this channel with the cache key of the updated data.
All the web servers are subscribed to this channel, so they instantly know to remove the old data from their in-memory caches.
Since the in-memory cache is wiped if the application isn't running, losing cache invalidation messages isn't a problem.
This keeps our caches consistent and ensures our users always see the most up-to-date information.

## In Summary

Redis Pub/Sub is not a silver bullet for every messaging need, but its simplicity and speed make it a valuable tool.
Channels allow us to easily implement communication between loosely coupled components.

Redis channels have at-most-once delivery semantics, so they're best suited for cases where the occasional dropped message is acceptable.

I used it to solve the challenge of synchronizing caches across multiple servers.
This allowed our system to serve up-to-date data without sacrificing performance.

**P.S.** When you're ready to dive deeper into creating message-driven systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
I have an entire module dedicated to building reliable distributed messaging and event-driven architecture.

Good luck out there, and see you next week.

---



<Sponsorships>

Streamline API development with [**Package Libary in Postman**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).
Add commonly-used scripts and tests to packages in your team's Package Library, and reuse them in your personal, private, and team workspaces using Postman.
[**Learn more here**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).

The [**IronOCR C# library**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan)
has hit a new milestone with 2 million NuGet downloads!
To celebrate, we're offering a 20% discount on IronOCR team licenses (Lite, Plus, Pro).
Plus, we're excited to introduce AdvancedScan, a powerful new feature that enhances OCR accuracy and Multiple-Language Support.
[**Try it Today**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

Modular monoliths strike a balance between the simplicity of monolithic architecture and the flexibility of microservices.
By breaking down applications into cohesive modules, modular monoliths enable easier development and maintenance.
However, they still have a single codebase and deployment unit.

A critical aspect of the success of a modular monolith is the interaction between its modules.

**System integration testing (SIT)** is an approach to verifying the collaboration of these modules.
It involves testing the integration points and communication mechanisms between modules.

System integration testing allows us to validate the entire system's behavior.
This allows us to catch problems early before they become big headaches.

In this article, we'll learn how to test a modular monolith using system integration testing.

We'll look at real-world examples and discuss why this testing approach is useful.

## What Is System Integration Testing?

System integration testing (SIT) is an approach to testing the interactions between various modules within a single system.
The system could contain many external services, which should also be included during testing.

System integration testing is the perfect testing approach for modular monoliths.
It allows you to mimic the real-world execution of your application (as you'll see later).

The key benefits of system integration testing are:

- **Detecting integration issues**: Discover problems from module integrations that unit testing might miss.
- **Validating business logic**: Confirm that end-to-end business processes function correctly across modules.
- **Ensuring data integrity**: Verify that data flows accurately and consistently between modules.
- **Improving system stability**: Identify and resolve issues early, leading to a more reliable system.
- **Building confidence**: Ensure that the system is well-integrated and ready for deployment.

## Modular Monolith System Example

A modular monolith consists of multiple modules, each with a distinct responsibility.
Modules represent high-level components with well-defined boundaries.
The modules essentially group together related functionalities (use cases).
If you want to learn more about this software architecture, check out this [**introduction to modular monoliths**](what-is-a-modular-monolith).

Let's consider a hypothetical ticketing system that consists of several modules:

- **Users Module**: Handles user administration and authentication.
- **Events Module**: Manages events, scheduling, and ticket availability.
- **Ticketing Module**: Allows users to purchase tickets for various events.
- **Attendance Module**: Allows users to check into events using their tickets.

<figure>
  <Image
    src="blogs/mnw_099/modular_monolith.png"
    width={2506}
    height={1282}
    alt="Modular monolith UML diagram."
  />
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>

During system integration testing, we want to ensure that all these modules interact correctly and
fulfill the business requirements of the ticketing system.

## Testing Modular Monoliths: User Registration

The modules in our ticketing system represent different [**bounded contexts**](monolith-to-microservices-how-a-modular-monolith-helps).
The **Users Module** uses the term `User` as its core entity.
However, the **Ticketing Module** uses the term `Customer` because it's more aligned with its core responsibility of selling tickets.
Conceptually, both entities represent the same person within our system.

Here's the scenario we want to test:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record

<Image
  src="blogs/mnw_099/user_registration.png"
  width={2866}
  height={877}
  alt="User registration flow diagram."
/>

If you haven't figured it out by now, our [**modules communicate asynchronously**](modular-monolith-communication-patterns) using messaging.

Why is this important for our tests?

There is a time delay from when something occurs in one module until the other modules are notified and handle the integration events.
Our system integration tests will have to take into account this delay.

Here's the test for this scenario:

```csharp
public class RegisterUserTests : BaseIntegrationTest
{
    public RegisterUserTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task RegisterUser_Should_PropagateToTicketingModule()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);

                var customerResult = await Sender.Send(query);

                return customerResult;
            });

        // Assert
        customerResult.IsSuccess.Should().BeTrue();
        customerResult.Value.Should().NotBeNull();
    }
}
```

A lot is going on here, so let's unpack the steps:

- We're using the `WebApplicationFactory` to run an in-memory application instance
- Any external dependencies run in a Docker container using [**Testcontainers**](testcontainers-integration-testing-using-docker-in-dotnet)
- We send the `RegisterUserCommand` to register a user in the **Users Module**
- Then, we have to poll the **Ticketing Module** by sending a `GetCustomerQuery`
- The `Poller` allows us to wait until the system eventually becomes consistent

You can learn more about [**integration testing with Testcontainers in this article**](testcontainers-integration-testing-using-docker-in-dotnet).

System integration tests take longer to execute than integration tests for one module.
This is a side effect of testing a bigger slice of the overall system in each test case.

The `Poller` class allows you to execute a delegate until you receive a successful result or a timeout occurs.
You can customize the timeout duration based on the scenario you are testing.

```csharp
internal static class Poller
{
    private static readonly Error Timeout =
        Error.Failure("Poller.Timeout", "The poller has time out");

    internal static async Task<Result<T>> WaitAsync<T>(
        TimeSpan timeout,
        Func<Task<Result<T>>> func)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));

        DateTime endTimeUtc = DateTime.UtcNow.Add(timeout);
        while (DateTime.UtcNow < endTimeUtc &&
               await timer.WaitForNextTickAsync())
        {
            Result<T> result = await func();

            if (result.IsSuccess)
            {
                return result;
            }
        }

        return Result.Failure<T>(Timeout);
    }
}
```

## Testing Modular Monoliths: Adding Ticket to Cart

Here's a more complex scenario testing adding a ticket to the customer's cart:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record
- The **Ticketing Module** creates a dummy event and adds a ticket to the customer's cart

This scenario mimics a user registering with our system, finding a ticket they want to purchase, and adding it to their cart.
We can extend this scenario with more test cases, like ticket purchases.

```csharp
public sealed class AddItemToCartTests : BaseIntegrationTest
{
    private const decimal Quantity = 10;

    public AddItemToCartTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Customer_ShouldBeAbleTo_AddItemToCart()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);
                var customerResult = await Sender.Send(query);
                return customerResult;
            });

        customerResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Add item to cart
        CustomerResponse customer = customerResult.Value;
        var ticketTypeId = Guid.NewGuid();

        await Sender.CreateEventAsync(Guid.NewGuid(), ticketTypeId, Quantity);

        Result result = await Sender.Send(
            new AddItemToCartCommand(customer.Id, ticketTypeId, Quantity));

        // Assert
        result.IsSuccess.Should().BeTrue();
    }
}
```

What I like about system integration testing is that the test cases aren't complicated to write.
The tests execute the use cases of our system and verify the side effects.
Building your system around [**use cases**](building-your-first-use-case-with-clean-architecture) has a few advantages,
the main one being that you can focus on the core business logic.
But it also makes (integration) testing easier.
The use case runs by sending a command or a query.
We can execute the use cases in some logical order and check that we get the expected result.

## In Conclusion

System integration testing is a crucial step in creating a successful modular monolith.
By testing how different modules work together, you can find and fix problems before they become bigger issues.
This makes your software more reliable and saves you time and resources in the long run.

If you want to dive deeper into building modular systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
There's an entire chapter dedicated to testing,
including advanced techniques like system integration testing, using Testcontainers for external services, and automated testing with CI/CD pipelines.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713)

Collaborate directly with your external partners in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/) to help your partners consume your team's APIs and work together on API projects.
Check out Postman's Partner Workspaces, now available for Professional and Enterprise plans in [**Postman v11**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/).

</Sponsorships>

This is a question I often hear: how do I design my use case with Clean Architecture?

I understand the confusion.
Figuring out what to place in the Domain, Application, and Infrastructure layer can seem complicated.
If that's not enough, we also have to decide what makes up a use case and what should be abstracted away.

However, things become simpler if we adhere to the main rule in Clean Architecture — **the Dependency Rule**.
This rule states that source code dependencies can only point inwards.

In this newsletter, we'll explore a practical example of how to apply Clean Architecture principles by building a user registration feature.

## Clean Architecture

[**Clean Architecture**](why-clean-architecture-is-great-for-complex-projects) has emerged as a guiding principle for crafting maintainable, scalable, and testable applications.
At its core, Clean Architecture emphasizes the **separation of concerns** and the **dependency rule**.
The dependency rule dictates that dependencies should point inward toward higher-level modules.
By following this rule, you create a system where the core business logic of your application is decoupled from external dependencies.
This makes it more adaptable to changes and easier to test.

<Image
  src="blogs/mnw_098/clean_architecture.png"
  width={1280}
  height={714}
  alt="Clean Architecture diagram."
/>

The Domain layer encapsulates enterprise-wide business rules.
It contains domain entities, where an entity is typically an object with methods.

The Application layer contains application-specific business rules and encapsulates all of the system's use cases.
A use case orchestrates the flow of data to and from the domain entities and calls the methods exposed by the entities
to achieve its goals.

The Infrastructure and Presentation layers deal with external concerns.
Here, you will implement any abstractions defined in the inner layers.

## Describing The Use Case

What does it mean for a user to register with our application?
It means they reserve an email address (or username) to identify themselves and be able to interact with our system.
The user could provide other information, such as a first and last name, an address, and a phone number.

The first step in building any feature is clearly defining the desired result.

For user registration, this is what the required operations are:

- The user provides an email and password for registration
- Verify that the email was not reserved previously by an existing account
- Hash the password using some cryptographic hash function (e.g., SHA-256, SHA-512)
- Store the user in the database and (optionally) return an access token to the client

We could also consider any domain-specific rules or validations that we must enforce.
A good example is password strength, where we could implement minimum length and complexity requirements.

Now that we have our requirements let's see how to translate them into a use case.

## Implementing the Use Case

With our requirements in place, we can now define the user registration use case.
In [**Clean Architecture**](clean-architecture-and-the-benefits-of-structured-software-design), use cases live in the Application layer and orchestrate the interactions between domain entities and external dependencies.

Let's name our use case `RegisterUser`.
Its input will be a `RegistrationRequest` object containing the user's registration data,
and its output will be a `RegistrationResult` object indicating the outcome of the registration attempt.
Notice that we are using a feature-driven name for the use case.

What about any external dependencies?
If the use case needs to interact with an external system or infrastructure component, we abstract that behind an interface.
Remember, your application's core business logic should be decoupled from external dependencies.

The `RegisterUser` class will use dependency injection to get the necessary dependencies:

- `IUserRepository`: An interface for accessing user data from the database.
- `IPasswordHasher`: An interface for hashing passwords securely.

The `RegisterUser` use case will follow these steps:

1. Validate input data
2. Check for existing `User`
3. Hash the password
4. Create a new `User` entity
5. Save the `User` to the database
6. Return the result

Finally, here's the code for our `RegisterUser` use case:

```csharp
public class RegisterUser(
    IUserRepository userRepository,
    IPasswordHasher passwordHasher)
{
    public async Task<RegistrationResult> Handle(RegistrationRequest request)
    {
        // Validation omitted for brevity

        if (await userRepository.ExistsAsync(request.Email))
        {
            return RegistrationResult.EmailNotUnique;
        }

        var passwordHash = passwordHasher.Hash(request.Password);

        var user = User.Create(
            request.FirstName, request.LastName, request.Email, passwordHash);

        await userRepository.InsertAsync(user);

        return RegistrationResult.Success;
    }
}
```

A big benefit of this approach is that we can immediately write tests for the `RegisterUser` use case.
We can provide mocks for external dependencies in the tests.
We don't need the implementations to exist for this code to compile.
With mocks, we can test our business rules and validate our implementation.

**Action step**: How would you extend the `RegisterUser` use case with more functionality?

Here are two examples:

- Adding an external identity provider
- Implementing email verification

## Where Clean Architecture Becomes Muddled

By designing our application with Clean Architecture, we produce a system independent of external concerns.
We define abstractions in the Application layer and implement them in the Infrastructure layer.
So far, so good.

However, this doesn't mean you can disregard how you integrate with external dependencies.

In theory, we should be able to "swap" the implementation for any external concern and call it a day.
In practice, this couldn't be further from the truth.

Let me give you two practical examples using the user registration flow.

### Race Conditions

The `RegisterUser` use case has a race condition.
Concurrent requests could pass the check for email uniqueness and proceed to register the user.

We could prevent this race condition by introducing a lock before checking for email uniqueness.
That way, only one request will pass the check and proceed to save the user in the database.

```csharp
if (await userRepository.ExistsAsync(request.Email))
{
    return RegistrationResult.EmailNotUnique;
}
```

However, there is a much more elegant way to solve this.
We can introduce a unique index on the `Email` column in the database.
A unique index guarantees that only one transaction can write the unique value to the database.
The losing transaction will return an error.

We can handle this exception on the application side and return an appropriate error message to the user.
The `IUserRepository.InsertAsync` method implementation can encapsulate this logic.

### Changing Hash Functions

Let's say we found a security flaw in the hash function used in the `IPasswordHasher` implementation.
So, we spend a few minutes switching to a more secure hash function.
The tests for the `RegisterUser` use case are all green, and everything seems fine.

The problem? All existing users can no longer log in to the system.

When an existing user tries to log in with their email and password, the new `IPasswordHasher.Hash` implementation
returns a different password hash from the one stored in the database.

The correct approach is to phase out the old password hash for existing users.
We can add a column in the database that says which hash function produced the hash.
We will verify the user's password using the correct hashing function during the login process.

If the user's password hash still uses the old hashing function, we will verify their password first.
Then, we can use the password (which we have in memory) to produce a hash using the new hash function.
We will store the hash in the database and update the hash function column to the new algorithm.

Slowly, we will phase out passwords using the old hash function.

## Conclusion

I hope this was helpful in understanding how to apply Clean Architecture principles to a real-world scenario.
By focusing on the core business logic first (what it means for a user to register), we can define the requirements for our use case.
Translating these requirements into a series of steps within the use case is the easy part.

But Clean Architecture won't save you from bad engineering.
If you don't understand what you are abstracting away, it will become a problem in the long term.

If you want to go deeper, my flagship course, [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture),
takes the guesswork out of structuring your project the right way.
I share my entire framework for building robust applications from the ground up -
from building a rich domain model to creating use cases to getting your application ready for production.

And that's all for this week.

See you next Saturday.

---



<Sponsorships>

Build and test automations with [**UiPath**](https://rebrand.ly/dev-dives-uipath).
Join this free webinar to learn how the UiPath Platform enables you to build C# coded automations and quickly test them using GenAI.
[**Save your seat here**](https://rebrand.ly/dev-dives-uipath)!

Design and mock APIs with [**Postman Mock Servers**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/)!
Build higher-quality APIs faster with Postman mock API servers which simplify design and planning, support split-stack development,
and help you ensure that your API will run the way it's supposed to in production.
[**Learn more here**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/).

</Sponsorships>

Microservices have revolutionized how we build and scale applications.
By breaking down larger systems into smaller, independent services, we gain flexibility, agility, and the ability to adapt to changing requirements quickly.
However, microservices systems are also very dynamic.
Services can come and go, scale up or down, and even move around within your infrastructure.

This dynamic nature presents a significant challenge. How do your services find and communicate with each other reliably?

Hardcoding IP addresses and ports is a recipe for fragility.
If a service instance changes location or a new instance spins up, your entire system could grind to a halt.

Service discovery acts as a central directory for your microservices.
It provides a mechanism for services to register themselves and discover the locations of other services.

In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.

## What is Service Discovery?

Service discovery is a pattern that allows developers to use logical names to refer to external services instead of physical IP addresses and ports.
It provides a centralized location for services to register themselves.
Clients can query the service registry to find out the service's physical address.
This is a common pattern in large-scale distributed systems, such as Netflix and Amazon.

Here's what the service discovery flow looks like:

1. The service will register itself with the service registry
2. The client must query the service registry to get the physical address
3. The client sends the request to the service using the resolved physical address

<Image
  src="blogs/mnw_097/service_discovery_flow.png"
  width={1377}
  height={983}
  alt="Service discovery flow."
/>

The same concept applies when we have multiple services we want to call.
Each service would register itself with the service registry.
The client uses a logical name to reference a service and resolves the physical address from the service registry.

<Image
  src="blogs/mnw_097/service_discovery_microservices.png"
  width={2074}
  height={883}
  alt="Service discovery with multiple microservices."
/>

The most popular solutions for service discovery are Netflix [Eureka](https://github.com/Netflix/eureka) and HashiCorp [Consul](https://www.consul.io/).

There is also a lightweight solution from Microsoft in the `Microsoft.Extensions.ServiceDiscovery` library.
It uses application settings to resolve the physical addresses for services, so some manual work is still required.
However, you can store service locations in [Azure App Configuration](https://azure.microsoft.com/en-us/products/app-configuration) for a centralized service registry.
I will explore this service discovery library in some future articles.

But now I want to show you how to integrate Consul with .NET applications.

## Setting Up the Consul Server

The simplest way to run the Consul server locally is using a Docker container.
You can create a container instance of the `hashicorp/consul` image.

Here's an example of configuring the Consul service as part of the `docker-compose` file:

```yml
consul:
  image: hashicorp/consul:latest
  container_name: Consul
  ports:
    - '8500:8500'
```

If you navigate to `localhost:8500`, you will be greeted by the Consul Dashboard.

<Image
  src="blogs/mnw_097/consul_dashboard.png"
  width={1902}
  height={907}
  alt="Consul dashboard."
/>

Now, let's see how to register our services with Consul.

## Service Registration in .NET With Consul

We'll use the [Steeltoe Discovery](https://docs.steeltoe.io/api/v3/discovery/) library to implement service discovery with Consul.
The Consul client implementation lets your applications register services with a Consul server and discover services registered by other applications.

Let's install the `Steeltoe.Discovery.Consul` library:

```powershell
Install-Package Steeltoe.Discovery.Consul
```

We have to configure some services by calling `AddServiceDiscovery` and explicitly configuring the Consul service discovery client.
The alternative is calling `AddDiscoveryClient` which uses reflection at runtime to determine which service registry is available.

```csharp {6}
using Steeltoe.Discovery.Client;
using Steeltoe.Discovery.Consul;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddServiceDiscovery(o => o.UseConsul());

var app = builder.Build();

app.Run();
```

Finally, our service can register with Consul by configuring the logical service name through application settings.
When the application starts, the `reporting-service` logical name will be added to the Consul service registry.
Consul will store the respective physical address of this service.

```json
{
  "Consul": {
    "Host": "localhost",
    "Port": 8500,
    "Discovery": {
      "ServiceName": "reporting-service",
      "Hostname": "reporting-api",
      "Port": 8080
    }
  }
}
```

When we start the application and open the Consul dashboard, we should be able to see the `reporting-service` and its respective physical address.

<Image
  src="blogs/mnw_097/consul_dashboard_with_service.png"
  width={1902}
  height={907}
  alt="Consul dashboard with registered service."
/>

## Using Service Discovery

We can use service discovery when making HTTP calls with an `HttpClient`.
Service discovery allows us to use a logical name for the service we want to call.
When sending a network request, the service discovery client will replace the logical name with a correct physical address.

In this example, we're configuring the base address of the `ReportingServiceClient` typed client to `http://reporting-service`
and adding service discovery by calling `AddServiceDiscovery`.

Load balancing is an optional step, and we can configure it by calling `AddRoundRobinLoadBalancer` or `AddRandomLoadBalancer`.
You can also configure a custom load balancing strategy by providing an `ILoadBalancer` implementation.

```csharp
builder.Services
    .AddHttpClient<ReportingServiceClient>(client =>
    {
        client.BaseAddress = new Uri("http://reporting-service");
    })
    .AddServiceDiscovery()
    .AddRoundRobinLoadBalancer();
```

We can use the `ReportingServiceClient` typed client like a regular `HttpClient` to make requests.
The service discovery client sends the request to the external service's IP address.

```csharp
app.MapGet("articles/{id}/report",
    async (Guid id, ReportingServiceClient client) =>
    {
        var response = await client
            .GetFromJsonAsync<Response>($"api/reports/article/{id}");

        return response;
    });
```

## Takeaway

Service discovery simplifies the management of microservices by automating service registration and discovery.
This eliminates the need for manual configuration updates, reducing the risk of errors.

Services can discover each other's locations on demand, ensuring that communication channels remain open even as the service landscape evolves.
By enabling services to discover alternative service instances in case of outages or failures, service discovery enhances the overall resilience of the microservices system.

Mastering service discovery gives you a powerful tool to build modern distributed applications.

You can grab the [source code for this example here](https://github.com/m-jovanovic/service-discovery-consul).

Thanks for reading, and I'll see you next week!

---



<Sponsorships>

Store secrets in your [**Postman Vault**](https://learning.postman.com/docs/sending-requests/postman-vault/postman-vault-secrets/)!
Postman Vault enables you to store sensitive data as vault secrets in your local instance of Postman.
This enables you to securely reuse sensitive data in your HTTP collections and requests.
[**Learn more and get started using Vault**](https://learning.postman.com/docs/sending-requests/postman-vault/postman-vault-secrets/)!

[**IronPDF**](https://ironpdf.com/?utm_source=milan&utm_medium=email&utm_campaign=milan):
Your essential C# PDF library for converting modern HTML templates into PDFs effortlessly.
With advanced CSS styling preservation and seamless integration with ASP.NET Core MVC and Razor, IronPDF ensures pixel-perfect results.
[**Try it today**](https://ironpdf.com/?utm_source=milan&utm_medium=email&utm_campaign=milan)!

</Sponsorships>

I'll never forget when I was working on a project that required generating weekly sales reports for a client.
The initial solution involved a clunky process of exporting data, manipulating it in spreadsheets, and manually creating PDFs.
It was tedious, error-prone, and it sucked up way too much of my time.
I knew there had to be a better way.

That's when I discovered the power of combining Razor views with HTML-to-PDF conversion.
You have more control over formatting the document.
You can use modern CSS to style the HTML markup, which will be applied when exporting to a PDF document.
It's also simple to implement in ASP.NET Core.

Here's what we'll cover:

- Understanding Razor views
- Converting Razor views to HTML
- HTML to PDF conversion in .NET
- Putting it all together with Minimal APIs

Let's dive in!

## Razor Views

Razor [views](https://learn.microsoft.com/en-us/aspnet/core/mvc/views)
are an HTML template with embedded [Razor](https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor) markup.
Razor allows you to write and execute .NET code inside a web page.
Views have a special `.cshtml` file extension.
They're commonly used in [ASP.NET Core MVC](https://learn.microsoft.com/en-us/aspnet/core/mvc/overview),
[Razor Pages](https://learn.microsoft.com/en-us/aspnet/core/razor-pages),
and [Blazor](https://learn.microsoft.com/en-us/aspnet/core/blazor).

However, you can define Razor views in a class library or ASP.NET Core Web API project.

You can use the `Model` object to pass in data to a Razor view.
Inside the `.cshtml` file, you'll specify the model type with the `@model` keyword.
In the example below, I'm specifying that the `Invoice` class is the model for this view.
You can access the model instance with the `Model` property in the view.

This is the `InvoiceReport.cshtml` view we'll use to generate a PDF invoice.

You can write CSS in the Razor view inline or reference a stylesheet.
I'm using the [Tailwind CSS](https://tailwindcss.com/) utility framework, which uses inline CSS.
I usually delegate this to a front-end engineer on my team so they can stylize the report as needed.

```csharp {4}
@using System.Globalization
@using HtmlToPdf.Contracts

@model HtmlToPdf.Contracts.Invoice

@{
    IFormatProvider cultureInfo = CultureInfo.CreateSpecificCulture("en-US");
    var subtotal = Model.LineItems.Sum(li => li.Price * li.Quantity).ToString("C", cultureInfo);
    var total = Model.LineItems.Sum(li => li.Price * li.Quantity).ToString("C", cultureInfo);
}

<script src="https://cdn.tailwindcss.com"></script>

<div class="min-w-7xl flex flex-col bg-gray-200 space-y-4 p-10">
    <h1 class="text-2xl font-semibold">Invoice #@Model.Number</h1>

    <p>Issued date: @Model.IssuedDate.ToString("dd/MM/yyyy")</p>
    <p>Due date: @Model.DueDate.ToString("dd/MM/yyyy")</p>

    <div class="flex justify-between space-x-4">
        <div class="bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2">
            <p class="font-medium">Seller:</p>
            <p>@Model.SellerAddress.CompanyName</p>
            <p>@Model.SellerAddress.Street</p>
            <p>@Model.SellerAddress.City</p>
            <p>@Model.SellerAddress.State</p>
            <p>@Model.SellerAddress.Email</p>
        </div>
        <div class="bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2">
            <p class="font-medium">Bill to:</p>
            <p>@Model.CustomerAddress.CompanyName</p>
            <p>@Model.CustomerAddress.Street</p>
            <p>@Model.CustomerAddress.City</p>
            <p>@Model.CustomerAddress.State</p>
            <p>@Model.CustomerAddress.Email</p>
        </div>
    </div>

    <div class="flex flex-col bg-white rounded-lg p-4 space-y-2">
        <h2 class="text-xl font-medium">Items:</h2>
        <div class="">
            <div class="flex space-x-4 font-medium">
                <p class="w-10">#</p>
                <p class="w-52">Name</p>
                <p class="w-20">Price</p>
                <p class="w-20">Quantity</p>
            </div>

            @foreach ((int index, LineItem item) in Model.LineItems.Select((li, i) => (i + 1, li)))
            {
                <div class="flex space-x-4">
                    <p class="w-10">@index</p>
                    <p class="w-52">@item.Name</p>
                    <p class="w-20">@item.Price.ToString("C", cultureInfo)</p>
                    <p class="w-20">@item.Quantity.ToString("N2")</p>
                </div>
            }
        </div>
    </div>

    <div class="flex flex-col items-end bg-gray-50 space-y-2 p-4 rounded-lg">
        <p>Subtotal: @subtotal</p>
        <p>Total: <span class="font-semibold">@total</span></p>
    </div>
</div>
```

## Converting Razor Views to HTML

The next thing we'll need is a way to convert the Razor view into HTML.
We can do this with the [`Razor.Templating.Core`](https://github.com/soundaranbu/Razor.Templating.Core) library.
It provides a simple API to render a `.cshtml` file into a `string`.

```powershell
Install-Package Razor.Templating.Core
```

You can use the `RazorTemplateEngine` static class to call the `RenderAsync` method.
It accepts the path to the Razor view and the model instance that will be passed to the view.

Here's what that will look like:

```csharp
Invoice invoice = invoiceFactory.Create();

string html = await RazorTemplateEngine.RenderAsync(
    "Views/InvoiceReport.cshtml",
    invoice);
```

Alternatively, you can use the `IRazorTemplateEngine` instead of the static class.
In that case, you must call `AddRazorTemplating` to register the required services with DI.
This is also required if you want to use dependency injection inside the Razor views with `@inject`.
It's recommended that you call `AddRazorTemplating` after registering all dependencies.

```csharp
services.AddRazorTemplating();
```

## HTML to PDF conversion

Now that we've converted our Razor view into HTML, we can use it to generate a PDF report.
Many libraries offer this functionality.
The library I've used most often is [IronPDF](https://ironpdf.com/).
It's a paid library (and well worth it), but I know developers also want free options, so I'll list some alternatives at the end.

We can use IronPDF's `ChromePdfRenderer`, which uses an embedded Chrome browser.
The renderer exposes the `RenderHtmlAsPdf` method, which generates a `PdfDocument`.
Once you have the document, you can store it on the file system or export it as binary data.

```csharp
var renderer = new ChromePdfRenderer();

using var pdfDocument = renderer.RenderHtmlAsPdf(html);

pdfDocument.SaveAs($"invoice-{invoice.Number}.pdf");
```

If you're looking for free options, check out [Puppeteer Sharp](https://github.com/hardkoded/puppeteer-sharp).
It's a .NET port of the [Puppeteer](https://github.com/puppeteer/puppeteer) library, which allows you to run a headless Chrome browser.

Another (conditionally) free option to consider is [NReco.PdfGenerator](https://www.nuget.org/packages/NReco.PdfGenerator/).
However, it's only free for single-server deployments.

## Putting It All Together

Let's use everything we discussed to create a Minimal API endpoint to generate an invoice PDF report and return it as a file response.
Here's the code snippet:

```csharp
app.MapGet("invoice-report", async (InvoiceFactory invoiceFactory) =>
{
    Invoice invoice = invoiceFactory.Create();

    var html = await RazorTemplateEngine.RenderAsync(
        "Views/InvoiceReport.cshtml",
        invoice);

    var renderer = new ChromePdfRenderer();

    using var pdfDocument = renderer.RenderHtmlAsPdf(html);

    return Results.File(
        pdfDocument.BinaryData,
        "application/pdf",
        $"invoice-{invoice.Number}.pdf");
});
```

This is what the generated PDF report looks like:

<Image
  src="blogs/mnw_096/invoice.png"
  width={1273}
  height={1937}
  alt="Invoice report."
/>

You can grab the source code for this sample [here](https://github.com/m-jovanovic/razor-view-html-to-pdf).
Feel free to try out a different library for HTML to PDF conversion.

## Summary

In this article, we've explored the power of using Razor views for flexible PDF reporting in .NET.
We've seen how to create report templates with Razor views, convert them to HTML, and then transform that HTML into beautifully formatted PDF documents.

Whether you need to generate invoices, sales reports, or any other kind of structured document, this approach offers a simple and customizable solution.

Here's what you can explore next:

- [PDF Reporting in .NET Using IronPDF and Razor Views](https://youtu.be/XYdcdVWsWos)
- [Creating PDF Reports With QuestPDF](how-to-easily-create-pdf-documents-in-aspnetcore)

That's all for this week. Stay awesome!

---



<Sponsorships>

Free eBook: [**Blazor Hybrid and Web in One Solution**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-ebook).
Dive into Blazor cross-platform apps, potential roadblocks, testing with Razor components, and more.
[**Get the eBook here**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-ebook)!

[**Postman v11**](https://blog.postman.com/announcing-postman-v11/): The future of API development is here.
Collaborate on APIs and supercharge your development workflow with AI.
AI features with Postbot.
Comment improvements in Postman Collections.
Postman Package Library.
[**Check it out here**](https://blog.postman.com/announcing-postman-v11/)!

</Sponsorships>

When you're dealing with thousands or even millions of records, efficiency is king.
That's where [**EF Core bulk update**](how-to-use-the-new-bulk-update-feature-in-ef-core-7) capabilities come into play.

EF Core 7 introduced two powerful new methods, `ExecuteUpdate` and `ExecuteDelete`.
They're designed to simplify bulk updates in your database.
Both methods have their respective async overloads - `ExecuteUpdateAsync` and `ExecuteDeleteAsync`.
[EF bulk updates](https://learn.microsoft.com/en-us/ef/core/saving/execute-insert-update-delete) offer significant performance advantages over traditional approaches.

However, there's an **important caveat**: these bulk operations bypass the EF Core **Change Tracker**.
This disconnect can lead to unexpected behavior if you're not aware of it.

In this week's issue, we'll dive into the details of bulk updates in EF Core.

## Understanding the EF Core ChangeTracker

When you load entities from the database with EF Core, the `ChangeTracker` starts tracking them.
As you update properties, delete entities, or add new ones, the `ChangeTracker` records these changes.

```csharp
using (var context = new AppDbContext())
{
    // Load a product
    var product = context.Products.FirstOrDefault(p => p.Id == 1);
    product.Price = 99.99; // Modify a property

    // At this point, the ChangeTracker knows that 'product' has been modified

    // Add a new product
    var newProduct = new Product { Name = "New Gadget", Price = 129.99 };
    context.Products.Add(newProduct);

    // Delete a product
    context.Products.Remove(product);

    context.SaveChanges(); // Persist all changes to the database
}
```

When you call `SaveChanges`, EF Core uses the `ChangeTracker` to determine which SQL commands to execute.
This ensures that the database is perfectly synchronized with your modifications.
The `ChangeTracker` acts as a bridge between your in-memory object model and your database.

If you're already familiar with how EF Core works, this serves mostly as a reminder.

## Bulk Updates and the ChangeTracker Disconnect

Now, let's focus on how [**bulk updates in EF Core**](how-to-use-the-new-bulk-update-feature-in-ef-core-7)
interact with the `ChangeTracker` - or rather, how they don't interact with it.
This design decision might seem counterintuitive, but there's a solid reason behind it: **performance**.

By directly executing SQL statements against the database, EF Core eliminates the overhead of tracking individual entity modifications.

```csharp
using (var context = new AppDbContext())
{
    // Increase price of all electronics by 10%
    context.Products
        .Where(p => p.Category == "Electronics")
        .ExecuteUpdate(
            s => s.SetProperty(p => p.Price, p => p.Price * 1.10));

    // In-memory Product instances with Category == "Electronics"
    // will STILL have their old price
}
```

In this example, we're increasing the price of all products in the `Electronics` category by 10%.
The `ExecuteUpdate` method efficiently translates the operation into a single SQL `UPDATE` statement.

```sql
UPDATE [p]
SET [p].[Price] = [p].[Price] * 1.10
FROM [Products] as [p];
```

However, if you inspect the `Product` instances that EF Core has already loaded into memory, you'll find that their `Price` properties haven't changed.
This might seem surprising if you aren't aware of how bulk updates interact with the change tracker.

Everything we discussed up to this point also applies to the `ExecuteDelete` method.

[**EF Core interceptors**](how-to-use-ef-core-interceptors) do not trigger for `ExecuteUpdate` and `ExecuteDelete` operations.
If you need to track or modify bulk update operations, you can create database triggers that fire whenever a relevant table is updated or deleted.
This allows you to log details and perform additional actions.

## The Problem: Maintaining Consistency

If `ExecuteUpdate` completes successfully, the changes are directly committed to the database.
This is because bulk operations bypass the `ChangeTracker` and don't participate in the usual transaction managed by `SaveChanges`.

If `SaveChanges` subsequently fails due to an error (e.g., validation error, database constraint violation, connection issue),
you'll be in an inconsistent state.
The changes made by `ExecuteUpdate` are already persisted.
Any changes made "in memory" are lost.

The most reliable way to ensure consistency is to wrap both `ExecuteUpdate` and the operations that lead to `SaveChanges` in a transaction:

```csharp {2}
using (var context = new AppDbContext())
using (var transaction = context.Database.BeginTransaction())
{
    try
    {
        context.Products
            .Where(p => p.Category == "Electronics")
            .ExecuteUpdate(
                s => s.SetProperty(p => p.Price, p => p.Price * 1.10));

        // ... other operations that modify entities

        context.SaveChanges();

        transaction.Commit();
    }
    catch (Exception ex)
    {
        // You could also let the transaction go out of scope.
        // This would automatically rollback any changes.
        transaction.Rollback();

        // Proceed to handle the exception...
    }
}
```

If `SaveChanges` fails, the transaction will be rolled back, reverting the changes made by both `ExecuteUpdate` and any other operations within the transaction.
This keeps your database in a consistent state.

## Summary

EF Core bulk update features, `ExecuteUpdate` and `ExecuteDelete`, are invaluable tools for optimizing performance.
By bypassing the `ChangeTracker` and executing raw SQL directly, they deliver significant speed improvements compared to traditional methods.

However, it's crucial to be mindful of the potential pitfalls associated with this approach.
The disconnect between in-memory entities and the database state can lead to unexpected results if not handled correctly.

My rule of thumb is to create an explicit [**database transaction**](working-with-transactions-in-ef-core) when I want to make additional entity changes.
We can be confident that all the changes will persist in the database or none of them will.

I hope this was helpful, and I'll see you next week.

**P.S.** Get the [source code](https://github.com/m-jovanovic/ef-bulk-updates) and try out the examples from this issue.

---



<Sponsorships>

Supercharge your [**API development with AI**](https://blog.postman.com/announcing-postman-v11/).
Postman v11 introduces many new features for collaboration, AI, and security.
Try out the new features like the VS Code extension, Partner Workspaces, and Postman Vault.
[**Learn more here**](https://blog.postman.com/announcing-postman-v11/).

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240615), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240615)

</Sponsorships>

I once led the development of a fitness tracking app.
We started with Transaction Scripts to handle features like workout creation and exercise logging.
It was a simple and effective approach for the app's early stages.

However, as we added more complex features, our business logic became bloated.
New business rules intertwined with existing logic, making the code difficult to maintain.
Each change could introduce unintended consequences.

We solved our problem by introducing a Domain Model.
The domain model shifts the focus from procedures to domain objects.
Our code became more expressive, easier to reason about, and less prone to errors.

This experience taught me a valuable lesson, that I want to share in this newsletter.

## Transaction Script

At its core, business applications operate through distinct interactions (transactions) with their users.
These transactions can range from simple data retrieval to complex operations involving multiple validations, calculations, and updates to the system's database.

The Transaction Script pattern provides a simple way to encapsulate the logic behind each transaction.
It organizes all the necessary steps, from data access to business rules, into a single, self-contained procedure.

> Organizes business logic by procedures where each procedure handles a single request from the presentation.

_&mdash; [Transaction Script](https://martinfowler.com/eaaCatalog/transactionScript.html), Patterns of Enterprise Application Architecture_

Here's an example of adding exercises to a workout:

```csharp
internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler<AddExercisesCommand>
{
    public async Task<Result> Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        List<Error> errors = [];
        foreach (ExerciseRequest exerciseDto in request.Exercises)
        {
            if (exerciseDto.TargetType == TargetType.Distance &&
                exerciseDto.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseDto.TargetType == TargetType.Time &&
                exerciseDto.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
```

There isn't much logic here. We're just checking whether the workout exists and whether the exercises are valid.

What happens when we start to add more logic?

Let's add another business rule.
We must enforce a limit on the number of exercises allowed in a single workout (e.g., no more than 10 exercises).

```csharp {47-52}
internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler<AddExercisesCommand>
{
    public async Task<Result> Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        List<Error> errors = [];
        foreach (ExerciseRequest exerciseDto in request.Exercises)
        {
            if (exerciseDto.TargetType == TargetType.Distance &&
                exerciseDto.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseDto.TargetType == TargetType.Time &&
                exerciseDto.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);

            if (workouts.Exercise.Count > 10)
            {
                return Result.Failure(
                    WorkoutErrors.MaxExercisesReached(workout.Id));
            }
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
```

We can continue adding more business logic to the transaction script.
For example, we can introduce exercise type restrictions or enforce a specific exercise order.
You can imagine how the complexity will keep increasing over time.

Another concern is code duplication between transaction scripts.
This could happen if we need similar logic in multiple transaction scripts.
You may be tempted to solve this by calling one transaction script from the other, but this will introduce a different set of problems.

So, how can we solve these problems?

## Refactoring to Domain Model

What is a domain model?

> An object model of the domain that incorporates both behavior and data.

_&mdash; [Domain Model](https://martinfowler.com/eaaCatalog/domainModel.html), Patterns of Enterprise Application Architecture_

A domain model lets you encapsulate domain logic (behavior) and state changes (data) inside an object.
In Domain-Driven Design terminology, we would call this an aggregate.

An aggregate in DDD is a cluster of objects treated as a single unit for data changes.
The aggregate represents a consistency boundary.
It helps maintain consistency by ensuring that certain invariants always hold true for the entire aggregate.
In our workout example, the `Workout` class can be treated as an aggregate root that encompasses all the exercises within it.

What does this have to do with a transaction script?

We can move the domain logic and state changes from the transaction script into the aggregate.
This is often called "pushing logic down" into the domain.

Here's what the domain model will look like when we extract the domain logic:

```csharp
public sealed class Workout
{
    private readonly List<Exercise> _exercises = [];

    // Omitting the constructor and other propreties for brevity.

    public Result AddExercises(ExerciseModel[] exercises)
    {
        List<Error> errors = [];
        foreach (var exerciseModel in exercises)
        {
            if (exerciseModel.TargetType == TargetType.Distance &&
                exerciseModel.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseModel.TargetType == TargetType.Time &&
                exerciseModel.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);

            if (workouts.Exercise.Count > 10)
            {
                return Result.Failure(
                    WorkoutErrors.MaxExercisesReached(workout.Id));
            }
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        return Result.Success();
    }
}
```

With the domain logic inside of the domain model, we can easily share it between transaction scripts.
Testing the domain model is simpler than testing the transaction script.
With a transaction script, we must provide any dependencies (possibly as mocks) for testing.
However, we can test the domain model in isolation.

The updated transaction script becomes much more straightforward and focused on its primary task:

```csharp
internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler<AddExercisesCommand>
{
    public async Task<Result> Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        var exercises = request.Exercises.Select(e => e.ToModel()).ToArray();

        var result = workout.AddExercises(exercises);

        if (result.IsFailure)
        {
            return result;
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
```

## Takeaway

Transaction Scripts are a practical starting point for simple applications.
They offer a straightforward approach to implementing use cases.
Transaction scripts are the recommended approach to start building [vertical slices](vertical-slice-architecture-structuring-vertical-slices).
However, transaction scripts can become difficult to maintain as the application grows.

[Refactoring toward a Domain Model](refactoring-from-an-anemic-domain-model-to-a-rich-domain-model) allows you to encapsulate business logic in domain objects.
This promotes code reusability and makes your application more adaptable to changes.
Pushing logic down also improves testability and maintainability.

Should you use a Transaction Script or a Domain Model?

Here's a pragmatic approach you should consider.
Start with a transaction script, but pay attention to growing complexity.
When you notice a transaction script has too many concerns, consider adding a domain model.
Remember, the domain model should encapsulate some of the complexity of the domain logic.

Thanks for reading, and I'll see you next week!

---



<Sponsorships>

[**dotConnect**](https://www.devart.com/dotconnect/?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dotconnect-general) is a feature-rich ADO.NET provider for developing data-related .NET applications.
It's fast, reliable, and works for all major databases and cloud services.
dotConnect is compatible with all Entity Framework and EF Core versions.
[**Start FREE trial today**](https://www.devart.com/dotconnect/?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dotconnect-general)!

Use [**Postman V11 collaboration features**](https://learning.postman.com/docs/collaborating-in-postman/collaborate-in-postman-overview/)
to collaborate with your team and API Consumers.
Features include changing your workspace visibility, assigning roles to collaborators, sharing workspaces, and using Postman's API version control.
[**Learn more here**](https://learning.postman.com/docs/collaborating-in-postman/collaborate-in-postman-overview/)!

</Sponsorships>

Caching is one of the simplest techniques to significantly improve your application's performance.
It's the process of temporarily storing data in a faster access location.
You will typically cache the results of expensive operations or frequently accessed data.

Caching allows subsequent requests for the same data to be served from the cache instead of fetching the data from its source.

ASP.NET Core offers several types of caches, such as `IMemoryCache`, `IDistributedCache`, and the upcoming `HybridCache` (.NET 9).

In this newsletter, we will explore how to implement [caching in ASP.NET Core](https://learn.microsoft.com/en-us/aspnet/core/performance/caching/memory) applications.

## How Caching Improves Application Performance

Caching improves your application's performance by reducing latency and server load while enhancing scalability and user experience.

- **Faster data retrieval**: Cached data can be accessed much faster than retrieving it from the source (like a database or an API).
  Caches are typically stored in memory (RAM).
- **Fewer database queries**: Caching frequently accessed data reduces the number of database queries.
  This reduces the load on the database server.
- **Lower CPU usage**: Rendering web pages or processing API responses can consume significant CPU resources.
  Caching the results reduces the need for repetitive CPU-intensive tasks.
- **Handling increased traffic**: By reducing the load on backend systems, caching allows your application to handle
  more concurrent users and requests.
- **Distributed caching**: Distributed cache solutions like [Redis](https://redis.io/) enable scaling the cache across multiple servers,
  further improving performance and resilience.

In a recent project I worked on, we used Redis to scale to more than 1,000,000 users.
We only had one SQL Server instance with a read-replica for reporting.
The power of caching, eh?

## Caching Abstractions in ASP.NET Core

ASP.NET Core provides two primary abstractions for working with caches:

- `IMemoryCache`: Stores data in the memory of the web server.
  Simple to use but not suitable for distributed scenarios.
- `IDistributedCache`: Offers a more robust solution for distributed applications.
  It allows you to store cached data in a distributed cache like Redis.

We have to register these services with DI to use them.
`AddDistributedMemoryCache` will configure the in-memory implementation of `IDistributedCache`, which isn't distributed.

```csharp
builder.Services.AddMemoryCache();

builder.Services.AddDistributedMemoryCache();
```

Here's how you can use the `IMemoryCache`.
We will first check if the cached value is present and return it directly if it's there.
Otherwise, we must fetch the value from the database and cache it for subsequent requests.

```csharp
app.MapGet(
    "products/{id}",
    (int id, IMemoryCache cache, AppDbContext context) =>
    {
        if (!cache.TryGetValue(id, out Product product))
        {
            product = context.Products.Find(id);

            var cacheEntryOptions = new MemoryCacheEntryOptions()
                .SetAbsoluteExpiration(TimeSpan.FromMinutes(10))
                .SetSlidingExpiration(TimeSpan.FromMinutes(2));

            cache.Set(id, product, cacheEntryOptions);
        }

        return Results.Ok(product);
    });
```

Cache expiration is another important topic to discuss.
We want to remove cache entries that aren't used and become stale.
You can pass in the `MemoryCacheEntryOptions`, allowing you to configure cache expiration.
For example, we can set the `AbsoluteExpiration` and `SlidingExpiration` values to control when the cache entry will expire.

## Cache-Aside Pattern

The cache-aside pattern is the most common caching strategy. Here's how it works:

1. **Check the cache**: Look for the requested data in the cache.
2. **Fetch from source (if cache miss)**: If the data isn't in the cache, fetch it from the source.
3. **Update the cache**: Store the fetched data in the cache for subsequent requests.

<Image
  src="blogs/mnw_093/cache_aside.png"
  width={1301}
  height={826}
  alt="Cache-aside pattern."
/>

Here's how you can implement the cache-aside pattern as an extension method for `IDistributedCache`:

```csharp
public static class DistributedCacheExtensions
{
    public static DistributedCacheEntryOptions DefaultExpiration => new()
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(2)
    };

    public static async Task<T> GetOrCreateAsync<T>(
        this IDistributedCache cache,
        string key,
        Func<Task<T>> factory,
        DistributedCacheEntryOptions? cacheOptions = null)
    {
        var cachedData = await cache.GetStringAsync(key);

        if (cachedData is not null)
        {
            return JsonSerializer.Deserialize<T>(cachedData);
        }

        var data = await factory();

        await cache.SetStringAsync(
            key,
            JsonSerializer.Serialize(data),
            cacheOptions ?? DefaultExpiration);

        return data;
    }
}
```

We're using `JsonSerializer` to manage serialization to and from a JSON string.
The `SetStringAsync` method also accepts a `DistributedCacheEntryOptions` argument to control cache expiration.

Here's how we would use this extension method:

```csharp
app.MapGet(
    "products/{id}",
    (int id, IDistributedCache cache, AppDbContext context) =>
    {
        var product = cache.GetOrCreateAsync($"products-{id}", async () =>
        {
            var productFromDb = await context.Products.FindAsync(id);

            return productFromDb;
        });

        return Results.Ok(product);
    });
```

## Pros and Cons of In-Memory Caching

Pros:

- Extremely fast
- Simple to implement
- No external dependencies

Cons:

- Cache data is lost if the server restarts
- Limited to the memory (RAM) of a single server
- Cache data is not shared across multiple instances of your application

## Distributed Caching With Redis

[Redis](https://redis.io/) is a popular in-memory data store often used as a high-performance distributed cache.
To use Redis in your ASP.NET Core application, you can use the `StackExchange.Redis` library.

However, there's also the `Microsoft.Extensions.Caching.StackExchangeRedis` library,
allowing you to integrate Redis with `IDistributedCache`.

```powershell
Install-Package Microsoft.Extensions.Caching.StackExchangeRedis
```

Here's how you can configure it with DI by providing a connection string to Redis:

```csharp
string connectionString = builder.Configuration.GetConnectionString("Redis");

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = connectionString;
});
```

An alternative approach is to register an `IConnectionMultiplexer` as a service.
Then, we will use it to provide a function for the `ConnectionMultiplexerFactory`.

```csharp
string connectionString = builder.Configuration.GetConnectionString("Redis");

IConnectionMultiplexer connectionMultiplexer =
    ConnectionMultiplexer.Connect(connectionString);

builder.Services.AddSingleton(connectionMultiplexer);

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.ConnectionMultiplexerFactory =
        () => Task.FromResult(connectionMultiplexer);
});
```

Now, when you inject `IDistributedCache`, it will use Redis under the hood.

## Cache Stampede and HybridCache

The in-memory cache implementations in ASP.NET Core are susceptible to race conditions, which can cause a cache stampede.
A [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede) happens when concurrent requests encounter a cache miss and try to fetch the data from the source.
This can overload your application and negate the benefits of caching.

Locking is one solution for the cache stampede problem.
.NET offers many options for [locking and concurrency control](introduction-to-locking-and-concurrency-control-in-dotnet-6).
The most commonly used locking primitives are the `lock` statement and the `Semaphore` (or `SemaphoreSlim`) class.

Here's how we could use `SemaphoreSlim` to introduce locking before fetching data:

```csharp {3,13,24}
public static class DistributedCacheExtensions
{
    private static readonly SemaphoreSlim Semaphore = new SemaphoreSlim(1, 1);

    // Arguments omitted for brevity
    public static async Task<T> GetOrCreateAsync<T>(...)
    {
        // Fetch data from cache, and return if present

        // Cache miss
        try
        {
            await Semaphore.WaitAsync();

            var data = await factory();

            await cache.SetStringAsync(
                key,
                JsonSerializer.Serialize(data),
                cacheOptions ?? DefaultExpiration);
        }
        finally
        {
            Semaphore.Release();
        }

        return data;
    }
}
```

The previous implementation has a lock contention issue since all requests have to wait for the semaphore.
A much better solution would be locking based on the `key` value.

.NET 9 introduces a new caching abstraction called `HybridCache`, which aims to solve the shortcomings of `IDistributedCache`.
Learn more about this in the [Hybrid cache documentation](https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid).

## Summary

Caching is a powerful technique for improving web application performance.
ASP.NET Core's caching abstractions make it easy to implement various caching strategies.

We can choose from `IMemoryCache` for in-memory cache and `IDistributedCache` for distributed caching.

Here are a few guidelines to wrap up this week's issue:

- Use `IMemoryCache` for simple, in-memory caching
- Implement the cache aside pattern to minimize database hits
- Consider Redis as a high-performance distributed cache implementation
- Use `IDistributedCache` for sharing cached data across multiple applications

That's all for today.

See you next week.

---



<Sponsorships>

APIs are the Future.
Learn more about the [**API Lifecycle and API Documentation**](https://www.postman.com/api-platform/api-documentation/) by checking out this resource
as part of an introduction to the Postman API Platform.
[**Check it out here**](https://www.postman.com/api-platform/api-documentation/).

[**Product for Engineers**](https://newsletter.posthog.com/) from PostHog is a newsletter helping engineers improve their product skills.
[**Subscribe for free**](https://newsletter.posthog.com/) to get curated advice on building great products, lessons (and mistakes) from PostHog, and deep dives on top startups.

</Sponsorships>

Are you tired of organizing your project across layers?

Vertical Slice Architecture is a compelling alternative to traditional layered architectures.
[VSA](vertical-slice-architecture) flips the script on how we structure code.

Instead of horizontal layers (Presentation, Application, Domain), VSA organizes code by feature.
Each feature encompasses everything it needs, from API endpoints to data access.

In this newsletter, we will explore how you can structure vertical slices in VSA.

## Understanding Vertical Slices

At its core, a vertical slice represents a self-contained unit of functionality.
It's a slice through the entire application stack.
It encapsulates all the code and components necessary to fulfill a specific feature.

In traditional layered architectures, code is organized horizontally across the various layers.
One feature implementation can be scattered across many layers.
Changing a feature requires modifying the code in multiple layers.

VSA addresses this by grouping all the code for a feature into a single slice.

This shift in perspective brings several advantages:

- **Improved cohesion**: Code related to a specific feature resides together, making it easier to understand, modify, and test.
- **Reduced complexity**: VSA simplifies your application's mental model by avoiding the need to navigate multiple layers.
- **Focus on business logic**: The structure naturally emphasizes the business use case over technical implementation details.
- **Easier maintenance**: Changes to a feature are localized within its slice, reducing the risk of unintended side effects.

<Image
  src="blogs/mnw_092/vertical_slices.png"
  width={2159}
  height={1276}
  alt="Vertical slices."
/>

## Implementing Vertical Slice Architecture

Here's an example vertical slice representing the `CreateProduct` feature.
We use a static class to represent the feature and group the related types.
Each feature can have a respective `Request` and `Response` class.
The use case with the business logic can be in a [Minimal API endpoint](automatically-register-minimal-apis-in-aspnetcore).

A vertical slice is likely either a command or a query.
This approach gives us [CQRS](cqrs-pattern-with-mediatr) out of the box.

```csharp
public static class CreateProduct
{
    public record Request(string Name, decimal Price);
    public record Response(int Id, string Name, decimal Price);

    public class Endpoint : IEndpoint
    {
        public void MapEndpoint(IEndpointRouteBuilder app)
        {
            app.MapPost("products", Handler).WithTags("Products");
        }

        public static IResult Handler(Request request, AppDbContext context)
        {
            var product = new Product
            {
                Name = request.Name,
                Price = request.Price
            };

            context.Products.Add(product);

            context.SaveChanges();

            return Results.Ok(
                new Response(product.Id, product.Name, product.Price));
        }
    }
}
```

I want to mention a few benefits of structuring your application like this.

The code for the entire `CreateProduct` feature is tightly grouped within a single file.
This makes it extremely easy to locate, understand, and modify everything related to this functionality.
We don't need to navigate multiple layers (like controllers, services, repositories, etc.).

Directly using `AppDbContext` within the endpoint might tightly couple the slice to your database technology.
Depending on your project's size and requirements, you could consider abstracting data access (using a repository pattern)
to make the slice more adaptable to changes in the persistence layer.

## Introducing Validation in Vertical Slices

Vertical slices usually need to solve some [cross-cutting concerns](balancing-cross-cutting-concerns-in-clean-architecture), one of which is validation.
Validation is the gatekeeper, preventing invalid or malicious data from entering your system.
We can easily implement [validation with the FluentValidation library](cqrs-validation-with-mediatr-pipeline-and-fluentvalidation).

Within your slice, you'd define a `Validator` class that encapsulates the rules specific to your feature's request model.
It also supports dependency injection, so we can run complex validations here.

```csharp
public class Validator : AbstractValidator<Request>
{
    public Validator()
    {
        RuleFor(x => x.Name).NotEmpty().MaximumLength(100);
        RuleFor(x => x.Price).GreaterThanOrEqualTo(0);
        // ... other rules
    }
}
```

This validator can then be injected into your endpoint using dependency injection, allowing you to perform validation before processing the request.

```csharp {6,20-24}
public static class CreateProduct
{
    public record Request(string Name, decimal Price);
    public record Response(int Id, string Name, decimal Price);

    public class Validator : AbstractValidator<Request> // { ... }

    public class Endpoint : IEndpoint
    {
        public void MapEndpoint(IEndpointRouteBuilder app)
        {
            app.MapPost("products", Handler).WithTags("Products");
        }

        public static IResult Handler(
            Request request,
            IValidator<Request> validator,
            AppDbContext context)
        {
            var validationResult = await validator.Validate(request);
            if (!validationResult.IsValid)
            {
                return Results.BadRequest(validationResult.Errors);
            }

            // ... (Create product and return response)
        }
    }
}
```

## Handling Complex Features and Shared Logic

The previous examples were simple.
But what do we do with complex features and shared logic?

VSA excels at managing self-contained features.
However, real-world applications often involve complex interactions and shared logic.

Here are a few strategies you can consider to address this:

- **Decomposition**: Break down complex features into smaller, more manageable vertical slices.
  Each slice should represent a cohesive piece of the overall feature.
- **Refactoring**: When a vertical slice becomes difficult to maintain, you can apply some refactoring techniques.
  The most common ones I use are `Extract method ` and `Extract class`.
- **Extract shared logic**: Identify common logic that's used across multiple features.
  Create a separate class (or extension method) to reference it from your vertical slices as needed.
- **Push logic down**: Write vertical slices using procedural code, like a [Transaction Script](https://martinfowler.com/eaaCatalog/transactionScript.html).
  Then, you can identify parts of the business logic that naturally belong to the domain entities.

You and your team will need to understand code smells and refactorings to make the most of VSA.

## Summary

Vertical Slice Architecture is more than just a way to structure your code.
By focusing on features, VSA allows you to create cohesive and maintainable applications.
Vertical slices are self-contained, making unit and integration testing more straightforward.

VSA brings benefits in terms of code organization and development speed, making it a valuable tool in your toolbox.
Code is grouped by feature, making it easier to locate and understand.
The structure aligns with the way business users think about features.
Changes are localized, reducing the risk of regressions and enabling faster iterations.

Consider embracing [Vertical Slice Architecture](vertical-slice-architecture) in your next project.
It's a big mindset shift from [Clean Architecture](/pragmatic-clean-architecture).
However, they both have their place and even share similar ideas.

That's all for this week. Stay awesome!

---



<Sponsorships>

Access the largest network of public APIs on the planet with the [**Postman Public API Network**](https://learning.postman.com/docs/collaborating-in-postman/public-api-network/verify-your-team/)
and [**Verified Teams**](https://learning.postman.com/docs/collaborating-in-postman/public-api-network/verify-your-team/).
Find public APIs faster and expand your API reach.
Those who have verified have seen API traffic increase by as much as 100%! [**Try it out today**](https://learning.postman.com/docs/collaborating-in-postman/public-api-network/verify-your-team/).

Looking to build interactive [**Blazor apps with WebAssembly**](https://www.telerik.com/blogs/building-interactive-blazor-apps-webassembly?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)?
Explore the latest features introduced in .NET 8, dive into the mechanics of Blazor WebAssembly, and create your first Blazor interactive WASM app.
[**Get started here**](https://www.telerik.com/blogs/building-interactive-blazor-apps-webassembly?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)!

</Sponsorships>

Picture this: You're part of a team building a shiny new .NET application.
You've carefully chosen your software architecture.
It could be microservices, a [**modular monolith**](/modular-monolith-architecture), or something else entirely.
You've decided which database you will use and all the other tools you need.
Everyone's excited, the code is flowing, and features are getting shipped.

Fast forward a few months (or years), and things might look different.

The codebase has grown, and new features have been added.
Maybe your team has even changed, with new developers coming on board.
Adding new features becomes a pain, and bugs are popping up left and right.

And slowly but surely, the neat architecture you started with has turned into a [big ball of mud](https://deviq.com/antipatterns/big-ball-of-mud).
What went wrong? And more importantly, what can we do about it?

Today, I want to show you how architecture testing can prevent this problem.

## Technical Debt

Technical debt is the consequence of prioritizing development speed over well-designed code.
It happens when teams cut corners to meet deadlines, make quick fixes, or don't understand the architecture clearly.

Each shortcut or hack adds to the pile, making the code harder to understand, change, and maintain.
But why do developers take these shortcuts in the first place?

Don't developers care about keeping the code clean?

Well, the truth is, most developers do care.
If you're reading this newsletter, odds are you also care.
But, developers are often under pressure to deliver features quickly.
Sometimes, the quickest way to do that is to take a shortcut.

Plus, not everyone has a deep understanding of software architecture, or they might disagree on what the "right" architecture is.
And let's be honest: some developers want to get their code working and move on to the next thing.

## Architecture Testing

Luckily, there's a way to enforce software architecture on your project before things get out of hand.
It's called [**architecture testing**](enforcing-software-architecture-with-architecture-tests).
These are automated tests that check whether your code follows the architectural rules you've set up.

With architecture testing, you can ["shift left"](https://en.wikipedia.org/wiki/Shift-left_testing).
This enables you to find and fix problems early in the development process when they're much easier and cheaper to deal with.

Think of it like a safety net for your software architecture and design rules.
If someone accidentally breaks a rule, the test will catch it and alert you.
Bonus points if you integrate architecture testing into your [**CI pipeline**](how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet).

There are a few libraries you can use for architecture testing.
I prefer working with the [NetArchTest](https://github.com/BenMorris/NetArchTest) library, which I'll use for the examples.

You can check out this article to learn the [**fundamentals of architecture testing**](enforcing-software-architecture-with-architecture-tests).

Let's see how to write some architecture tests.

## Architecture Testing: Modular Monolith

You built an application using the [**modular monolith architecture**](what-is-a-modular-monolith).
But how can you maintain the constraints between the modules?

- Modules aren't allowed to reference each other
- Modules can only call the public API of other modules

Here's an architecture test that enforces these module constraints.
The `Ticketing` module is not allowed to reference the other modules directly.
However, it can reference the public API of other modules (integration events in this example).
The entry point is the `Types` class, which exposes a fluent API to build the rules you want to enforce.
NetArchTest allows us to enforce the direction of dependencies between modules.

```csharp
[Fact]
public void TicketingModule_ShouldNotHaveDependencyOn_AnyOtherModule()
{
    string[] otherModules = [
        UsersNamespace,
        EventsNamespace,
        AttendanceNamespace];

    string[] integrationEventsModules = [
        UsersIntegrationEventsNamespace,
        EventsIntegrationEventsNamespace,
        AttendanceIntegrationEventsNamespace];

    List<Assembly> ticketingAssemblies =
    [
        typeof(Order).Assembly,
        Modules.Ticketing.Application.AssemblyReference.Assembly,
        Modules.Ticketing.Presentation.AssemblyReference.Assembly,
        typeof(TicketingModule).Assembly
    ];

    Types.InAssemblies(ticketingAssemblies)
        .That()
        .DoNotHaveDependencyOnAny(integrationEventsModules)
        .Should()
        .NotHaveDependencyOnAny(otherModules)
        .GetResult()
        .ShouldBeSuccessful();
}
```

If you want to learn how to build robust and scalable systems using this architectural approach, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).

## Architecture Testing: Clean Architecture

We can also write architecture tests for [**Clean Architecture**](why-clean-architecture-is-great-for-complex-projects).
The inner layers aren't allowed to reference the outer layers.
Instead, the inner layers define abstractions and the outer layers implement these abstractions.

For example, the `Domain` layer isn't allowed to reference the `Application` layer.
Here's an architecture test enforcing this rule:

```csharp
[Fact]
public void DomainLayer_ShouldNotHaveDependencyOn_ApplicationLayer()
{
    Types.InAssembly(DomainAssembly)
        .Should()
        .NotHaveDependencyOn(ApplicationAssembly.GetName().Name)
        .GetResult()
        .ShouldBeSuccessful();
}
```

It's also simple introduce a rule that the `Application` layer isn't allowed to reference the `Infrastructure` layer.
The architecture test will fail whenever someone in the team breaks the dependency rule.

```csharp
[Fact]
public void ApplicationLayer_ShouldNotHaveDependencyOn_InfrastructureLayer()
{
    Types.InAssembly(ApplicationAssembly)
        .Should()
        .NotHaveDependencyOn(InfrastructureAssembly.GetName().Name)
        .GetResult()
        .ShouldBeSuccessful();
}
```

We can introduce more architecture tests for the `Infrastructure` and `Presentation` layers, if needed.

Ready to learn more about building production-ready applications using this architectural approach?
You should check out [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture).

## Architecture Testing: Design Rules

Architecture testing is also useful for enforcing design rules in your code.
If your team has coding standards everyone should follow, architecture testing can help you enforce them.

For example, we want to ensure that all domain events are sealed types.
You can use the `BeSealed` method to enforce a design rule that types implementing `IDomainEvent` or `DomainEvent` should be sealed.

```csharp
[Fact]
public void DomainEvents_Should_BeSealed()
{
    Types.InAssembly(DomainAssembly)
        .That()
        .ImplementInterface(typeof(IDomainEvent))
        .Or()
        .Inherit(typeof(DomainEvent))
        .Should()
        .BeSealed()
        .GetResult()
        .ShouldBeSuccessful();
}
```

An interesting design rule could be requiring all domain entities not to have a public constructor.
Instead, you would create an `Entity` instance through a static factory method.
This approach improves the encapsulation of your `Entity`.

Here's an architecture test enforcing this design rule:

```csharp
[Fact]
public void Entities_ShouldOnlyHave_PrivateConstructors()
{
    IEnumerable<Type> entityTypes = Types.InAssembly(DomainAssembly)
        .That()
        .Inherit(typeof(Entity))
        .GetTypes();

    var failingTypes = new List<Type>();
    foreach (Type entityType in entityTypes)
    {
        ConstructorInfo[] constructors = entityType
            .GetConstructors(BindingFlags.Public | BindingFlags.Instance);

        if (constructors.Any())
        {
            failingTypes.Add(entityType);
        }
    }

    failingTypes.Should().BeEmpty();
}
```

Another thing you can do with architecture tests is enforce naming conventions in your code.
Here's an example of requiring all command handlers to have a name ending with `CommandHandler`:

```csharp
[Fact]
public void CommandHandler_ShouldHave_NameEndingWith_CommandHandler()
{
    Types.InAssembly(ApplicationAssembly)
        .That()
        .ImplementInterface(typeof(ICommandHandler<>))
        .Or()
        .ImplementInterface(typeof(ICommandHandler<,>))
        .Should()
        .HaveNameEndingWith("CommandHandler")
        .GetResult()
        .ShouldBeSuccessful();
}
```

## Summary

Even the most well-planned software projects decay because of technical debt.
Most developers have good intentions.
However, time pressure, misunderstandings, and resistance to rules all contribute to this problem.

[**Architecture testing**](enforcing-software-architecture-with-architecture-tests) acts as a safeguard.
It prevents your codebase from turning into a big ball of mud.
By catching architectural violations early on, you can shift left.
Short feedback loops avoid costly rework and improve developer productivity.
It also ensures the long-term health of your project.

A few key takeaways:

- **Technical debt is inevitable**: It slows down development, introduces bugs, and frustrates developers.
- **Architecture testing is your safety net**: It helps you catch architectural violations before they become problematic.
- **Start small and iterate**: You don't have to test everything at once. Focus on the most critical rules first.
- **Make it part of your workflow**: Integrate architecture tests into your CI/CD pipeline so they run automatically.

**Action point**: Start by exploring popular .NET architecture testing libraries like [ArchUnitNET](https://github.com/BenMorris/NetArchTest)
or [NetArchTest](https://github.com/TNG/ArchUnitNET).
Experiment with writing tests for common architectural rules and gradually integrate them into your development workflow.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240518), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240518)

[**Collaborate directly with your external partners**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/) in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/)
to help your partners consume your team's APIs and work together on API projects.
[Learn more here](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/).

</Sponsorships>

Managing database schemas as your applications grow can quickly become a headache.
Manual changes are error-prone and time-consuming.
This can easily lead to inconsistencies between development and production environments.
I've seen these issues firsthand on countless projects, and it's not pretty.
How can we do better?

Enter Entity Framework (EF) Migrations, a powerful tool that lets you version your database schemas.

Imagine this: Instead of writing SQL scripts, you define your changes in code.
Need to add a column?
Rename a table?
No problem - EF Migrations has you covered.
It tracks every modification to the data model.
You can review, test, and apply changes confidently, even across different environments.

In this newsletter, we'll break down the essentials of EF Migrations:

- **Creating Migrations**: Defining and generating migrations that capture your schema changes.
- **Migration SQL Scripts**: Understanding the SQL generated by your migrations and how to use it.
- **Applying Migrations**: Different ways to apply migrations to your database.
- **Migration Tools**: Exploring additional tools and frameworks for managing database migration.
- **EF Migration Best Practices:**: I'll share my recommendations from using EF for years.

We have many examples to cover, so let's dive in.

## Creating Migrations

If you're completely new to EF migrations, I recommend checking out the [EF migrations docs](https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations)
to grasp the fundamentals.
Moving forward, I'll assume you have some prior knowledge of EF Core.

We'll need an entity and a database context before we can create migrations with EF.

Let's define a simple `Product` entity:

```csharp
public class Product
{
    public int Id { get; set; }

    public string Name { get; set; } = string.Empty;

    public string? Description { get; set; }

    public decimal Price { get; set; }
}
```

We will also need a `DbContext` implementation, so let's define the `AppDbContext` class.
In the `OnModelCreating` method, we're going to configure the `Product` entity.

```csharp
public class AppDbContext : DbContext
{
    public AppDbContext(DbContextOptions<AppDbContext> options)
        : base(options)
    {
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Product>(builder =>
        {
            builder.ToTable("Products", tableBuilder =>
            {
                tableBuilder.HasCheckConstraint(
                    "CK_Price_NotNegative",
                    sql: $"{nameof(Product.Price)} > 0");
            });

            builder.HasKey(p => p.Id);

            builder.Property(p => p.Name).HasMaxLength(100);

            builder.Property(p => p.Description).HasMaxLength(1000);

            builder.Property(p => p.Price).HasPrecision(18, 2);

            builder.HasIndex(p => p.Name).IsUnique();
        });
    }
}
```

Let's break down a few of the methods we're using:

- `ToTable` - Configures the table name for the specific entity.
  It also allows us to provide `TableBuilder` delegate.
  We can use it to configure a check constraint using `HasCheckConstraint`.
- `HasKey` - Configures the table's primary key.
  EF will also pick up the `Id` property by convention, so this step is optional.
- `HasProperty` - Represents the entry point for configuring individual properties of the entity.
- `HasIndex` - Defines an index on the specified property (or properties).
  We can also declare that the index should be unique by calling `IsUnique`.

We're now ready to create our first migration.
I'm going to use the PowerShell syntax:

```
Add-Migration Create_Database
```

This will create the first database migration called `Create_Database`.
The migration will apply the configuration we defined in the `OnModelCreating` method.
It contains the `Up` and `Down` methods, allowing us to apply or revert changes to the database.

Note that some operations are destructive (like removing a column) and can't be easily reverted.
It's up to you to examine the generated migration and prevent any possible data loss.

```csharp
using Microsoft.EntityFrameworkCore.Migrations;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;

public partial class Create_Database : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.CreateTable(
            name: "Products",
            columns: table => new
            {
                Id = table.Column<int>(type: "integer", nullable: false)
                    .Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
                Name = table.Column<string>(type: "character varying(100)", maxLength: 100, nullable: false),
                Description = table.Column<string>(type: "character varying(1000)", maxLength: 1000, nullable: true),
                Price = table.Column<decimal>(type: "numeric(18,2)", precision: 18, scale: 2, nullable: false)
            },
            constraints: table =>
            {
                table.PrimaryKey("PK_Products", x => x.Id);
                table.CheckConstraint("CK_Price_NotNegative", "Price > 0");
            });

        migrationBuilder.CreateIndex(
            name: "IX_Products_Name",
            table: "Products",
            column: "Name",
            unique: true);
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.DropTable(
            name: "Products");
    }
}
```

## Customizing Migrations

We can also modify the migration files if we need to apply some custom changes.

A notable example is renaming a column.
Let's say we rename the `Description` property to `ShortDescription`.
In some EF versions, this would result in the following migration:

```csharp
migrationBuilder.DropColumn(
    name: "Description",
    table: "Customers");

migrationBuilder.AddColumn<string>(
    name: "ShortDescription",
    table: "Products",
    nullable: true);
```

What's the problem here?
By calling `DropColumn` first, we will remove the column from the database and lose valuable data.

What we actually want to do is rename the existing column.
So, we can modify the migration file to use the `RenameColumn` method:

```csharp
migrationBuilder.RenameColumn(
    name: "Description",
    table: "Products",
    newName: "ShortDescription");
```

Another example is executing custom SQL commands from your migrations.
Custom SQL commands are helpful when we can't express something through the EF fluent API.
I've used it in the past to migrate data from one column to another or define a complex index.

```csharp
public partial class Update_Products : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.Sql("<YOUR CUSTOM SQL HERE>");
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        // You are also responsible for reverthing any changes.
    }
}
```

## Migration SQL Scripts

You can use the `Script-Migration` command to generate SQL scripts from your migrations.
This is useful for reviewing changes before applying them to the database.
SQL scripts allow us to execute migrations in environments without direct access to the EF tooling.

Remember, you are responsible for preventing any data loss when executing EF migrations.
Review the migrations carefully before applying them to the database.

Here are a few ways you can execute the `Script-Migration` command:

```
Script-Migration

Script-Migration <FromMigration>

Script-Migration <FromMigration> <ToMigration>
```

The `<FromMigration>` argument should be the name of the last migration applied to the database.
It's your responsibility to apply the script appropriately, and only to databases in the correct migration state.

Here's what the SQL script for the `Create_Database` migration looks like:

```sql
CREATE TABLE IF NOT EXISTS "__EFMigrationsHistory" (
    "MigrationId" character varying(150) NOT NULL,
    "ProductVersion" character varying(32) NOT NULL,
    CONSTRAINT "PK___EFMigrationsHistory" PRIMARY KEY ("MigrationId")
);

START TRANSACTION;

CREATE TABLE "Products" (
    "Id" integer GENERATED BY DEFAULT AS IDENTITY,
    "Name" character varying(100) NOT NULL,
    "Description" character varying(1000),
    "Price" numeric(18,2) NOT NULL,
    CONSTRAINT "PK_Products" PRIMARY KEY ("Id"),
    CONSTRAINT "CK_Price_NotNegative" CHECK (Price > 0)
);

CREATE UNIQUE INDEX "IX_Products_Name" ON "Products" ("Name");

INSERT INTO "__EFMigrationsHistory" ("MigrationId", "ProductVersion")
VALUES ('20240516095344_Create_Database', '8.0.5');

COMMIT;
```

You can also specify an `-Idempotent` argument to the `Script-Migration` command.
The `Script-Migration` command will generate SQL scripts that only apply migrations that haven't been applied already.
This is useful if you're not sure what the last migration applied to the database.

```
Script-Migration -Idempotent
```

Here's what the idempotent SQL script looks like:

```sql
CREATE TABLE IF NOT EXISTS "__EFMigrationsHistory" (
    "MigrationId" character varying(150) NOT NULL,
    "ProductVersion" character varying(32) NOT NULL,
    CONSTRAINT "PK___EFMigrationsHistory" PRIMARY KEY ("MigrationId")
);

START TRANSACTION;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM "__EFMigrationsHistory" WHERE "MigrationId" = '20240516095344_Create_Database') THEN
    CREATE TABLE "Products" (
        "Id" integer GENERATED BY DEFAULT AS IDENTITY,
        "Name" character varying(100) NOT NULL,
        "Description" character varying(1000),
        "Price" numeric(18,2) NOT NULL,
        CONSTRAINT "PK_Products" PRIMARY KEY ("Id"),
        CONSTRAINT "CK_Price_NotNegative" CHECK (Price > 0)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM "__EFMigrationsHistory" WHERE "MigrationId" = '20240516095344_Create_Database') THEN
    CREATE UNIQUE INDEX "IX_Products_Name" ON "Products" ("Name");
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM "__EFMigrationsHistory" WHERE "MigrationId" = '20240516095344_Create_Database') THEN
    INSERT INTO "__EFMigrationsHistory" ("MigrationId", "ProductVersion")
    VALUES ('20240516095344_Create_Database', '8.0.5');
    END IF;
END $EF$;
COMMIT;
```

## Applying Migrations

How do we apply EF migrations to the database?

We have a few options:

- SQL scripts
- Command-line tools
- Apply migrations through code
- Migration bundles

We discussed SQL scripts in the previous section, so I won't mention them again.

### Command-line Tools

The most common approach to applying database migrations is using the CLI.
You can use either the `dotnet ef` tool or the PowerShell commands.
For example, you can execute the `Update-Database` command from PowerShell to apply any pending migrations.

```
Update-Database -Migration <ToMigration> -Connection <ConnectionString>
```

Here are the documentation links if you want to learn more:

- [EF Core CLI documentation](https://learn.microsoft.com/en-us/ef/core/cli/)
- [EF Core PowerShell documentation](https://learn.microsoft.com/en-us/ef/core/cli/powershell)

### Applying Migrations through Code

Here's a helper method for applying database migrations.
It uses an `IServiceScope` to resolve a `DbContext` instance and uses it to call the `Migrate` method.

```csharp
public static void ApplyMigration<TDbContext>(IServiceScope scope)
    where TDbContext : DbContext
{
    using TDbContext context = scope.ServiceProvider
        .GetRequiredService<TDbContext>();

    context.Database.Migrate();
}
```

You can apply migrations when the application is starting.
I **don't recommend** using this approach for production environments.
Migration can fail, concurrency issues exist, and rolling back migrations is challenging.
However, this approach can be helpful in local development and when scaffolding databases for integration testing.

```csharp
var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
    using IServiceScope scope = app.ApplicationServices.CreateScope();

    ApplyMigration<AppDbContext>(scope);
}

app.Run();
```

### Migration Bundles

Migration bundles are executable files that you can use to apply database migrations.
They're self-contained and can be executed from CI pipelines.

You can use the `Bundle-Migration` command to create a migration bundle:

```
Bundle-Migration -Connection <ConnectionString>
```

This will create an `efbundle.exe` file that we can run to apply any pending database migrations.

I recommend reading the [migration bundles documentation](https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/applying#bundles) to learn more.

## Additional Database Migration Tools

What can you do if you don't want to use EF Core migrations?

I wanted to mention some additional tools you can use for database schema versioning and running migrations:

- [FluentMigrator](https://github.com/fluentmigrator/fluentmigrator): A migration framework for .NET with a fluent API for defining migrations.
- [DbUp](https://github.com/DbUp/DbUp): A lightweight library for applying SQL scripts to databases.
- [Grate](https://erikbra.github.io/grate/): An automated database deployment (change management) system that relies on SQL scripts.
- [Flyway](https://flywaydb.org/): An open-source database migration tool that simplifies the management and versioning of database schema changes.

We won't do a deep dive on these tools, but I recommend you check out their documentation.

## EF Core Migrations Best Practices

I want to wrap up this issue with a few tips from my experience of working with EF Core migrations over the years:

- **Use meaningful migration names**: Don't name migrations with dates or generic descriptions.
  Use clear, descriptive names that indicate the purpose of the migration.
  Good examples: `AddProductsTable`, `RenameDescriptionToShortDescription`.
  This makes it much easier to understand your migration history and find specific changes.
- **Keep migrations small and focused**: Avoid creating massive migrations containing multiple unrelated changes.
  Smaller migrations are easier to review, test, and troubleshoot if something goes wrong.
  Aim for one migration per feature or logical change.
- **Test migrations thoroughly**: Before applying migrations to production, test them in a development or staging environment.
  Development and staging environments should mirror your production setup as closely as possible.
  This will help catch any unexpected issues or data loss risks before they affect real users.
- **Beware of destructive changes**: Some operations, like dropping columns or tables, can lead to irreversible data loss.
  Carefully consider the consequences before including such changes in migrations.
  Provide a way to migrate data or create a backup plan.
- **Avoid merge conflicts**: Solving merge conflicts for EF migration snapshots can be a real headache.
  Be mindful of this when working in a team that creates many database migrations.
  It's recommended to always be up-to-date with the latest migration before creating a new one.
  This should minimize the chance of creating merge conflicts.

My preferred approach to applying migrations is using SQL scripts.
Depending on the project scope and complexity, we could do this manually or through a tool that automates the process.
This allows me to review the migration and identify any potential problems.

I hope this was helpful!

Thanks for reading, and I'll see you next week.

---



<Sponsorships>

Build API applications visually using [**Postman Flows**](https://learning.postman.com/docs/postman-flows/gs/flows-overview/).
Postman Flows is a visual tool for building API-driven applications for the API-first world.
You can use Flows to chain requests, handle data, and create real-world workflows in your Postman workspace.
[**Get started here**](https://learning.postman.com/docs/postman-flows/gs/flows-overview/).

[**9 Best Practices for Building Blazor Web Applications**](https://www.telerik.com/blogs/blazor-basics-9-best-practices-building-blazor-web-applications?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness):
In this article, you will learn nine best practices for building Blazor web applications by the .NET developer and YouTube influencer Claudio Bernasconi.
[**Read it here**](https://www.telerik.com/blogs/blazor-basics-9-best-practices-building-blazor-web-applications?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness).

</Sponsorships>

From my experience working with microservices systems, things don't always go as planned.
Network requests randomly fail, application servers become overloaded, and unexpected errors appear.
That's where resilience comes in.

Resilient applications can recover from transient failures and continue to function.
Resilience is achieved by designing applications that can handle failures gracefully and recover quickly.

By designing your applications with resilience in mind, you can create robust and reliable systems, even when the going gets tough.

In this newsletter, we'll explore the tools and techniques we have in .NET to build resilient systems.

## Resilience: Why You Should Care

Sending HTTP requests is a common approach for remote communication between services.
However, HTTP requests are susceptible to failures from network or server issues.
These failures can disrupt service availability, especially as dependencies increase and the risk of cascading failures grows.

So, how can you improve the resilience of your applications and services?

Here are a few strategies you can consider to increase resilience:

- **Retries**: Retry requests that fail due to transient errors.
- **Timeouts**: Cancel requests that exceed a specified time limit.
- **Fallbacks**: Define alternative actions or results for failed operations.
- **Circuit Breakers**: Temporarily suspend communication with unavailable services.

You can use these strategies individually or in combination for optimal HTTP request resilience.

Let's see how we can introduce resilience in a .NET application.

## Resilience Pipelines

With .NET 8, integrating resilience into your applications has become much simpler.
We can use `Microsoft.Extensions.Resilience` and `Microsoft.Extensions.Http.Resilience`, which are built on top of [Polly](https://github.com/App-vNext/Polly).
Polly is a .NET resilience and transient fault-handling library.
Polly allows us to define resilience strategies such as retry, circuit breaker, timeout, rate-limiting, fallback, and hedging.

Polly received a new API surface in its latest version (V8), which was implemented in collaboration with Microsoft.
You can learn more about the [**Polly V8 API in this video**](https://youtu.be/PqVQFUCTzUM).

If you were previously using `Microsoft.Extensions.Http.Polly`, it is recommended that you switch to one of the previously mentioned packages.

Let's start by installing the required NuGet packages:

```powershell
Install-Package Microsoft.Extensions.Resilience
Install-Package Microsoft.Extensions.Http.Resilience
```

To use resilience, you must first build a pipeline consisting of resilience [strategies](https://www.pollydocs.org/strategies/).
Each strategy that we configure as part of the pipeline will execute in order of configuration.
Order is important with resilience pipelines.
Keep that in mind.

We start by creating an instance of `ResiliencePipelineBuilder`, which allows us to configure resilience strategies.

```csharp
ResiliencePipeline pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        ShouldHandle = new PredicateBuilder().Handle<ConflictException>(),
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    })
    .AddTimeout(new TimeoutStrategyOptions
    {
        Timeout = TimeSpan.FromSeconds(10)
    })
    .Build();

await pipeline.ExecuteAsync(
    async ct => await httpClient.GetAsync("https://modularmonolith.com", ct),
    cancellationToken);
```

Here's what we're adding to the resilience pipeline:

- `AddRetry` - Configures a retry resilience strategy, which we can further configure by passing in a `RetryStrategyOptions` instance.
  We can provide a predicate for the `ShouldHandle` property to define which exceptions the resilience strategy should handle.
  The retry strategy also comes with some sensible [default values](https://www.pollydocs.org/strategies/retry.html#defaults).
- `AddTimeout` - Configures a timeout strategy that will throw a `TimeoutRejectedException` if the delegate does not complete before the timeout.
  We can provide a custom timeout by passing in a `TimeoutStrategyOptions` instance.
  The default timeout is 30 seconds.

Finally, we can `Build` the resilience pipeline and get back a configured `ResiliencePipeline` instance that will apply the respective resilience strategies.
To use the `ResiliencePipeline`, we can call the `ExecuteAsync` method and pass in a delegate.

## Resilience Pipelines and Dependency Injection

Configuring a resilience pipeline every time we want to use it is cumbersome.
.NET 8 introduces a new extension method for the `IServiceCollection` interface that allows us to register resilience pipelines with dependency injection.

Instead of manually configuring resilience every time, you ask for a pre-made pipeline by name.

We start by calling the `AddResiliencePipeline` method, which allows us to configure the resilience pipeline.
Each resilience pipeline needs to have a unique key.
We can use this key to resolve the respective resilience pipeline instance.

In this example, we're passing in a `string` key which allows us to configure the non-generic `ResiliencePipelineBuilder`.

```csharp
services.AddResiliencePipeline("retry", builder =>
{
    builder.AddRetry(new RetryStrategyOptions
    {
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    });
})
```

However, we can also specify generic arguments when calling `AddResiliencePipeline`.
This allows us to configure a typed resilience pipeline using `ResiliencePipelineBuilder<TResult>`.
Using this approach, we can access the [hedging](https://www.pollydocs.org/strategies/hedging.html) and [fallback](https://www.pollydocs.org/strategies/fallback.html) strategies.

In the following example, we're configuring a fallback strategy by calling `AddFallback`.
This allows us to provide a fallback value that we can return in case of a failure.
The fallback could be a static value or come from another HTTP request or the database.

```csharp
services.AddResiliencePipeline<string, GitHubUser?>("gh-fallback", builder =>
{
    builder.AddFallback(new FallbackStrategyOptions<GitHubUser?>
    {
        FallbackAction = _ =>
            Outcome.FromResultAsValueTask<GitHubUser?>(GitHubUser.Empty)
    });
});
```

To use resilience pipelines configured with dependency injection, we can use the `ResiliencePipelineProvider`.
It exposes a `GetPipeline` method for obtaining the pipeline instance.
We have to provide the key used to register the resilience pipeline.

```csharp {3,6}
app.MapGet("users", async (
    HttpClient httpClient,
    ResiliencePipelineProvider<string> pipelineProvider) =>
{
    ResiliencePipeline<GitHubUser?> pipeline =
        pipelineProvider.GetPipeline<GitHubUser?>("gh-fallback");

    var user = await pipeline.ExecuteAsync(async token =>
        await httpClient.GetAsync("api/users", token),
        cancellationToken);
})
```

## Resilience Strategies and Polly

[Resilience strategies](https://www.pollydocs.org/strategies/) are the core component of Polly.
They're designed to run custom callbacks while introducing an additional layer of resilience.
We can't run these strategies directly.
Instead, we execute them through a resilience pipeline.

Polly categorizes resilience strategies into **reactive** and **proactive**.
Reactive strategies handle specific exceptions or results.
Proactive strategies decide to cancel or reject the execution of callbacks using a rate limiter or a timeout resilience strategy.

Polly has the following built-in resilience strategies:

- **Retry**: The classic "try again" approach.
  Works great for temporary network glitches.
  You can configure how many retries you have and even add some randomness (jitter) to avoid overloading the system if everyone retries at once.
- **Circuit-breaker**: Like an electrical circuit breaker, this prevents hammering a failing system.
  If errors pile up, the circuit breaker "trips" temporarily to give the system time to recover.
- **Fallback**: Provides a safe, default response if your primary call fails.
  It might be a cached result or a simple "service unavailable" message.
- **Hedging**: Makes multiple requests simultaneously, taking the first successful response.
  It is helpful if your system has numerous ways of handling something.
- **Timeout**: Prevents requests from hanging forever by terminating them if the timeout is exceeded.
- **Rate-limiter**: Throttles outgoing requests to prevent overwhelming external services.

## HTTP Request Resilience

Sending HTTP calls to external services is how your application interacts with the outside world.
These could be third-party services like payment gateways and identity providers or other services your team owns and operates.

The `Microsoft.Extensions.Http.Resilience` library comes with ready-to-use resilience pipelines for sending HTTP requests.

We can add resilience to outgoing [**HttpClient requests**](the-right-way-to-use-httpclient-in-dotnet) using the `AddStandardResilienceHandler` method.

```csharp {5}
services.AddHttpClient<GitHubService>(static (httpClient) =>
{
    httpClient.BaseAddress = new Uri("https://api.github.com/");
})
.AddStandardResilienceHandler();
```

This also means you can eliminate any [**delegating handlers**](extending-httpclient-with-delegating-handlers-in-aspnetcore) you previously used for resilience.

The standard resilience handler combines five Polly strategies to create a resilience pipeline suitable for most scenarios.
The standard pipeline contains the following strategies:

- **Rate limiter**: Limits the maximum number of concurrent requests sent to the dependency.
- **Total request timeout**: Introduces a total timeout, including any retry attempts.
- **Retry**: Retries a request if it fails because of a timeout or a transient error.
- **Circuit breaker**: Prevents sending further requests if too many failures are detected.
- **Attempt timeout**: Introduces a timeout for an individual request.

You can customize any aspect of the standard resilience pipeline by configuring the `HttpStandardResilienceOptions`.

## Takeaway

Resilience isn't just a buzzword; it's a core principle for building reliable software systems.
We're fortunate to have powerful tools like `Microsoft.Extensions.Resilience` and Polly at our disposal.
We can use them to design systems that gracefully handle any transient failures.

Good [**monitoring and observability**](introduction-to-distributed-tracing-with-opentelemetry-in-dotnet)
are essential to understand how your resilience mechanisms work in production.
Remember, the goal isn't to eliminate failures but to gracefully handle them and keep your application functioning.

Ready to dive deeper into resilient architecture?
My advanced course on [**building modular monoliths**](/modular-monolith-architecture) will equip you with the skills to design and implement robust, scalable systems.
Check out [**Modular Monolith Architecture**](/modular-monolith-architecture).

**Challenge**: Take a look at your existing .NET projects.
Are there any critical areas where a little resilience could go a long way?
Pick one and try applying some of the techniques we've discussed here.

That's all for today.

See you next week.

---



<Sponsorships>

[**Announcing Postman v11**](https://blog.postman.com/announcing-postman-v11/): supercharging development with AI and APIs.
Streamline API development with [**Postman v11's**](https://blog.postman.com/announcing-postman-v11/) new features for collaboration, AI, and security.
Try out the new features like VS Code extension, Partner Workspaces, and Postman Vault to streamline your API development workflow.

[**Product for Engineers**](https://newsletter.posthog.com/) from PostHog is a newsletter helping engineers improve their product skills.
[**Subscribe for free**](https://newsletter.posthog.com/) to get curated advice on building great products, lessons (and mistakes) from PostHog, and deep dives on top startups.

</Sponsorships>

API gateways provide clients with a single point of entry.
This streamlines their interactions with your system and ensures the security of your microservices or distributed system.

One critical aspect of API gateways is authentication - ensuring only authorized users and applications can access your valuable data and resources.

In this newsletter, we'll explore how you can implement API gateway authentication using [YARP](https://microsoft.github.io/reverse-proxy/index.html) (Yet Another Reverse Proxy),
a powerful and flexible reverse proxy library for .NET applications.

Here's what we will cover:

- The role of API gateways
- Configuring authentication with YARP
- Creating custom authorization policies

Let's dive in.

## The Role of API Gateways

An [**API gateway**](implementing-an-api-gateway-for-microservices-with-yarp) is the "front door" to your backend services and APIs.
It acts as an intermediary layer, handling client requests and routing them to the appropriate destinations.

The key benefits of API gateways are:

- **Centralized access**: All incoming requests must first pass through the gateway. This simplifies management and monitoring.
- **Service abstraction**: Clients interact only with the gateway. We can hide the complexity of the backend architecture from clients.
- **Performance enhancement**: Implement techniques like caching and [**load balancing**](horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing) to optimize API performance.
- **Authentication and Authorization**: API gateways verify user and application identities, enforcing whether a request is allowed or not.

<figure>
  <Image
    src="blogs/mnw_088/api_gateway.png"
    width={2115}
    height={1591}
    alt="API Gateway diagram."
  />
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>

## Configuring Authentication With YARP

We can use the API gateway to authenticate and authorize requests before they are proxied to the destination servers.
This can reduce the load on the destination servers, and introduce a layer of security.
Implementing authentication on the API gateway ensures consistent policies are implemented across your applications.

If you're new to YARP, I recommend first reading about [**how to implement an API gateway with YARP**](implementing-an-api-gateway-for-microservices-with-yarp).

By default, YARP won't authenticate or authorize requests unless enabled in the route or application configuration.

We can start by introducing [authentication](https://docs.microsoft.com/aspnet/core/security/authentication/)
and [authorization](https://learn.microsoft.com/en-us/aspnet/core/security/authorization/introduction) middleware:

```csharp {1,2}
app.UseAuthentication();
app.UseAuthorization();

app.MapReverseProxy();
```

This allows us to configure the authorization policy by providing the `AuthorizationPolicy` value in the route configuration.

There are two special values we can specify in a route's authorization parameter:

- `default` - The route will require an authenticated user.
- `anonymous` - The route will not require authorization regardless of any other configuration.

Here's how we can enforce that all incoming requests must be authenticated:

```json {6}
{
  // This is how we define reverse proxy routes.
  "Routes": {
    "api-route": {
      "ClusterId": "api-cluster",
      "AuthorizationPolicy": "default",
      "Match": {
        "Path": "api/{**catch-all}"
      }
    }
  }
}
```

We want to authorize any incoming request as soon as it hits the API gateway.
However, the destination server may still need to know who the user is (authentication) and what they can do (authorization).

YARP will pass any credentials to the proxied request.
By default, cookies, bearer tokens, and API keys will flow to the destination server.

## Creating Custom Authentication Policies

YARP can utilize the powerful [authorization policies](https://learn.microsoft.com/en-us/aspnet/core/security/authorization/policies) feature in ASP.NET Core.
We can specify a policy per route in the proxy configuration, and the rest is handled by existing ASP.NET Core authentication and authorization components.

```json {6,13}
{
  // This is how we define auth policies for reverse proxy routes.
  "Routes": {
    "api-route1": {
      "ClusterId": "api-cluster",
      "AuthorizationPolicy": "is-vip",
      "Match": {
        "Path": "api/hello-vip"
      }
    },
    "api-route2": {
      "ClusterId": "api-cluster",
      "AuthorizationPolicy": "default",
      "Match": {
        "Path": "api/{**catch-all}"
      }
    }
  }
}
```

Here's how we can create a custom `is-vip` policy with two components.
It requires an authenticated user and `vip` claim with one of the defined allowed values to be present .
To use this policy, we can just specify it as the value for the `AuthorizationPolicy` in the route configuration.

```csharp
services.AddAuthorization(options =>
{
    options.AddPolicy("is-vip", policy =>
        policy
            .RequireAuthenticatedUser()
            .RequireClaim("vip", allowedValues: true.ToString()));
});
```

## Summary

API gateways provide a unified access point, streamlining client interactions and securing your backend services.
Authentication is an essential element of API gateway security, controlling who can access your resources.

YARP offers a versatile solution for building .NET API gateways.
By integrating with ASP.NET Core's authentication and authorization frameworks, YARP enables robust security mechanisms.

This flexibility really shines with support for custom authorization policies.
This allows you to define [**granular access control**](master-claims-transformation-for-flexible-aspnetcore-authorization)
based on user roles, claims, or other attributes.

Thanks for reading, and I'll see you next week.

**P.S.** Here's the complete [source code](https://github.com/m-jovanovic/yarp-authentication) for this article if you want to try it out.

---



<Sponsorships>

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

Free eBook: [**How to Choose a Component Library?**](https://www.telerik.com/campaigns/devcraft/how-to-choose-a-component-library?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)
Practical advice for choosing the right component libraries for your business requirements.
[**Try the expert-curated solutions now!**](https://www.telerik.com/campaigns/devcraft/how-to-choose-a-component-library?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)

</Sponsorships>

Building distributed applications might seem simple at first. It's just servers talking to each other. Right?

However, it opens a set of potential problems you must consider.
What if the network has a hiccup?
A service unexpectedly crashes?
You try to scale, and everything crumbles under the load?
This is where the way your distributed system communicates becomes critical.

Traditional synchronous communication, where services call each other directly, is inherently fragile.
It creates tight coupling, making your whole application vulnerable to single points of failure.

To combat this, we can turn to distributed messaging
(and introduce an entirely different set of problems, but that's a story for another issue).

One powerful tool for achieving this in the .NET world is MassTransit.

In this week's issue, we'll explore MassTransit's implementation of the request-response pattern.

## Request-Response Messaging Pattern Introduction

Let's start by explaining how the request-response messaging pattern works.

The request-response pattern is just like making a traditional function call but over the network.
One service, the requester, sends a request message and waits for a corresponding response message.
This is a [**synchronous communication approach**](modular-monolith-communication-patterns) from the requester's side.

The good parts:

- **Loose Coupling**: Services don't need direct knowledge of each other, only of the message contracts.
  This makes changes and scaling easier.
- **Location Transparency**: The requester doesn't need to know _where_ the responder is located, leading to improved flexibility.

The bad parts:

- **Latency**: The overhead of messaging adds some additional latency.
- **Complexity**: Introducing a messaging system and managing the additional infrastructure can increase project complexity.

<Image
  src="blogs/mnw_087/request_response.png"
  width={1617}
  height={848}
  alt="Request response messaging pattern diagram."
/>

## Request-Response Messaging With MassTransit

[**MassTransit**](using-masstransit-with-rabbitmq-and-azure-service-bus)
supports the [request-response](https://masstransit.io/documentation/concepts/requests) messaging pattern out of the box.
We can use a **request client** to send requests and wait for a response.
The request client is asynchronous and supports the `await` keyword.
The request will also have a timeout of 30 seconds by default, to prevent waiting for the response for too long.

Let's imagine a scenario where you have an order processing system that needs to fetch an order's latest status.
We can fetch the status from an Order Management service.
With MassTransit, you'll create a request client to initiate the process.
This client will send a `GetOrderStatusRequest` message onto the bus.

```csharp
public record GetOrderStatusRequest
{
    public string OrderId { get; init; }
}
```

On the Order Management side, a responder (or consumer) will be listening for `GetOrderStatusRequest` messages.
It receives the request, potentially queries a database to get the status,
and then sends a `GetOrderStatusResponse` message back onto the bus.
The original request client will be waiting for this response and can then process it accordingly.

```csharp
public class GetOrderStatusRequestConsumer : IConsumer<GetOrderStatusRequest>
{
    public async Task Consume(ConsumeContext<GetOrderStatusRequest> context)
    {
        // Get the order status from a database.

        await context.ResponseAsync<GetOrderStatusResponse>(new
        {
            // Set the respective response properties.
        });
    }
}
```

## Getting User Permissions In a Modular Monolith

Here's a real-world scenario where my team decided to implement this pattern.
We were building a [**modular monolith**](/modular-monolith-architecture),
and one of the modules was responsible for managing user permissions.
The other modules could call out to the Users module to get the user's permissions.
And this works great while we are still inside a monolith system.

However, at one point we needed to extract one module into a separate service.
This meant that the communication with the Users module using simple method calls would no longer work.

Luckily, we were already using MassTransit and RabbitMQ for messaging inside the system.

So, we decided to use the MassTransit request-response feature to implement this.

The new service will inject an `IRequestClient<GetUserPermissions>`.
We can use it to send a `GetUserPermissions` message and await a response.

A very powerful feature of MassTransit is that you can await more than one response message.
In this example, we're waiting for a `PermissionsResponse` or an `Error` response.
This is great, because we also have a way to handle failures in the consumer.

```csharp {11,13,18}
internal sealed class PermissionService(
    IRequestClient<GetUserPermissions> client)
    : IPermissionService
{
    public async Task<Result<PermissionsResponse>> GetUserPermissionsAsync(
        string identityId)
    {
        var request = new GetUserPermissions(identityId);

        Response<PermissionsResponse, Error> response =
            await client.GetResponse<PermissionsResponse, Error>(request);

        if (response.Is(out Response<Error> errorResponse))
        {
            return Result.Failure<PermissionsResponse>(errorResponse.Message);
        }

        if (response.Is(out Response<PermissionsResponse> permissionResponse))
        {
            return permissionResponse.Message;
        }

        return Result.Failure<PermissionsResponse>(NotFound);
    }
}
```

In the Users module, we can easily implement the `GetUserPermissionsConsumer`.
It will respond with a `PermissionsResponse` if the permissions are found or an `Error` in case of a failure.

```csharp
public sealed class GetUserPermissionsConsumer(
    IPermissionService permissionService)
    : IConsumer<GetUserPermissions>
{
    public async Task Consume(ConsumeContext<GetUserPermissions> context)
    {
        Result<PermissionsResponse> result =
            await permissionService.GetUserPermissionsAsync(
                context.Message.IdentityId);

        if (result.IsSuccess)
        {
            await context.RespondAsync(result.Value);
        }
        else
        {
            await context.RespondAsync(result.Error);
        }
    }
}
```

## Closing Thoughts

By embracing messaging patterns with MassTransit, you're building on a much sturdier foundation.
Your .NET services are now less tightly coupled, giving you the flexibility to evolve them independently
and weather those inevitable network glitches or service outages.

The [request-response pattern](https://youtu.be/NjsoykEOkrk) is a powerful tool in your messaging arsenal.
MassTransit makes it remarkably easy to implement, ensuring that requests and responses are delivered reliably.

We can use request-response to implement communication between modules in a [**modular monolith**](/modular-monolith-architecture).
However, don't take this to the extreme, or your system might suffer from increased latency.

Start small, experiment, and see how the reliability and flexibility of messaging can transform your development experience.

That's all for this week. Stay awesome!

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240420), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240420)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

If you're building or maintaining distributed .NET applications, understanding how they behave is key to ensuring reliability and performance.

Distributed systems offer flexibility but introduce complexity, making troubleshooting a headache.
Understanding how requests flow through your system is crucial for debugging and performance optimization.

OpenTelemetry is an open-source observability framework that makes this possible.

In this article, we'll dive into what OpenTelemetry is, how to use it in your .NET projects, and the powerful insights it provides.

## OpenTelemetry Introduction

[OpenTelemetry](https://opentelemetry.io/) (OTel) is a vendor-neutral, open-source standard for instrumenting applications to generate telemetry data.
OpenTelemetry contains APIs, SDKs, tools, and integrations for creating and managing this telemetry data (traces, metrics, and logs).

Telemetry data includes:

- **Traces**: Represent the flow of requests through distributed systems, showing timings and relationships between services.
- **Metrics**: Numerical measurements of system behavior over time (e.g., request counts, error rates, memory usage).
- **Logs**: Textual records of events with rich contextual information. Structured logs.

<figure>
  <Image
    src="blogs/mnw_086/otel.png"
    width={1500}
    height={996}
    alt="OpenTelemetry Reference Architecture."
  />
  <figcaption>
    Source: [https://opentelemetry.io/docs/](https://opentelemetry.io/docs/)
  </figcaption>
</figure>

OpenTelemetry provides a unified way to collect this data, making it easier to understand the behavior and health of complex distributed applications.

We can export the telemetry data we are collecting to a service capable of processing it and providing us with an interface to analyze it.

We're going to configure OpenTelemetry to export traces directly to [Jaeger](https://www.jaegertracing.io/).

## Adding OpenTelemetry to .NET Applications

OpenTelemetry provides libraries and SDKs to add code (instrumentation) into your .NET applications.
These instrumentations automatically capture the traces, metrics, and logs we are interested in.

We're going to install the following NuGet packages:

```powershell
# Automatic tracing, metrics
Install-Package OpenTelemetry.Extensions.Hosting

# Telemetry data exporter
Install-Package OpenTelemetry.Exporter.OpenTelemetryProtocol

# Instrumentation packages
Install-Package OpenTelemetry.Instrumentation.Http
Install-Package OpenTelemetry.Instrumentation.AspNetCore
Install-Package OpenTelemetry.Instrumentation.EntityFrameworkCore
Install-Package OpenTelemetry.Instrumentation.StackExchangeRedis
Install-Package Npgsql.OpenTelemetry
```

Once we have these NuGet packages installed, it's time to configure some services.

```csharp
services
    .AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService(serviceName))
    .WithTracing(tracing =>
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddEntityFrameworkCoreInstrumentation()
            .AddRedisInstrumentation()
            .AddNpgsql();

        tracing.AddOtlpExporter();
    });
```

- `AddAspNetCoreInstrumentation` - This enables ASP.NET Core instrumentation.
- `AddHttpClientInstrumentation` - This enables `HttpClient` instrumentation for outgoing requests.
- `AddEntityFrameworkCoreInstrumentation` - This enables EF Core instrumentation.
- `AddRedisInstrumentation` - This enables Redis instrumentation.
- `AddNpgsql` - This enables PostgreSQL instrumentation.

With all of these instrumentations configured, our application will start collecting a lot of valuable traces at runtime.

We also need to configure an environment variable for the exporter added with `AddOtlpExporter` to work correctly.
We can set `OTEL_EXPORTER_OTLP_ENDPOINT` through application settings.
The address specified here will point to a local Jaeger instance.

```
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
```

## Running Jaeger Locally

[Jaeger](https://www.jaegertracing.io/) is an open source, distributed tracing platform.
Jaeger maps the flow of requests and data as they travel through a distributed system.
These requests could be calling out to multiple services, and Jaeger knows how to piece all of this information together.

Here's how to run Jaeger inside a Docker container:

```
docker run -d -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:latest
```

We're using the `jaegertracing/all-in-one:latest` image, and exposing the `4317` to accept telemetry data.
The Jaeger user interface will be exposed on the `16686` port.

## Distributed Tracing

After installing the OpenTelemetry libraries and configuring tracing in our applications, we can send some requests to generate telemetry data.
We can then access Jaeger to start analyzing our distributed traces.

**Registering a new user**

Here's an example of registering a new user with the system.
We're accessing the API gateway (`Evently.Gateway`) service, which proxies the request to the `Evently.Api` service.
And you can see that the `Evently.Api` service makes a few HTTP requests before persisting a new record in the database.

<Image
  src="blogs/mnw_086/trace_1.png"
  width={1920}
  height={650}
  alt="Distributed trace."
/>

**Publishing a message with MassTransit**

Here's another distributed trace where we publish the `UserRegisteredIntegrationEvent` over a message bus.
You can see that it's being consumed by two different services that write some data to the database.

<Image
  src="blogs/mnw_086/trace_2.png"
  width={1920}
  height={596}
  alt="Distributed trace."
/>

**Examining additional trace information**

Distributed traces can include some useful contextual information.
Here's an example trace representing a database command.
This comes from the PostgreSQL instrumentation, and we can see the SQL query that we are executing.

<Image
  src="blogs/mnw_086/trace_3.png"
  width={1920}
  height={500}
  alt="Distributed trace."
/>

**Complex distributed traces**

Here's a more complex distributed trace, which includes:

- Three .NET applications
- PostgreSQL database
- Redis distributed cache

We're sending a request to get the customer's cart.
The request will first hit the API gateway, which proxies it to the `Evently.Ticketing.Api` service that owns the data.
However, the `Evently.Ticketing.Api` service needs to reach out to the `Evently.Api` service to get the authorization information.
And all of this leads to the distributed trace you can see below.

<Image
  src="blogs/mnw_086/trace_4.png"
  width={1920}
  height={850}
  alt="Distributed trace."
/>

## Summary

Understanding modern applications, especially distributed ones, can be a real mind-bender.
OpenTelemetry is like having X-ray vision into your system.

While adding OpenTelemetry takes some upfront work, consider it an investment.
That investment pays off big time when problems pop up.
Instead of frantic guesswork, you have precise data to zero in on issues fast.

Is OpenTelemetry a magic bullet for all your problems? Nope.

But it's an excellent tool to add to your troubleshooting arsenal, especially as your .NET applications grow and get more complex.

If you're curious where the distributed traces come from, it's from the application we're building in my [**Modular Monolith course**](/modular-monolith-architecture).

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

[**Xamarin Days Are Numbered**](https://www.telerik.com/blogs/xamarin-days-numbered-ready-migrate-net-maui?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=end-of-xamarin-support) -
Are You Ready to Migrate to .NET MAUI?
Microsoft's decision to halt support for Xamarin on May 1, 2024, is rooted in the evolution of technology.
What's next?
[**Find out here.**](https://www.telerik.com/blogs/xamarin-days-numbered-ready-migrate-net-maui?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=end-of-xamarin-support)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

Sometimes, especially in high-traffic scenarios, you absolutely need to ensure that only one process can modify a piece of data at a time.

Imagine you're building the ticket sales system for a wildly popular concert.
Customers are eagerly grabbing tickets, and the last few could sell out simultaneously.
If you're not careful, multiple customers might think they've secured the final seat, leading to overbooking and disappointment!

Entity Framework Core is a fantastic tool, but it doesn't have a direct mechanism for pessimistic locking.
[Optimistic locking](solving-race-conditions-with-ef-core-optimistic-locking) (using versions) can work, but in high-contention scenarios, it can lead to a lot of retries.

So, how can we solve this problem with EF Core?

## The Scenario in More Detail

Here's a simplified code snippet to illustrate our ticketing challenge:

```csharp
public async Task Handle(CreateOrderCommand request)
{
    await using DbTransaction transaction = await unitOfWork
        .BeginTransactionAsync();

    Customer customer = await customerRepository.GetAsync(request.CustomerId);

    Order order = Order.Create(customer);
    Cart cart = await cartService.GetAsync(customer.Id);

    foreach (CartItem cartItem in cart.Items)
    {
        // Uh-oh... what if two requests hit this at the same time?
        TicketType ticketType = await ticketTypeRepository.GetAsync(
            cartItem.TicketTypeId);

        ticketType.UpdateQuantity(cartItem.Quantity);

        order.AddItem(ticketType, cartItem.Quantity, cartItem.Price);
    }

    orderRepository.Insert(order);

    await unitOfWork.SaveChangesAsync();

    await transaction.CommitAsync();

    await cartService.ClearAsync(customer.Id);
}
```

The example above is contrived, but it should be enough to explain the problem.
During checkout, we verify the `AvailableQuantity` for each ticket.

What will happen if we get concurrent requests trying to purchase the same ticket?

The worst-case scenario is we end up "overselling" the tickets.
Concurrent requests could see available tickets for sale and complete the checkout.

So, how do we solve this?

## Raw SQL to the Rescue!

Since EF Core doesn't offer pessimistic locking directly, we'll dip into a bit of good old-fashioned SQL.
We will replace the `GetAsync` call to fetch the ticket with `GetWithLockAsync`.

Thankfully, EF Core makes this easy with [raw SQL queries](ef-core-raw-sql-queries):

```csharp {10}
public async Task<TicketType> GetWithLockAsync(Guid id)
{
    return await context
        .TicketTypes
        .FromSql(
            $@"
            SELECT id, event_id, name, price, currency, quantity
            FROM ticketing.ticket_types
            WHERE id = {id}
            FOR UPDATE NOWAIT") // PostgreSQL: Lock or fail immediately
        .SingleAsync();
}
```

Understanding the magic:

- `FOR UPDATE NOWAIT`: This is the heart of pessimistic locking in PostgreSQL.
  It tells the database "Grab this row, lock it for me, and if it's already locked, raise an error right now."
- **Error Handling**: We'd wrap our `GetWithLockAsync` call in a `try-catch` block to gracefully handle locking failures, either retrying or notifying the user.

Since there isn't a built-in way in EF Core to add query hints, we have to write raw SQL queries.
We can use the PostgreSQL `SELECT FOR UPDATE` statement to acquire a row-level lock on the selected rows.
Any competing transactions will be blocked until the current transaction releases the lock.
This is a very simple way to implement pessimistic locking.

## Flavors of Locking and When to Use Them

To prevent the operation from waiting for other transactions to release any locked rows, you can combine `FOR UPDATE` with:

- `NO WAIT` - Reports an error if the row can't be locked instead of waiting.
- `SKIP LOCKED` - Skips any selected rows that cannot be locked.

Skipping locked rows comes with a caveat - you will get inconsistent results from the database.
However, this can be useful to avoid lock contention when multiple consumers access a queue-like table.
Implementing the [Outbox pattern](outbox-pattern-for-reliable-microservices-messaging) is a great example of this.

**SQL Server**: You'd use the `WITH (UPDLOCK, READPAST)` query hint for a similar effect.

## Pessimistic Locking vs. Serializable Transactions

**Serializable** transactions offer the highest level of data consistency.
They guarantee that all transactions are executed as if they happened in a strict, sequential order, even if they occur simultaneously.
This eliminates the possibility of anomalies like dirty reads (seeing uncommitted data) or non-repeatable reads (data changing between reads).

Here's how it works:

- When a transaction starts under the Serializable isolation level, the database locks all the data the transaction might access.
- These locks are held until the entire transaction is committed or rolled back.
- Any other transaction attempting to access the locked data will be blocked until the first transaction releases its locks.

While Serializable transactions provide the ultimate isolation, they come with a significant cost:

- **Performance Overhead**: Locking a large chunk of data can severely impact performance,
  especially in high-concurrency scenarios.
- **Deadlocks**: With so much locking happening, there's a higher risk of deadlocks.
  These occur when two or more transactions are waiting for locks held by each other, creating a stalemate.

Pessimistic locking with `SELECT FOR UPDATE` offers a more targeted approach to data isolation.
You explicitly lock the specific rows you need to modify.
Other transactions attempting to access the locked rows are blocked until the lock is released.

By locking only the necessary data, pessimistic locking avoids the performance overhead associated with locking everything.
Since you're locking fewer resources, the chances of deadlocks are lower.

## When to Use Each Approach

The best approach depends on your specific needs:

- **Serializable Transactions**: Ideal for scenarios involving highly sensitive data where even the slightest inconsistency is unacceptable.
  Examples include financial transactions and medical record updates.
- **Pessimistic Locking**: A great choice for most use cases, especially in high-traffic applications.
  It provides strong consistency while maintaining good performance and reducing deadlock risks.

## Takeaway

I hope this exploration of pessimistic locking has been helpful.
It's a powerful tool to have in your arsenal if you have scenarios where absolute data consistency is paramount.

Both **Serializable transactions** and pessimistic locking with `SELECT FOR UPDATE` are excellent options for ensuring data consistency.
Consider the level of isolation required, potential performance impact, and the likelihood of deadlocks when making your choice.

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240406), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240406)

[IronXL for .NET](https://ironsoftware.com/csharp/excel/?utm_source=milan&utm_medium=email&utm_campaign=milan):
Streamline Excel management with the intuitive IronXL NET Excel library.
No Microsoft Office or Excel Interop needed.
Edit files across all .NET Core and Framework versions.
[Try it free for 30 days](https://ironsoftware.com/csharp/excel/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

[Claims-based authorization](https://learn.microsoft.com/en-us/aspnet/core/security/authorization/claims) mechanisms are central to modern authorization in ASP.NET Core.
However, the access tokens issued by your Identity Provider (IDP) might not always perfectly align with your application's internal authorization needs.

External IDPs like [Microsoft Entra ID](https://www.microsoft.com/en-us/security/business/identity-access/microsoft-entra-id) (previously Azure AD) or
[Auth0](https://auth0.com) might have their own schema for claims or might not directly issue all the claims your application needs for its authorization logic.

The solution? Claims transformation.

Claims transformation allows you to modify the claims before the application uses them for authorization.

In today's issue, we will:

- Explore the concept of claims transformation in ASP.NET Core
- Explore the `IClaimsTransformation` interface with practical examples
- Address considerations for security and RBAC (Role-Based Access Control)

## How Does Claims Transformation Work?

They say a picture is worth a thousand words.
In software engineering, we have something called [UML](https://en.wikipedia.org/wiki/Unified_Modeling_Language) diagrams that we can use to paint a picture.

Here's a [sequence diagram](https://en.wikipedia.org/wiki/Sequence_diagram) showing the claims transformation flow:

1. The user authenticates with the Identity Provider
2. The user calls the backend API and provides an access token
3. The backend API performs claims transformation and authorization
4. If the user is correctly authorized, the backend API returns a response

<Image
  src="blogs/mnw_084/claims_transformation_sequence_diagram.png"
  width={1614}
  height={1357}
  alt="Claims transformation sequence diagram."
/>

Let's see how to implement this in ASP.NET Core.

## Simple Claims Transformation

Claims can be created from any user or identity data issued by a trusted identity provider.
A claim is a name-value pair that represents the subject's identity, not what the subject can do.

The core of [claims transformation](https://learn.microsoft.com/en-us/aspnet/core/security/authentication/claims#extend-or-add-custom-claims-using-iclaimstransformation)
in ASP.NET Core is the [`IClaimsTransformation`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.authentication.iclaimstransformation) interface.

It exposes a single method to transform claims:

```csharp
public interface IClaimsTransformation
{
    Task<ClaimsPrincipal> TransformAsync(ClaimsPrincipal principal);
}
```

Here's a simple example of using `IClaimsTransformation` to add a custom claim:

```csharp
internal static class CustomClaims
{
    internal const string CardType = "card_type";
}

internal sealed class CustomClaimsTransformation : IClaimsTransformation
{
    public Task<ClaimsPrincipal> TransformAsync(ClaimsPrincipal principal)
    {
        if (principal.HasClaim(claim => claim.Type == CustomClaims.CardType))
        {
            return Task.FromResult(principal);
        }

        ClaimsIdentity claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(new Claim(CustomClaims.CardType, "platinum"));

        principal.AddIdentity(claimsIdentity);

        return Task.FromResult(principal);
    }
}
```

The `CustomClaimsTransformation` class should be registered as a service:

```csharp
builder.Services
    .AddTransient<IClaimsTransformation, CustomClaimsTransformation>();
```

Finally, you can define a custom authorization policy that uses this claim:

```csharp
builder.Services.AddAuthorization(options =>
{
    options.AddPolicy(
        "HasPlatinumCard",
        builder => builder
            .RequireAuthenticatedUser()
            .RequireClaim(CustomClaims.CardType, "platinum"));
});
```

There are a few caveats with using `IClaimsTransformation` you should be aware of:

- **Might execute multiple times**: The `TransformAsync` method might get called multiple times.
  Claims transformation should be idempotent to avoid adding the same claim multiple times to the `ClaimsPrincipal`.
- **Potential performance impact**: Since it's executed on authentication requests, be mindful of your transformation logic's performance,
  especially if it involves external calls (database, APIs). Consider caching where appropriate.

## Implementing RBAC With Claims Transformation

[Role-Based Access Control (RBAC)](https://auth0.com/docs/manage-users/access-control/rbac) is an authorization model where permissions are assigned to roles,
and users are granted roles.
Claims transformation helps implement RBAC smoothly.
By adding role claims and potentially permission claims, authorization logic throughout your application can be simplified.
Another benefit is that you can keep the access token smaller and free of any role or permission claims.

Let's consider a scenario where your application manages resources at a granular level,
but your identity provider only provides coarse-grained roles like `Registered` or `Member`.
You could use claims transformation to map the `Member` role to specific fine-grained permissions like `SubmitOrder` and `PurchaseTicket`.

Here's a more complex `CustomClaimsTransformation` implementation.
We send a database query using `GetUserPermissionsQuery` and get the `PermissionsResponse` back.
The `PermissionsResponse` contains the user's permissions, which are added as custom claims.

```csharp
internal sealed class CustomClaimsTransformation(
    IServiceProvider serviceProvider)
    : IClaimsTransformation
{
    public async Task<ClaimsPrincipal> TransformAsync(
        ClaimsPrincipal principal)
    {
        if (principal.HasClaim(c => c.Type == CustomClaims.Sub ||
                                    c.Type == CustomClaims.Permission))
        {
            return principal;
        }

        using IServiceScope scope = serviceProvider.CreateScope();

        ISender sender = scope.ServiceProvider.GetRequiredService<ISender>();

        string identityId = principal.GetIdentityId();

        Result<PermissionsResponse> result = await sender.Send(
            new GetUserPermissionsQuery(identityId));

        if (result.IsFailure)
        {
            throw new ClaimsAuthorizationException(
                nameof(GetUserPermissionsQuery), result.Error);
        }

        var claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(
            new Claim(CustomClaims.Sub, result.Value.UserId.ToString()));

        foreach (string permission in result.Value.Permissions)
        {
            claimsIdentity.AddClaim(
                new Claim(CustomClaims.Permission, permission));
        }

        principal.AddIdentity(claimsIdentity);

        return principal;
    }
}
```

Now that the `ClaimsPrincipal` contains the permissions as custom claims, you can do some interesting things.
For example, you can implement a permission-based `AuthorizationHandler`:

```csharp {8}
internal sealed class PermissionAuthorizationHandler
    : AuthorizationHandler<PermissionRequirement>
{
    protected override Task HandleRequirementAsync(
        AuthorizationHandlerContext context,
        PermissionRequirement requirement)
    {
        HashSet<string> permissions = context.User.GetPermissions();

        if (permissions.Contains(requirement.Permission))
        {
            context.Succeed(requirement);
        }

        return Task.CompletedTask;
    }
}
```

## Takeaway

Claims transformation is an elegant way to bridge the gap between claims provided by identity providers and the needs of your ASP.NET Core application.
The `IClaimsTransformation` interface enables you to customize the claims of the current `ClaimsPrincipal`.
Whether you need to add roles, map external groups to internal permissions, or extract additional information from a user profile,
claims transformation offers the flexibility to do so.

However, it's important to use claims transformation with a few key considerations in mind:

- Claims transformations are executed on each request.
- The `IClaimsTransformation` should be idempotent. It should not add existing claims to the `ClaimsPrincipal` if executed multiple times.
- Design your transformations efficiently, and consider caching the results if you're fetching external data to enrich your claims.

If you want to see a complete implementation of RBAC in ASP.NET Core, check out this [Authentication & Authorization playlist](https://www.youtube.com/playlist?list=PLYpjLpq5ZDGtJOHUbv7KHuxtYLk1nJPw5).

Hope this was helpful.

See you next week.

---



<Sponsorships>

[**Build Interactive Blazor Apps**](https://www.telerik.com/blogs/building-interactive-blazor-apps-webassembly?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-general-awareness) with WebAssembly:
See what's new for Blazor WebAssembly in .NET 8 and create your first interactive WASM app.
[**Learn more here.**](https://www.telerik.com/blogs/building-interactive-blazor-apps-webassembly?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-general-awareness)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

Modern web applications need to serve increasing numbers of users and handle surges in traffic.
When a single server reaches its limits, performance degrades, leading to slow response times, errors, or complete downtime.

Load balancing is a key technique to address these challenges and improve the scalability of your application.

In this article, we will explore:

- How to use [YARP (Yet Another Reverse Proxy)](implementing-an-api-gateway-for-microservices-with-yarp) to implement load balancing
- How to leverage horizontal scaling for performance gains
- How to utilize K6 as a load testing tool

We'll dive into load balancing, why it matters, and how YARP simplifies the process for .NET applications.

## Types of Software Scalability

Before exploring YARP and load balancing further, let's cover the fundamentals of scaling.

There are two main approaches:

- **Vertical Scaling**: Involves upgrading individual servers with more powerful hardware - more CPU cores, RAM, and faster storage.
  However, this has a few limitations: costs escalate quickly, and you'll still hit a performance ceiling.
- **Horizontal Scaling**: Involves adding more servers to your infrastructure and distributing the load intelligently among them.
  This approach offers greater scalability potential, as you can continue adding servers to handle more traffic.

Horizontal scaling is where load balancing comes in, and YARP shines bright in this approach.

## Adding a Reverse Proxy

[YARP](https://microsoft.github.io/reverse-proxy/index.html) is a high-performance reverse proxy library from Microsoft.
It's designed with modern microservice architectures in mind.
A reverse proxy sits in front of your backend servers, acting as a traffic director.

Setting up YARP is quite straightforward.
You'll install the YARP NuGet package, create a basic configuration to define your backend destinations, and then activate the YARP middleware.
YARP allows you to perform routing and transformation tasks on incoming requests before they reach your backend servers.

First, let's install the `Yarp.ReverseProxy` NuGet package:

```powershell
Install-Package Yarp.ReverseProxy
```

Then, we're going to configure the required application services and introduce the YARP middleware to the request pipeline:

```csharp {3,4,8}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
```

All that's left is adding the YARP configuration to our `appsettings.json` file.
YARP uses `Routes` to represent incoming requests to the reverse proxy and `Clusters` to define the downstream services.
The `{**catch-all}` pattern allows us to easily route all incoming requests.

```json
{
  "ReverseProxy": {
    "Routes": {
      "api-route": {
        "ClusterId": "api-cluster",
        "Match": {
          "Path": "{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      }
    },
    "Clusters": {
      "api-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "http://api:8080"
          }
        }
      }
    }
  }
}
```

This configures YARP as a pass-through proxy, but let's update it to support horizontal scaling.

## Scaling Out With YARP Load Balancing

The core of horizontal scaling with YARP lies in its various [load balancing strategies](https://microsoft.github.io/reverse-proxy/articles/load-balancing.html):

- `PowerOfTwoChoices`: Selects two random destinations and selects the one with the least assigned requests.
- `FirstAlphabetical`: Selects the alphabetically first available destination server.
- `LeastRequests`: Sends requests to servers with the least assigned requests.
- `RoundRobin`: Distributes requests evenly across backend servers.
- `Random`: Randomly selects a backend server for each request.

You configure these strategies within YARP's configuration file.
The load balancing strategy can be configured using the `LoadBalancingPolicy` property on the cluster.

Here's what the updated YARP configuration looks like with `RoundRobin` load balancing:

```json {14,16-23}
{
  "ReverseProxy": {
    "Routes": {
      "api-route": {
        "ClusterId": "api-cluster",
        "Match": {
          "Path": "{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      }
    },
    "Clusters": {
      "api-cluster": {
        "LoadBalancingPolicy": "RoundRobin",
        "Destinations": {
          "destination1": {
            "Address": "http://api-1:8080"
          },
          "destination2": {
            "Address": "http://api-2:8080"
          },
          "destination3": {
            "Address": "http://api-3:8080"
          }
        }
      }
    }
  }
}
```

Here's a diagram of what our system could look like with a YARP load balancer and horizontally scaled application servers.

The incoming API requests will first hit YARP, distributing the traffic to the application servers based on the load balancing strategy.
In this example, there's one database serving multiple application instances.

<Image
  src="blogs/mnw_083/horizontal_scaling.png"
  width={1615}
  height={919}
  alt="Horizontal scaling with a YARP load balancer."
/>

And now, let's do some performance testing.

## Performance Testing with K6

To see the impact of our horizontal scaling efforts, we need to do some load testing.
[K6](https://k6.io/) is a modern, developer-friendly load testing tool.
We'll write K6 scripts to simulate user traffic on our application and compare metrics like average response time and the number of successful requests per second.

The application we're going to scale horizontally has two API endpoints.
The `POST /users` endpoint creates a new user, saves the user to a [PostgreSQL](https://www.postgresql.org/) database, and returns the user's identifier.
The `GET /users/id` endpoint returns a user with the given identifer if it exists.

Here's a k6 performance test that will:

- Ramp up to **20 virtual users**
- Send a `POST` request to the `/users` endpoint
- Check that the response is `201 Created`
- Send a `GET` request to the `/users/{id}` endpoint
- Check that the response is `200 OK`

Note that all API requests go through the YARP load balancer.

```js
import { check } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '10s', target: 20 },
    { duration: '1m40s', target: 20 },
    { duration: '10s', target: 0 }
  ]
};

export default function () {
  const proxyUrl = 'http://localhost:3000';

  const response = http.post(`${proxyUrl}/users`);

  check(response, {
    'response code was 201': (res) => res.status == 201
  });

  const userResponse = http.get(`${proxyUrl}/users/${response.body}`);

  check(userResponse, {
    'response code was 200': (res) => res.status == 200
  });
}
```

To make the performance testing results more consistent, we can limit the available resources on the [Docker](https://www.docker.com) containers to `1 CPU` and `0.5G` of RAM.

```yml {4,5}
services:
  api:
    image: ${DOCKER_REGISTRY-}loadbalancingapi
    cpus: 1
    mem_limit: '0.5G'
    ports:
      - 5000:8080
    networks:
      - proxybackend
```

Finally, here are the k6 performance testing results:

```
|    API Instances   |     Request Duration     |    Requests Per Second    |
|------------------- |------------------------- |---------------------------:
|         1          |         9.68 ms          |          2260/s           |
|--------------------|------------------------- |---------------------------|
|         2          |         6.57 ms          |          2764/s           |
|--------------------|------------------------- |---------------------------|
|         3          |         5.62 ms          |          3227/s           |
|--------------------|------------------------- |---------------------------|
|         5          |         4.65 ms          |          3881/s           |
```

## Summary

Horizontal scaling, coupled with effective load balancing, can significantly enhance the performance and scalability of your web applications.
The benefits of horizontal scaling become especially apparent in high-traffic scenarios where a single server can no longer cope with the demand.

YARP is a powerful and user-friendly reverse proxy server for .NET applications.
However, highly complex, large-scale distributed systems might benefit from specialized, standalone load-balancing solutions.
These dedicated solutions can offer more granular control and sophisticated features.

If you want to learn more, here's how to [build an API Gateway using YARP](implementing-an-api-gateway-for-microservices-with-yarp).

You can find the [source code](https://github.com/m-jovanovic/yarp-load-balancing) for this example on GitHub.

That's all for today. I'll see you next week.

---



<Sponsorships>

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240323), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240323)

</Sponsorships>

Whether you're building a data analytics platform, migrating a legacy system, or onboarding a surge of new users,
there will likely come a time when you'll need to insert a massive amount of data into your database.

Inserting the records one by one feels like watching paint dry in slow motion.
Traditional methods won't cut it.

So, understanding fast bulk insert techniques with C# and EF Core becomes essential.

In today's issue, we'll explore several options for performing bulk inserts in C#:

- Dapper
- EF Core
- EF Core Bulk Extensions
- SQL Bulk Copy

The examples are based on a `User` class with a respective `Users` table in **SQL Server**.

```csharp
public class User
{
    public int Id { get; set; }
    public string Email { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string PhoneNumber { get; set; }
}
```

This isn't a complete list of bulk insert implementations.
There are a few options I didn't explore, like manully generating SQL statements
and using [Table-Valued parameters](https://learn.microsoft.com/en-us/sql/relational-databases/tables/use-table-valued-parameters-database-engine?view=sql-server-ver16).

## EF Core Simple Approach

Let's start with a simple example using EF Core.
We're creating an `ApplicationDbContext` instance, adding a `User` object, and calling `SaveChangesAsync`.
This will insert each record to the database one by one.
In other words, each record requires one round trip to the database.

```csharp
using var context = new ApplicationDbContext();

foreach (var user in GetUsers())
{
    context.Users.Add(user);

    await context.SaveChangesAsync();
}
```

The results are as poor as you'd expect:

```
EF Core - Add one and save, for 100 users: 20 ms
EF Core - Add one and save, for 1,000 users: 260 ms
EF Core - Add one and save, for 10,000 users: 8,860 ms
```

I omitted the results with `100,000` and `1,000,000` records because they took too long to execute.

We'll use this as a "how not to do bulk inserts" example.

## Dapper Simple Insert

[Dapper](https://github.com/DapperLib/Dapper) is a simple SQL-to-object mapper for .NET.
It allows us to easily insert a collection of objects into the database.

I'm using Dapper's feature to unwrap a collection into a SQL `INSERT` statement.

```csharp
using var connection = new SqlConnection(connectionString);
connection.Open();

const string sql =
    @"
    INSERT INTO Users (Email, FirstName, LastName, PhoneNumber)
    VALUES (@Email, @FirstName, @LastName, @PhoneNumber);
    ";

await connection.ExecuteAsync(sql, GetUsers());
```

The results are much better than the initial example:

```
Dapper - Insert range, for 100 users: 10 ms
Dapper - Insert range, for 1,000 users: 113 ms
Dapper - Insert range, for 10,000 users: 1,028 ms
Dapper - Insert range, for 100,000 users: 10,916 ms
Dapper - Insert range, for 1,000,000 users: 109,065 ms
```

## EF Core Add and Save

However, EF Core still didn't throw in the towel.
The first example was poorly implemented on purpose.
EF Core can batch multiple SQL statements together, so let's use that.

If we make a simple change, we can get significantly better performance.
First, we're adding all the objects to the `ApplicationDbContext`.
Then, we're going to call `SaveChangesAsync` only once.

EF will create a batched SQL statement - group many `INSERT` statements together - and send them to the database together.
This reduces the number of round trips to the database, giving us improved performance.

```csharp
using var context = new ApplicationDbContext();

foreach (var user in GetUsers())
{
    context.Users.Add(user);
}

await context.SaveChangesAsync();
```

Here are the benchmark results of this implementation:

```
EF Core - Add all and save, for 100 users: 2 ms
EF Core - Add all and save, for 1,000 users: 18 ms
EF Core - Add all and save, for 10,000 users: 203 ms
EF Core - Add all and save, for 100,000 users: 2,129 ms
EF Core - Add all and save, for 1,000,000 users: 21,557 ms
```

Remember, it took Dapper **109 seconds** to insert `1,000,000` records.
We can achieve the same with EF Core batched queries in **~21 seconds**.

## EF Core AddRange and Save

This is an alternative to the previous example.
Instead of calling `Add` for all objects, we can call `AddRange` and pass in a collection.

I wanted to show this implementation because I prefer it over the previous one.

```csharp
using var context = new ApplicationDbContext();

context.Users.AddRange(GetUsers());

await context.SaveChangesAsync();
```

The results are very similar to the previous example:

```
EF Core - Add range and save, for 100 users: 2 ms
EF Core - Add range and save, for 1,000 users: 18 ms
EF Core - Add range and save, for 10,000 users: 204 ms
EF Core - Add range and save, for 100,000 users: 2,111 ms
EF Core - Add range and save, for 1,000,000 users: 21,605 ms
```

## EF Core Bulk Extensions

There's an awesome library called [EF Core Bulk Extensions](https://github.com/borisdj/EFCore.BulkExtensions) that we can use to squeeze out more performance.
You can do a lot more than bulk inserts with this library, so I recommend exploring it.
This library is open source, and has a community license if you meet the free usage criteria.
Check the [licensing section](https://github.com/borisdj/EFCore.BulkExtensions?#license) for more details.

For our use case, the `BulkInsertAsync` method is an excellent choice.
We can pass the collection of objects, and it will perform an SQL bulk insert.

```csharp
using var context = new ApplicationDbContext();

await context.BulkInsertAsync(GetUsers());
```

The performance is equally amazing:

```
EF Core - Bulk Extensions, for 100 users: 1.9 ms
EF Core - Bulk Extensions, for 1,000 users: 8 ms
EF Core - Bulk Extensions, for 10,000 users: 76 ms
EF Core - Bulk Extensions, for 100,000 users: 742 ms
EF Core - Bulk Extensions, for 1,000,000 users: 8,333 ms
```

For comparison, we needed **~21 seconds** to insert `1,000,000` records with EF Core batched queries.
We can do the same with the [Bulk Extensions](https://github.com/borisdj/EFCore.BulkExtensions) library in just **8 seconds**.

## SQL Bulk Copy

Lastly, if we can't get the desired performance from EF Core, we can try using [`SqlBulkCopy`](https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlbulkcopy).
SQL Server supports [bulk copy operations](https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/bulk-copy-operations-in-sql-server) natively, so let's use this.

This implementation is slightly more complex than the EF Core examples.
We need to configure the `SqlBulkCopy` instance and create a `DataTable` containing the objects we want to insert.

```csharp
using var bulkCopy = new SqlBulkCopy(ConnectionString);

bulkCopy.DestinationTableName = "dbo.Users";

bulkCopy.ColumnMappings.Add(nameof(User.Email), "Email");
bulkCopy.ColumnMappings.Add(nameof(User.FirstName), "FirstName");
bulkCopy.ColumnMappings.Add(nameof(User.LastName), "LastName");
bulkCopy.ColumnMappings.Add(nameof(User.PhoneNumber), "PhoneNumber");

await bulkCopy.WriteToServerAsync(GetUsersDataTable());
```

However, the performance is blazing fast:

```
SQL Bulk Copy, for 100 users: 1.7 ms
SQL Bulk Copy, for 1,000 users: 7 ms
SQL Bulk Copy, for 10,000 users: 68 ms
SQL Bulk Copy, for 100,000 users: 646 ms
SQL Bulk Copy, for 1,000,000 users: 7,339 ms
```

Here's how you can create a `DataTable` and populate it with a list of objects:

```csharp
DataTable GetUsersDataTable()
{
    var dataTable = new DataTable();

    dataTable.Columns.Add(nameof(User.Email), typeof(string));
    dataTable.Columns.Add(nameof(User.FirstName), typeof(string));
    dataTable.Columns.Add(nameof(User.LastName), typeof(string));
    dataTable.Columns.Add(nameof(User.PhoneNumber), typeof(string));

    foreach (var user in GetUsers())
    {
        dataTable.Rows.Add(
            user.Email, user.FirstName, user.LastName, user.PhoneNumber);
    }

    return dataTable;
}
```

## Results

Here are the results for all the bulk insert implementations:

```
| Method             |   Size     |      Speed
|------------------- |----------- |----------------:
| EF_OneByOne        | 100        |      19.800 ms |
| EF_OneByOne        | 1000       |     259.870 ms |
| EF_OneByOne        | 10000      |   8,860.790 ms |
| EF_OneByOne        | 100000     |            N/A |
| EF_OneByOne        | 1000000    |            N/A |

| Dapper_Insert      | 100        |      10.650 ms |
| Dapper_Insert      | 1000       |     113.137 ms |
| Dapper_Insert      | 10000      |   1,027.979 ms |
| Dapper_Insert      | 100000     |  10,916.628 ms |
| Dapper_Insert      | 1000000    | 109,064.815 ms |

| EF_AddAll          | 100        |       2.064 ms |
| EF_AddAll          | 1000       |      17.906 ms |
| EF_AddAll          | 10000      |     202.975 ms |
| EF_AddAll          | 100000     |   2,129.370 ms |
| EF_AddAll          | 1000000    |  21,557.136 ms |

| EF_AddRange        | 100        |       2.035 ms |
| EF_AddRange        | 1000       |      17.857 ms |
| EF_AddRange        | 10000      |     204.029 ms |
| EF_AddRange        | 100000     |   2,111.106 ms |
| EF_AddRange        | 1000000    |  21,605.668 ms |

| BulkExtensions     | 100        |       1.922 ms |
| BulkExtensions     | 1000       |       7.943 ms |
| BulkExtensions     | 10000      |      76.406 ms |
| BulkExtensions     | 100000     |     742.325 ms |
| BulkExtensions     | 1000000    |   8,333.950 ms |

| BulkCopy           | 100        |       1.721 ms |
| BulkCopy           | 1000       |       7.380 ms |
| BulkCopy           | 10000      |      68.364 ms |
| BulkCopy           | 100000     |     646.219 ms |
| BulkCopy           | 1000000    |   7,339.298 ms |
```

## Takeaway

`SqlBulkCopy` holds the crown for maximum raw speed.
However, [EF Core Bulk Extensions](https://github.com/borisdj/EFCore.BulkExtensions) deliver fantastic performance while maintaining the ease of use that Entity Framework Core is known for.

The best choice hinges on your project's specific demands:

- Performance is all that matters? `SqlBulkCopy` is your solution.
- Need excellent speed and streamlined development? EF Core is a smart choice.

I leave it up to you to decide which option is best for your use case.

Hope this was helpful.

See you next week.

---



<Sponsorships>

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w81)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Learn more here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w81)

</Sponsorships>

To delete or not to delete, that is the question (pun intended).

The traditional way to remove information in a database is through a "hard delete."
A hard delete permanently erases a record from the database table.
While this seems straightforward, it presents a significant risk: once that data is gone, it's gone for good.

Instead of physically removing a record, a soft delete marks it as deleted, usually by setting a flag like `IsDeleted` to `true`.
The record remains in the database, but it's effectively hidden from regular application queries.

Today, we'll dive into the details of how to implement soft deletes using EF Core.
We'll discuss global query filters, explore efficient ways to handle soft-deleted data, and weigh the trade-offs.

## What Is a Soft Delete?

A soft delete is a data persistence strategy that prevents the permanent deletion of records from your database.
Instead of removing data from the database, a flag is set on the record, indicating it as "deleted."

This approach allows the application to ignore these records during normal queries.
However, you can restore these records if necessary.
Soft delete is also practical if you want to keep foreign key constraints in place.
Soft delete is a "non-destructive" operation in contrast with hard delete, where data is completely removed from the database.

A hard delete uses the SQL `DELETE` statement:

```sql
DELETE FROM bookings.Reviews
WHERE Id = @BookingId;
```

A soft delete, on the other hand, uses an `UPDATE` statement:

```sql
UPDATE bookings.Reviews
SET IsDeleted = 1, DeletedOnUtc = @UtcNow
WHERE Id = @BookingId;
```

The data is still present in the database, and the operation can be undone.

But you need to remember to filter out soft-deleted data when querying the database:

```sql
SELECT *
FROM bookings.Reviews
WHERE IsDeleted = 0;
```

Let's see how we can implement soft delete with [EF Core](https://learn.microsoft.com/en-us/ef/core/).

## Soft Deletes Using EF Core Interceptors

[EF Core interceptors](https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/interceptors)
provide a powerful mechanism for intercepting and modifying database operations.
For example, you can intercept the saving changes operation to implement soft delete functionality.

Let's create an `ISoftDeletable` marker interface to represent soft-deletable entities:

```csharp
public interface ISoftDeletable
{
    bool IsDeleted { get; set; }

    DateTime? DeletedOnUtc { get; set; }
}
```

The entities that should support soft delete will implement this interface.
You will need to apply the respective database migration to create these columns.

The next component we need is a `SaveChangesInterceptor`, which allows us to hook into the `SavingChangesAsync` (or `SavingChanges`) method.
We can access the `ChangeTracker` and look for entries that implement `ISoftDeletable` and are flagged for deletion.
We can figure this out by checking if the entity state is `EntityState.Deleted`.

When we find the entities flagged for deletion, we loop through them and update their state to `EntityState.Modified`.
You should also set the respective values for the `IsDeleted` and `DeletedOnUtc` properties.
This will cause EF to generate an `UPDATE` operation instead of a `DELETE` operation.

```csharp
public sealed class SoftDeleteInterceptor : SaveChangesInterceptor
{
    public override ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is null)
        {
            return base.SavingChangesAsync(
                eventData, result, cancellationToken);
        }

        IEnumerable<EntityEntry<ISoftDeletable>> entries =
            eventData
                .Context
                .ChangeTracker
                .Entries<ISoftDeletable>()
                .Where(e => e.State == EntityState.Deleted);

        foreach (EntityEntry<ISoftDeletable> softDeletable in entries)
        {
            softDeletable.State = EntityState.Modified;
            softDeletable.Entity.IsDeleted = true;
            softDeletable.Entity.DeletedOnUtc = DateTime.UtcNow;
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }
}
```

This approach ensures that all delete operations across the application respect the soft delete policy.

You'll need to register the `SoftDeleteInterceptor` with dependency injection and configure it with the `ApplicationDbContext`.

```csharp
services.AddSingleton<SoftDeleteInterceptor>();

services.AddDbContext<ApplicationDbContext>(
    (sp, options) => options
        .UseSqlServer(connectionString)
        .AddInterceptors(
            sp.GetRequiredService<SoftDeleteInterceptor>()));
```

If you want to learn more, here's an article with a few practical use cases for [EF Core interceptors](how-to-use-ef-core-interceptors).

## Automatically Filtering Soft-Deleted Data

To ensure that soft-deleted records are automatically excluded from queries, we can use [EF Core global query filters](how-to-use-global-query-filters-in-ef-core).
We can apply query filters to entities using the `OnModelCreating` method to automatically exclude records marked as deleted.
This feature dramatically simplifies writing queries.

Here's how to configure the soft delete query filter:

```csharp {8}
public sealed class ApplicationDbContext(
    DbContextOptions<UsersDbContext> options) : DbContext(options)
{
    public DbSet<Review> Reviews { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Review>().HasQueryFilter(r => !r.IsDeleted);
    }
}
```

A limitation is you can't have more than one query filter configured per entity.

However, it's sometimes useful to explicitly include soft-deleted records.
You can achieve this using the `IgnoreQueryFilters` method.

```csharp {2}
dbContex.Reviews
    .IgnoreQueryFilters()
    .Where(r => r.ApartmentId == apartmentId)
    .ToList();
```

## Faster Queries Using Filtered Index

To enhance query performance, especially in tables with a significant number of soft-deleted records, you can use **filtered indexes**.
A filtered index only includes records that meet the specified criteria.
This reduces the index size and improves query execution times for operations that exclude filtered records.
Most popular databases support filtered indexes.

Here's how you can configure a [filtered index with EF Core](https://learn.microsoft.com/en-us/ef/core/modeling/indexes?tabs=data-annotations#index-filter):

```csharp {10-12}
public sealed class ApplicationDbContext(
    DbContextOptions<UsersDbContext> options) : DbContext(options)
{
    public DbSet<Review> Reviews { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Review>().HasQueryFilter(r => !r.IsDeleted);

        modelBuilder.Entity<Review>()
            .HasIndex(r => r.IsDeleted)
            .HasFilter("IsDeleted = 0");
    }
}
```

The `HasFilter` method accepts the SQL filter for records that will be included in the index.

You can also create a filtered index using SQL:

```sql
CREATE INDEX IX_Reviews_IsDeleted
ON bookings.Reviews (IsDeleted)
WHERE IsDeleted = 0;
```

You can learn more about filtered indexes from the documentation:

- [SQL Server filtered index](https://learn.microsoft.com/en-us/sql/relational-databases/indexes/create-filtered-indexes?view=sql-server-ver16)
- [PostgreSQL partial index](https://www.postgresql.org/docs/16/indexes-partial.html)

## Do You Really Need Soft Deletes?

It's worthwhile to think through if you even need to soft delete records.

In enterprise systems, you're typically not thinking about "deleting" data.
There are business concepts that don't involve deleting data.
A few examples are canceling an order, refunding a payment, or voiding an invoice.
These "destructive" operations return the system to a previous state.
But from a business perspective, you aren't really deleting data.

Soft deletes are helpful if there is a risk of accidental deletion.
They allow you to easily restore soft-deleted records.

In any case, consider if soft deletes make sense from a business perspective.

## Takeaway

Soft deletes offer a valuable safety net for data recovery and can enhance historical data tracking.
However, it's crucial to assess whether they truly align with your application's specific requirements.
Consider factors like the importance of deleted data recovery, any auditing needs, and your industry's regulations.
Creating a filtered index can improve query performance on tables with soft-deleted records.

If you decide that soft deletes are a good fit, EF Core provides the tools necessary for a streamlined implementation.

Thanks for reading, and I'll see you next week!

---



<Sponsorships>

Extend your existing [**Blazor Applications**](https://www.telerik.com/blazor-ui/blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-general-awareness) to Desktop and Mobile.
Bring the best of web UI to native client development.
Embed Telerik UI for Blazor web components in .NET MAUI, WPF or WinForms applications.
[**Download free trial today.**](https://www.telerik.com/blazor-ui/blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-general-awareness)

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240309), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240309)

</Sponsorships>

I've worked with many different software architectures over the years.

There's one that clearly stands out for its benefits: **Modular Monolith architecture**.

Modular monoliths blend the simplicity and robustness of traditional monolithic applications with the flexibility and scalability of microservices.
I'm tempted to say they bring together the best of both worlds.

The modular monolith architecture allows you to work in a unified codebase with clearly defined boundaries and independent modules.
You can have a high development velocity without the complexity of distributed systems.

Today, I'll introduce you to the modular monolith architecture and why you should know about it.

## What is a Modular Monolith?

A **modular monolith** is an architectural pattern that structures the application into independent modules or components with well-defined boundaries.
The modules are split based on logical boundaries, grouping together related functionalities.
This approach significantly improves the cohesion of the system.

The modules are loosely coupled, which further promotes modularity and separation of concerns.
Modules communicate through a public API, and you can learn more about this in my article on [modular monolith communication patterns](modular-monolith-communication-patterns).

But what are the benefits of a modular design?

<Image
  src="blogs/mnw_080/modular_monolith.png"
  width={2934}
  height={1554}
  alt="Modular monolith."
/>

If we take the example of an apartment booking system illustrated above.
During the holiday season, the system is expecting a traffic spike.
The bookings and payments modules need to scale so they can be deployed independently.
At the end of the holiday season, they can be merged back into a single deployment.
Modular monoliths give you this kind of flexibility.

## Modular Architecture

Modular monoliths introduce a few important technical challenges that we will need to solve.

To achieve a modular architecture, the modules:

- Must be independent and interchangeable
- Must be able to provide the required functionality
- Must have a well-defined interface exposed to other modules

Is it possible for a module to be completely independent?
Not really.
That would mean it's not integrated with other modules.
We want loosely coupled modules and to keep the number of dependencies low.
We can use a few techniques to keep the modules independent, and having good [data isolation](modular-monolith-data-isolation) is one example.

Another factor you need to consider is how strong the dependency is.
If two modules are very "chatty", you might have incorrectly defined the boundaries.
You should consider merging these modules together.

Remember, a module is a grouping of related functionalities accessed via a [well-defined interface](modular-monolith-communication-patterns).

Having a modular architecture allows you to easily extract modules into separate services.

## Monolith First

Microservices have become the most popular architectural pattern in recent years, and for good reason.
Microservices offer many benefits like clearly defined service boundaries, independent deployments, independent scalability, and much more.

However, most teams would be better off starting with a monolith application.

A monolith is an architectural pattern where all components are deployed as a single physical deployment unit.

Here's an interesting quote from Martin Fowler:

> You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.

_&mdash; [Martin Fowler](https://martinfowler.com/bliki/MonolithFirst.html)_

And I wholeheartedly agree with this. Better yet, consider starting with a modular monolith.

Even Google is jumping on board the modular monolith trend in their recent research paper, [Towards Modern Development of Cloud Applications](https://dl.acm.org/doi/pdf/10.1145/3593856.3595909).

Here are the five main challenges Google identified with microservices:

- **Performance** - The overhead of serializing data and sending it across the network has a noticeable impact on performance.
- **Correctness** - It's difficult to reason about the correctness of a distributed system when there are many interactions between components.
- **Management** - We have to manage multiple different applications, each with its release schedule.
- **Frozen APIs** - Once an API is established, it becomes hard to change without breaking any existing API consumers.
- **Development speed** - Making a change in one microservice may affect many other microservices, which requires carefully planning deployments.

When you factor in the complexity of distributed systems, starting with a modular monolith becomes increasingly compelling.
I also recommend reading about the [fallacies of distributed computing](https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing) if you're unfamiliar with them.

Well-defined, in-process components (modules) can be an excellent stepping stone to out-of-process components (services).

## Benefits of a Modular Monolith

Modular monoliths have many benefits. So, I want to highlight a few that I consider important:

- **Simplified deployment** - Unlike microservices, which require complex deployment strategies, a modular monolith can be deployed as a single unit.
- **Improved performance** - Communication between modules occurs in-process. This means that there's no network latency or data serialization/deserialization overhead.
- **Enhanced development velocity** - There's a single codebase to manage, simplifying debugging and the overall development experience.
- **Easier transaction management** - Managing transactions in a distributed system is very challenging. Modular monoliths simplify this since modules can share the same database.
- **Lower operational complexity** - Modular monoliths reduce the operational overhead that comes with managing and deploying a distributed microservices system.
- **Easier transition to Microservices** - A well-structured modular monolith offers a clear path to a microservices architecture.
  You can gradually [extract modules into separate services](monolith-to-microservices-how-a-modular-monolith-helps) when the need arises.

## Modular Monolith vs Microservices

The biggest difference between modular monoliths and microservices is how they're deployed.
Microservices elevate the logical boundaries inside a modular monolith into physical boundaries.

Microservices give you a clear strategy for modularity and decomposing the bounded contexts.
But, you can also achieve this without building a distributed system.
The problem is people end up using microservices to enforce code boundaries.

<Image
  src="blogs/mnw_080/modular_monolith_vs_microservices.png"
  width={2828}
  height={1592}
  alt="Modular monolith vs. microservices."
/>

Instead, you can build a modular monolith to get most of the same benefits.
Modular monoliths give you high cohesion, low coupling, data encapsulation, focus on business functionalities, and more.

Microservices give you all that, plus independent deployments, independent scalability, and the ability to use different technology stacks per service.

> Choose microservices for the benefits, not because your monolithic codebase is a mess.

_&mdash; [Simon Brown](https://twitter.com/simonbrown)_

## Next Steps

Modular monoliths offer a compelling way to structure applications.
They balance the benefits of well-organized code, scalability potential, and a smooth path for transitioning to microservices if needed.
If you want to improve the maintainability and adaptability of your software, consider exploring modular monoliths.

Want to dive deeper into modular monoliths? Check out these resources:

- [Modular Monolith Communication Patterns](modular-monolith-communication-patterns)
- [Modular Monolith Data Isolation](modular-monolith-data-isolation)
- [Monolith to Microservices: How a Modular Monolith Helps](monolith-to-microservices-how-a-modular-monolith-helps)
- [Modular Monoliths: How To Build One & Lessons Learned](https://youtu.be/Xo3rsiZYsJQ)
- [How to Structure a Modular Monolith Project in .NET](https://youtu.be/z3piPJ7x4WU)
- [Getting Started with Modular Monoliths in .NET](https://youtu.be/5dilYMii9T4)
- [Modular Monolith Architecture course](/modular-monolith-architecture)

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

Iron Software's Suite Expansion: [**Unveiling 9 Developer Libraries**](https://ironsoftware.com/suite/?utm_source=email&utm_medium=newsletter&utm_campaign=milan).
Iron Software amplifies its toolkit, introducing a suite encompassing PDF, OCR [(see case study),](https://ironsoftware.com/customers/case-studies/opyn-market-case-study/?utm_source=email&utm_medium=newsletter&utm_campaign=milan)
barcode, and spreadsheet and more.
[**Explore the IronSuite.**](https://ironsoftware.com/suite/?utm_source=email&utm_medium=newsletter&utm_campaign=milan)

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w79)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Learn more here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w79)

</Sponsorships>

Suppose you're building a modular monolith, a type of software architecture where different components are organized into loosely coupled modules.
Or you might need to process data asynchronously.
You'll need a tool or service that allows you to implement this.

Messaging plays a crucial role in modern software architecture, enabling communication and coordination between loosely coupled components.

An in-memory message bus is particularly useful when high performance and low latency are critical requirements.

In today's issue, we will:

- Create the required messaging abstractions
- Build an in-memory message bus using channels
- Implement an integration event processor background job
- Demonstrate how to publish and consume messages asynchronously

Let's dive in.

## When To Use an In-Memory Message Bus

I have to preface this by saying that an in-memory message bus is far from a silver bullet.
There are many caveats to using it, as you will soon learn.

But first, let's start with the pros of using an in-memory message bus:

- Because it works in memory, you have a very low-latency messaging system
- You can implement asynchronous (non-blocking) communication between components

However, there are a few drawbacks to this approach:

- Potential for losing messages if the application process goes down
- It only works inside of a single process, so it's not useful in distributed systems

A practical use case for an in-memory message bus is when building a modular monolith.
You can implement [communication between modules](modular-monolith-communication-patterns) using integration events.
When you need to extract some modules into a separate service, you can replace the in-memory bus with a distributed one.

## Defining The Messaging Abstractions

We will need a few abstractions to build our simple messaging system.
From the client's perspective, we really only need two things.
One abstraction is to publish messages, and another is to define a message handler.

The `IEventBus` interface exposes the `PublishAsync` method.
This is what we will use to publish messages.
There's also a generic constraint defined that only allows passing in an `IIntegrationEvent` instance.

```csharp
public interface IEventBus
{
    Task PublishAsync<T>(
        T integrationEvent,
        CancellationToken cancellationToken = default)
        where T : class, IIntegrationEvent;
}
```

I want to be practical with the `IIntegrationEvent` abstraction, so I'll use [MediatR](https://github.com/jbogard/MediatR) for the pub-sub support.
The `IIntegrationEvent` interface will inherit from `INotification`.
This allows us to easily define `IIntegrationEvent` handlers using `INotificationHandler<T>`.
Also, the `IIntegrationEvent` has an identifier, so we can track its execution.

The abstract `IntegrationEvent` serves as a base class for concrete implementations.

```csharp
using MediatR;

public interface IIntegrationEvent : INotification
{
    Guid Id { get; init; }
}

public abstract record IntegrationEvent(Guid Id) : IIntegrationEvent;
```

## Simple In-Memory Queue Using Channels

The `System.Threading.Channels` namespace provides data structures for asynchronously passing messages between producers and consumers.
Channels implement the [producer/consumer pattern](https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem).
Producers asynchronously produce data, and consumers asynchronously consume that data.
It's an essential pattern for building loosely coupled systems.

One of the primary motivations behind the adoption of [.NET Channels](https://learn.microsoft.com/en-us/dotnet/core/extensions/channels)
is their exceptional performance characteristics.
Unlike traditional message queues, Channels operate entirely in memory.
This has the disadvantage of the potential for message loss if the application crashes.

The `InMemoryMessageQueue` creates an unbounded channel using the `Channel.CreateUnbounded` bounded.
This means the channel can have any number of readers and writers.
It also exposes a `ChannelReader` and `ChannelWriter`, which allow consumers to publish and consume messages.

```csharp
internal sealed class InMemoryMessageQueue
{
    private readonly Channel<IIntegrationEvent> _channel =
        Channel.CreateUnbounded<IIntegrationEvent>();

    public ChannelReader<IIntegrationEvent> Reader => _channel.Reader;

    public ChannelWriter<IIntegrationEvent> Writer => _channel.Writer;
}
```

You also need to register the `InMemoryMessageQueue` as a singleton with dependency injection:

```csharp
builder.Services.AddSingleton<InMemoryMessageQueue>();
```

## Implementing The Event Bus

The `IEventBus` implementation is now straightforward with the use of channels.
The `EventBus` class uses the `InMemoryMessageQueue` to access the `ChannelWriter` and write an event to the channel.

```csharp
internal sealed class EventBus(InMemoryMessageQueue queue) : IEventBus
{
    public async Task PublishAsync<T>(
        T integrationEvent,
        CancellationToken cancellationToken = default)
        where T : class, IIntegrationEvent
    {
        await queue.Writer.WriteAsync(integrationEvent, cancellationToken);
    }
}
```

We will register the `EventBus` as a singleton service with dependency injection because it's stateless:

```csharp
builder.Services.AddSingleton<IEventBus, EventBus>();
```

## Consuming Integration Events

With the `EventBus` implementing the producer, we need a way to consume the published `IIntegrationEvent`.
We can implement a simple [background service](running-background-tasks-in-asp-net-core) using the built-in `IHostedService` abstraction.

The `IntegrationEventProcessorJob` depends on the `InMemoryMessageQueue`, but this time for reading (consuming) messages.
We'll use the `ChannelReader.ReadAllAsync` method to get back an `IAsyncEnumerable`.
This allows us to consume all the messages in the `Channel` asynchronously.

The `IPublisher` from MediatR helps us connect the `IIntegrationEvent` with the respective handlers.
It's important to resolve it from a custom scope if you want to inject scoped services into the event handlers.

```csharp
internal sealed class IntegrationEventProcessorJob(
    InMemoryMessageQueue queue,
    IServiceScopeFactory serviceScopeFactory,
    ILogger<IntegrationEventProcessorJob> logger)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await foreach (IIntegrationEvent integrationEvent in
            queue.Reader.ReadAllAsync(stoppingToken))
        {
            try
            {
                using IServiceScope scope = serviceScopeFactory.CreateScope();

                IPublisher publisher = scope.ServiceProvider
                    .GetRequiredService<IPublisher>();

                await publisher.Publish(integrationEvent, stoppingToken);
            }
            catch (Exception ex)
            {
                logger.LogError(
                    ex,
                    "Something went wrong! {IntegrationEventId}",
                    integrationEvent.Id);
            }
        }
    }
}
```

Don't forget to register the hosted service:

```csharp
builder.Services.AddHostedService<IntegrationEventProcessorJob>();
```

## Using The In-Memory Message Bus

With all of the necessary abstractions in place, we can finally use the in-memory message bus.

The `IEventBus` service will write the message to the `Channel` and immediately return.
This allows you to publish messages in a non-blocking way, which can improve performance.

```csharp {3, 16-19}
internal sealed class RegisterUserCommandHandler(
    IUserRepository userRepository,
    IEventBus eventBus)
    : ICommandHandler<RegisterUserCommand>
{
    public async Task<User> Handle(
        RegisterUserCommand command,
        CancellationToken cancellationToken)
    {
        // First, register the user.
        User user = CreateFromCommand(command);

        userRepository.Insert(user);

        // Now we can publish the event.
        await eventBus.PublishAsync(
            new UserRegisteredIntegrationEvent(user.Id),
            cancellationToken);

        return user;
    }
}
```

This solves the producer side, but we also need to create a consumer for the `UserRegisteredIntegrationEvent` message.
This part is greatly simplified because I'm using MediatR in this implementation.

We need to define an `INotificationHandler` implementation handling the integration event `UserRegisteredIntegrationEvent`.
This will be the `UserRegisteredIntegrationEventHandler`.

When the background job reads the `UserRegisteredIntegrationEvent` from the `Channel`, it will publish the message and execute the handler.

```csharp
internal sealed class UserRegisteredIntegrationEventHandler
    : INotificationHandler<UserRegisteredIntegrationEvent>
{
    public async Task Handle(
        UserRegisteredIntegrationEvent event,
        CancellationToken cancellationToken)
    {
        // Asynchronously handle the event.
    }
}
```

## Improvement Points

While our basic in-memory message bus is functional, there are several areas we can improve:

- **Resilience** - We can introduce retries when we run into exceptions, which will improve the reliability of the message bus.
- **Idempotency** - Ask yourself if you want to handle the same message twice.
  The [idempotent consumer pattern](idempotent-consumer-handling-duplicate-messages) elegantly solves this problem.
- **Dead Letter Queue** - Sometimes, we won't be able to handle a message correctly.
  It's a good idea to introduce a persistent storage for these messages.
  This is called a [Dead Letter Queue](https://aws.amazon.com/what-is/dead-letter-queue/), and it allows for troubleshooting at a later time.

We've covered the key aspects of building an in-memory message bus using .NET Channels.
You can extend this further by implementing the improvements for a more robust solution.

Remember that this implementation only works inside of one process.
Consider using a real message broker if you need a more reliable solution.

That's all for today. I'll see you next week.

---



<Sponsorships>

Which [**Blazor Web App template**](https://www.telerik.com/blogs/unified-blazor-web-app-project-template-fully-explained?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)
should you use?
Deciding which Blazor Web App template is the best fit depends on the type of application you want to build.
[**Find the answer here.**](https://www.telerik.com/blogs/unified-blazor-web-app-project-template-fully-explained?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

In ASP.NET Core applications using [Minimal APIs](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis/overview?view=aspnetcore-8.0),
registering each API endpoint with `app.MapGet`, `app.MapPost`, etc., can introduce repetitive code.
As projects grow, this manual process becomes increasingly time-consuming and prone to maintenance headaches.

You can try grouping the Minimal API endpoints using extension methods so as not to clutter the `Program` file.
This approach scales well as the project grows.
However, it feels like reinventing controllers.

I like to view each Minimal API endpoint as a standalone component.

The vision I have in my mind aligns nicely with the concept of [vertical slices.](vertical-slice-architecture)

Today, I'll show you how to register your Minimal APIs automatically with a simple abstraction.

## The Endpoint Comes First

Automatically registering Minimal APIs significantly reduces boilerplate, streamlining development.
It makes your codebase more concise and improves maintainability by establishing a centralized registration mechanism.

Let's create a simple `IEndpoint` abstraction to represent a single endpoint.

The `MapEndpoint` accepts an `IEndpointRouteBuilder`, which we can use to call `MapGet`, `MapPost`, etc.

```csharp
public interface IEndpoint
{
    void MapEndpoint(IEndpointRouteBuilder app);
}
```

Each `IEndpoint` implementation should contain exactly one Minimal API endpoint definition.

Nothing prevents you from registering multiple endpoints in the `MapEndpoint` method.
But you (really) shouldn't.

Additionally, you could implement a code analyzer or [architecture test](enforcing-software-architecture-with-architecture-tests) to enforce this rule.

```csharp
public class GetFollowerStats : IEndpoint
{
    public void MapEndpoint(IEndpointRouteBuilder app)
    {
        app.MapGet("users/{userId}/followers/stats", async (
            Guid userId,
            ISender sender) =>
        {
            var query = new GetFollowerStatsQuery(userId);

            Result<FollowerStatsResponse> result = await sender.Send(query);

            return result.Match(Results.Ok, CustomResults.Problem);
        })
        .WithTags(Tags.Users);
    }
}
```

## Sprinkle Some Reflection Magic

Reflection allows us to dynamically examine code at runtime.
For Minimal API registration, we'll use reflection to scan our .NET assemblies and find classes that implement `IEndpoint`.
Then, we will configure them as services with dependency injection.

The `Assembly` parameter should be the assembly that contains the `IEndpoint` implementations.
If you want to have endpoints in multiple assemblies (projects), you can easily extend this method to accept a collection.

```csharp
public static IServiceCollection AddEndpoints(
    this IServiceCollection services,
    Assembly assembly)
{
    ServiceDescriptor[] serviceDescriptors = assembly
        .DefinedTypes
        .Where(type => type is { IsAbstract: false, IsInterface: false } &&
                       type.IsAssignableTo(typeof(IEndpoint)))
        .Select(type => ServiceDescriptor.Transient(typeof(IEndpoint), type))
        .ToArray();

    services.TryAddEnumerable(serviceDescriptors);

    return services;
}
```

We only need to call this method once from the `Program` file:

```csharp
builder.Services.AddEndpoints(typeof(Program).Assembly);
```

## Registering Minimal APIs

The final step in our implementation is to register the endpoints automatically.
We can create an extension method on the `WebApplication`, which lets us resolve services using the `IServiceProvider`.

We're looking for all registrations of the `IEndpoint` service.
These will be the endpoint classes we can now register with the application by calling `MapEndpoint`.

I'm also adding an option to pass in a `RouteGroupBuilder` if you want to apply conventions to all endpoints.
A great example is adding a route prefix, authentication, or [API versioning.](api-versioning-in-aspnetcore)

```csharp
public static IApplicationBuilder MapEndpoints(
    this WebApplication app,
    RouteGroupBuilder? routeGroupBuilder = null)
{
    IEnumerable<IEndpoint> endpoints = app.Services
        .GetRequiredService<IEnumerable<IEndpoint>>();

    IEndpointRouteBuilder builder =
        routeGroupBuilder is null ? app : routeGroupBuilder;

    foreach (IEndpoint endpoint in endpoints)
    {
        endpoint.MapEndpoint(builder);
    }

    return app;
}
```

## Putting It All Together

Here's what the `Program` file could look like when we put it all together.

We're calling `AddEndpoints` to register the `IEndpoint` implementations.

Then, we're calling `MapEndpoints` to automatically register the Minimal APIs.

I'm also configuring a route prefix and API Versioning for each endpoint using a `RouteGroupBuilder`.

```csharp {6,19}
WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

builder.Services.AddEndpoints(typeof(Program).Assembly);

WebApplication app = builder.Build();

ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .ReportApiVersions()
    .Build();

RouteGroupBuilder versionedGroup = app
    .MapGroup("api/v{version:apiVersion}")
    .WithApiVersionSet(apiVersionSet);

app.MapEndpoints(versionedGroup);

app.Run();
```

## Takeaway

Automatic Minimal API registration with techniques like reflection can significantly improve developer efficiency and project maintainability.

While highly beneficial, it's important to acknowledge the potential **performance impact of reflection** on application startup.

So, an improvement point could be using source generators for pre-compiled registration logic.

A few alternatives worth exploring:

- [Extension methods](how-to-structure-minimal-apis)
- [FastEndpoints](https://fast-endpoints.com/)
- [Carter](https://github.com/CarterCommunity/Carter)

Hope this was helpful.

See you next week.

**P.S.** Here's the complete [source code](https://github.com/m-jovanovic/minimal-endpoints) for this article.

---



<Sponsorships>

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240217), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240217)

</Sponsorships>

Did you ever need to inject a scoped service into a singleton service?

I often need to resolve a scoped service, like the EF Core `DbContext`, in a background service.

Another example is when you need to resolve a scoped service in ASP.NET Core middleware.

If you ever tried this, you were probably greeted with an exception similar to this one:

```
System.InvalidOperationException: Cannot consume scoped service 'Scoped' from singleton 'Singleton'.
```

Today, I'll explain how you can solve this problem and safely use scoped services from within singletons in ASP.NET Core.

## ASP.NET Core Service Lifetimes

ASP.NET Core has three [service lifetimes](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes):

- Transient
- Singleton
- Scoped

[Transient services](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#transient) are created each time they're requested from the service container.

[Scoped services](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#scoped) are created once within the scope's lifetime.
For ASP.NET Core applications, a new scope is created for each request.
This is how you can resolve scoped services within a given request.

ASP.NET Core applications also have a root `IServiceProvider` used to resolve [singleton services.](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#singleton)

So, what can we do if resolving a scoped service from a singleton throws an exception?

## The Solution - `IServiceScopeFactory`

What if you want to resolve a scoped service inside a [background service](running-background-tasks-in-asp-net-core)?

You can create a new scope (`IServiceScope`) with its own `IServiceProvider` instance.
The scoped `IServiceProvider` can be used to resolve scoped services.
When the scope is disposed, all disposable services created within that scope are also disposed.

Here's an example of using the `IServiceScopeFactory` to create a new `IServiceScope`.
We're using the scope to resolve the `ApplicationDbContext`, which is a scoped service.

The `BackgroundJob` is registered as a singleton when calling `AddHostedService<BackgroundJob>`.

```csharp {6,8,9,10}
public class BackgroundJob(IServiceScopeFactory serviceScopeFactory)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = serviceScopeFactory.CreateScope();

        var dbContext = scope
            .ServiceProvider
            .GetRequiredService<ApplicationDbContext>();

        // Do some background processing with the EF database context.
        await DoWorkAsync(dbContext);
    }
}

```

## Scoped Services in Middleware

What if you want to use a scoped service in [ASP.NET Core middleware](3-ways-to-create-middleware-in-asp-net-core)?

Middleware is constructed once per application lifetime.

If you try injecting a scoped service, you'll get an exception:

```
System.InvalidOperationException: Cannot resolve scoped service 'Scoped' from root provider.
```

There are two ways to get around this.

First, you could use the previous approach with creating a new scope using `IServiceScopeFactory`.
You'll be able to resolve scoped services.
But, they won't share the same lifetime as the other scoped service in the same request.
This could even be a problem depending on your requirements.

Is there a better way?

Middleware allows you to inject scoped services in the `InvokeAsync` method.
The injected services will use the current request's scope, so they'll have the same lifetime as any other scoped service.

```csharp {5,7}
public class ConventionalMiddleware(RequestDelegate next)
{
    public async Task InvokeAsync(
        HttpContext httpContext,
        IMyScopedService scoped)
    {
        scoped.DoSomething();

        await _next(httpContext);
    }
}
```

## `IServiceScopeFactory` vs. `IServiceProvider`

You might see examples using the `IServiceProvider` to create a scope instead of the `IServiceScopeFactory`.

What's the difference between these two approaches?

The [`CreateScope` method from `IServiceProvider`](https://github.com/aspnet/DependencyInjection/blob/94b9cc9ace032f838e068702cc70ce57cc883bc7/src/DI.Abstractions/ServiceProviderServiceExtensions.cs#L125)
resolves an `IServiceScopeFactory` instance and calls `CreateScope()` on it:

```csharp
public static IServiceScope CreateScope(this IServiceProvider provider)
{
    return provider.GetRequiredService<IServiceScopeFactory>().CreateScope();
}
```

So, if you want to use the `IServiceProvider` directly to create a scope, that's fine.

However, the `IServiceScopeFactory` is a more direct way to achieve the desired result.

## Summary

Understanding the difference between Transient, Scoped, and Singleton lifetimes is crucial for managing dependencies in ASP.NET Core applications.

The `IServiceScopeFactory` provides a solution when you need to resolve scoped services from singletons.
It allows you to create a new scope, which you can use to resolve scoped services.

In middleware, we can inject scoped services into the `InvokeAsync` method.
This also ensures the services use the current request's scope and lifecycle.

Thanks for reading, and I'll see you next week!

---



<Sponsorships>

[**Blending AI with UI:**](https://www.telerik.com/blogs/blending-ai-ui-crafting-llm-uis-blazor-angular-react-vue?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)
Crafting LLM UIs in Blazor, Angular, React and Vue.
See how user interfaces are benefitting from LLMs and how you can start to implement them into your Blazor, Angular, React or Vue app.
[**Learn more here.**](https://www.telerik.com/blogs/blending-ai-ui-crafting-llm-uis-blazor-angular-react-vue?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w76)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Learn more here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w76)

</Sponsorships>

The applications you build serve your users (customers) to help them solve some problems.
It's a common requirement that you will need to know who the current application user is.

How do you get the current user's information in a Clean Architecture use case?

Use cases live in the Application layer, where you can't introduce external concerns.
Otherwise, you will be breaking the dependency rule.

Let's say you want to know who the current user is to determine if they can access some resource.
This is your typical resource-based authorization check.
But you have to interact with the identity provider to get this information.
This breaks the [dependency rule in Clean Architecture.](clean-architecture-and-the-benefits-of-structured-software-design)

I've seen this problem confuse developers who are new to Clean Architecture.

In today's issue, I'll show you how to access the current user's information in a clean way.

## Start With an Abstraction

The inner layers in Clean Architecture define abstractions for external concerns.
From the Application layer's perspective, authentication and user identity are external concerns.

The Infrastructure layer deals with external concerns, including authentication and identity management.
This is where you would implement the abstraction.

My preferred approach is creating an `IUserContext` abstraction.
The main information I need is the `UserId` of the current user.
But you can expand the `IUserContext` with any other data you think is necessary.

```csharp
public interface IUserContext
{
    bool IsAuthenticated { get; }

    Guid UserId { get; }
}
```

Let's see how to implement the `IUserContext`.

## Implementing the UserContext

The `UserContext` class is the `IUserContext` implementation in the Infrastructure layer.
We need to inject the `IHttpContextAccessor`, which allows us to access the [`ClaimsPrincipal`](https://learn.microsoft.com/en-us/dotnet/api/system.security.claims.claimsprincipal?view=net-8.0)
through the `User` property.
The `ClaimsPrincipal` gives you access to the current user's claims, containing the required information.

In this example, I'm throwing an exception if any of the properties evaluate to `null`.
You can decide if throwing an exception makes sense for you.

I also want to share an important remark here about `IHttpContextAccessor`.
We're using it to access the `HttpContext` instance — **which only exists during an API request**.
Outside an API request, the `HttpContext` will be null, and the `UserContext` will throw an exception when accessing its properties.

```csharp
internal sealed class UserContext(IHttpContextAccessor httpContextAccessor)
    : IUserContext
{
    public Guid UserId =>
        httpContextAccessor
            .HttpContext?
            .User
            .GetUserId() ??
        throw new ApplicationException("User context is unavailable");

    public bool IsAuthenticated =>
        httpContextAccessor
            .HttpContext?
            .User
            .Identity?
            .IsAuthenticated ??
        throw new ApplicationException("User context is unavailable");
}
```

Here's the `GetUserId` extension method that's used in the `UserContext.UserId` property.
It's looking for a claim with the `ClaimTypes.NameIdentifier` name, and parsing that value into a `Guid`.
You can replace this with a different type to match the user identity in your system.

```csharp
internal static class ClaimsPrincipalExtensions
{
    public static Guid GetUserId(this ClaimsPrincipal? principal)
    {
        string? userId = principal?.FindFirstValue(ClaimTypes.NameIdentifier);

        return Guid.TryParse(userId, out Guid parsedUserId) ?
            parsedUserId :
            throw new ApplicationException("User id is unavailable");
    }
}
```

## Using The Current User Information

Now that you have the `IUserContext`, you can use it from the Application layer.

A common requirement is checking if the current user can access some resources.

Here's an example using the `GetInvoiceQueryHandler`, which queries the database for an invoice.
After projecting the result to an `InvoiceResponse` object, we check if the current user is the one to whom the invoice was issued.
You can also apply this check as part of the database query.
But performing it in memory lets you return a different response to the user when they aren't authorized.
For example, a [403 Forbidden](https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3) might be appropriate.

```csharp {16}
class GetInvoiceQueryHandler(IAppDbContext dbContext, IUserContext userContext)
    : IQueryHandler<GetInvoiceQuery, InvoiceResponse>
{
    public async Task<Result<InvoiceResponse>> Handle(
        GetInvoiceQuery request,
        CancellationToken cancellationToken)
    {
        InvoiceResponse? invoiceResponse = await dbContext
            .Invoices
            .ProjectTo<InvoiceResponse>()
            .FirstOrDefaultAsync(
                invoice => invoice.Id == request.InvoiceId,
                cancellationToken);

        if (invoiceResponse is null ||
            invoiceResponse.IssuedToUserId != userContext.UserId)
        {
            return Result.Failure<InvoiceResponse>(InvoiceErrors.NotFound);
        }

        return invoiceResponse;
    }
}
```

## Takeaway

Incorporating user identity and authentication into [Clean Architecture](why-clean-architecture-is-great-for-complex-projects)
doesn't have to compromise the integrity of your design.
The Application layer should remain decoupled from external concerns such as identity management.

We respect the [Clean Architecture dependency rule](clean-architecture-and-the-benefits-of-structured-software-design) by
abstracting user-related information through the `IUserContext` interface and implementing it within the Infrastructure layer.

With this strategy, you can effectively manage user information, support authorization checks, and ensure your application remains robust and adaptable to future changes.

Remember, the key is in defining clear abstractions and respecting the architecture's boundaries.

Hope this was helpful.

See you next week.

---



<Sponsorships>

How do you [**choose the right PDF library?**](https://ironpdf.com/blog/case-studies/jeff-fritz-pdf-library-comparison/?utm_source=milan&utm_medium=email&utm_campaign=milan)
Microsoft MVP Jeff Fritz compared the most popular HTML-to-PDF libraries: iTextSharp, Syncfusion, Aspose, and IronPDF.
[**Find out which PDF library is the best.**](https://ironpdf.com/blog/case-studies/jeff-fritz-pdf-library-comparison/?utm_source=milan&utm_medium=email&utm_campaign=milan)

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240203), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn out more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240203)

</Sponsorships>

[EF Core](https://learn.microsoft.com/en-us/ef/core/) is a fantastic ORM if you're building .NET applications.

But it's a tool like any other. And you can end up using it in a suboptimal way.

Today, I'll show you a simple idea I used to get an almost **4x performance improvement**.

I'm not saying you'll see the same result, but understanding the idea will make your queries faster.

## Why This Query is Suboptimal

Here's the example I want to use to explain this powerful idea.
It's taken from a production app I was working on, but I simplified it for this example.

We're using an `InvoiceService` to get a collection of invoices for a given company.
The invoices could come from a third-party API or some other persistence store.
We're lacking detailed line item information, so we're querying the database to fill in the missing data.

The highlighted [LINQ query](https://learn.microsoft.com/en-us/ef/core/querying/) below isn't bad by itself.
It returns all the line items in one database query (round trip).

But it's missing one important realization that can unlock further performance gains.

Because we're iterating over the invoices, we're **querying the database many times**.

```csharp {21-32}
app.MapGet("invoices/{companyId}", (
    long companyId,
    InvoiceService invoiceService,
    AppDbContext dbContext) =>
{
    IEnumerable<Invoice> invoices = invoiceService.GetForCompanyId(
        companyId,
        take: 10);

    var invoiceDtos = new List<InvoiceDto>();
    foreach (var invoice in invoices)
    {
        var invoiceDto = new InvoiceDto
        {
            Id = invoice.Id,
            CompanyId = invoice.CompanyId,
            IssuedDate = invoice.IssuedDate,
            DueDate = invoice.DueDate,
            Number = invoice.Number
        };

        var lineItemDtos = await dbContext
            .LineItems
            .Where(li => invoice.LineItemIds.Contains(li.Id))
            .Select(li => new LineItemDto
            {
                Id = li.Id,
                Name = li.Name,
                Price = li.Price,
                Quantity = li.Quantity
            })
            .ToArrayAsync();

        invoiceDto.LineItems = lineItemDtos;

        invoiceDtos.Add(invoiceDto);
    }

    return invoiceDtos;
});
```

Once you figure this out, the solution comes down to applying a simple idea.

Instead of fetching the line items for each invoice, we can query all the line items ahead of time.

## Batching to the Rescue

Here's the same query, but refactored to only query the line items once.
This means there's just a single round trip to the database.

There are three components to the final design:

- Querying all the `LineItems` in a single database round-trip
- Creating a `LineItemDto` dictionary for fast lookup

Once we have the dictionary, we can loop through the invoices and assign the line items.
Populating a line item becomes a dictionary lookup (cheap) instead of a database query (expensive).

Before deciding if this solution makes sense, you should consider a few more things.

How many records can you load from the database at once?

Each invoice contains ~20 line items on average, and we're only fetching ten invoices.
So, we're loading ~200 line items from the database.
Most applications can handle this load.
But things could be different if you're fetching thousands of rows.

```csharp {10-12,26-28,41}
app.MapGet("invoices/{companyId}", (
    long companyId,
    InvoiceService invoiceService,
    AppDbContext dbContext) =>
{
    IEnumerable<Invoice> invoices = invoiceService.GetForCompanyId(
        companyId,
        take: 10);

    long[] lineItemIds = invoices
        .SelectMany(invoice => invoice.LineItemIds)
        .ToArray();

    var lineItemDtos = await dbContext
        .LineItems
        .Where(li => lineItemIds.Contains(li.Id))
        .Select(li => new LineItemDto
        {
            Id = li.Id,
            Name = li.Name,
            Price = li.Price,
            Quantity = li.Quantity
        })
        .ToListAsync();

    Dictionary<long, LineItemDto> lineItemsDictionary =
        lineItemDtos.ToDictionary(keySelector: li => li.Id);

    var invoiceDtos = new List<InvoiceDto>();
    foreach (var invoice in invoices)
    {
        var invoiceDto = new InvoiceDto
        {
            Id = invoice.Id,
            CompanyId = invoice.CompanyId,
            IssuedDate = invoice.IssuedDate,
            DueDate = invoice.DueDate,
            Number = invoice.Number,
            LineItems = invoice
                .LineItemIds
                .Select(li => lineItemsDictionary[li])
                .ToArray()
        };

        invoiceDtos.Add(invoiceDto);
    }

    return invoiceDtos;
})
```

## How Much Faster?

It seems plausible that the batch variant would be faster. Right?

We have N queries (one per invoice) in the first version and a single query in the batched version.

Here are the benchmark results I got using [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet):

<Image src="blogs/mnw_075/benchmark.png" width={1613} height={286} alt="" />

The foreach version takes **1913.3 us** (microseconds) on average.\
The batched version takes **558.6 us** on average.

That's **3.42x faster** with the batched version. This is with a local SQL database.

The batched version should be even faster if you're querying a remote database because of the impact of network round-trip time.
It quickly adds up when you have N queries (foreach version).

## Takeaway

The power of this approach lies in its simplicity and efficiency.
By batching database queries, we significantly reduce the number of round trips to the database.
This is often one of the biggest performance bottlenecks.

But it's crucial to understand that this approach is not a one-size-fits-all solution.

EF Core offers many features and optimizations, but it's up to the developer to use them effectively.

Finally, always remember to measure and benchmark.
The improvements we saw in this case were quantified through benchmarks.
Without proper measurement, it's easy to make changes that inadvertently degrade performance.

Thanks for reading, and stay awesome!

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240127), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Find out more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240127)

Free eBook: [**How to Choose a Component Library?**](https://www.telerik.com/campaigns/devcraft/how-to-choose-a-component-library?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)
Practical advice for choosing the right component libraries for your business requirements.
[**Try the expert-curated solutions now!**](https://www.telerik.com/campaigns/devcraft/how-to-choose-a-component-library?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=dt_devcraft_wp_productivity_choose_component_library)

</Sponsorships>

A **URL shortener** is a simple yet powerful tool that converts long URLs into more manageable, shorter versions.
This is particularly useful for sharing links on platforms with character limits or improving user experience by reducing clutter.
Two popular URL shorteners are [Bitly](https://bitly.com/) and [TinyURL](https://tinyurl.com/app).
Designing a URL shortener is an interesting challenge with fun problems to solve.

But how would you build a URL shortener in .NET?

URL shorteners have two core functionalities:

- Generating a unique code for a given URL
- Redirecting users who access the short link to the original URL

Today, I'll guide you through the design, implementation, and considerations for creating your URL shortener.

## URL Shortener System Design

Here's the high-level system design for our URL shortener.
We want to expose two endpoints.
One to shorten a long URL and the other to redirect users based on a shortened URL.
The shortened URLs are stored in a [PostgreSQL](https://www.postgresql.org/) database in this example.
We can introduce a distributed cache like [Redis](https://redis.io/) to the system to improve read performance.

<Image
  src="blogs/mnw_074/url_shortener.png"
  width={2856}
  height={1617}
  alt="URL shortener system design. It contains two API endpoints, a PostgreSQL database, and a Redis cache."
/>

We first need to ensure a large number of short URLs.
We're going to assign a unique code to each long URL, and use it to generate the shortened URL.
The unique code length and set of characters determine how many short URLs the system can generate.
We will discuss this in more detail when we implement unique code generation.

We're going to use the random code generation strategy.
It's straightforward to implement and has an acceptably low rate of collisions.
The trade-off we're making is increased latency, but we will also explore other options.

## The Data Model

Let's start by figuring out what we will store in the database.
Our data model is straightforward.
We have a `ShortenedUrl` class representing the URLs stored in our system:

```csharp
public class ShortenedUrl
{
    public Guid Id { get; set; }

    public string LongUrl { get; set; } = string.Empty;

    public string ShortUrl { get; set; } = string.Empty;

    public string Code { get; set; } = string.Empty;

    public DateTime CreatedOnUtc { get; set; }
}
```

This class includes properties for the original URL (`LongUrl`), the shortened URL (`ShortUrl`), and a unique code (`Code`) that represents the shortened URL.
The `Id` and `CreatedOnUtc` fields are used for database and tracking purposes.
The users will send the unique `Code` to our system, which will try to find a matching `LongUrl` and redirect them.

In addition, we will also define an EF `ApplicationDbContext` class, which is responsible for configuring our entity and setting up our database context.
I'm doing two things here to improve performance:

- Configuring the `Code` maximum length with `HasMaxLength`
- Defining a unique index on the `Code` column

A unique index shields us from concurrency conflicts, so we will never have duplicate `Code` values persisted in the database.
Setting the maximum length for this column saves storage space, and it's a requirement for indexing string columns in some databases.

Note that some databases treat strings in a case-insensitive way.
This severely reduces the number of available short URLs.
You want to configure the database to treat the unique code in a case-sensitive way.

```csharp
public class ApplicationDbContext : DbContext
{
    public ApplicationDbContext(DbContextOptions options)
        : base(options)
    {
    }

    public DbSet<ShortenedUrl> ShortenedUrls { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<ShortenedUrl>(builder =>
        {
            builder
                .Property(shortenedUrl => shortenedUrl.Code)
                .HasMaxLength(ShortLinkSettings.Length);

            builder
                .HasIndex(shortenedUrl => shortenedUrl.Code)
                .IsUnique();
        });
    }
}
```

## Unique Code Generation

The most crucial part of our URL shortener is generating a unique code for each URL.
There are a few different algorithms you can choose to implement this.
We want an even distribution of unique codes across all possible values.
This helps to reduce potential collisions.

I will implement a random, unique code generator with a predefined alphabet.
It's simple to implement, and the chance of collision is relatively low.
Still, there are more performant solutions than this, but more on this later.

Let's define a `ShortLinkSettings` class that contains two constants.
One is for defining the length of the unqualified code we will generate.
The other constant is the alphabet we will use to generate the random code.

```csharp
public static class ShortLinkSettings
{
    public const int Length = 7;
    public const string Alphabet =
        "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
}
```

The alphabet has `62` characters, which gives us `62^7` possible unique code combinations.

If you're wondering, this is `3,521,614,606,208` combinations.

Spelled out: three trillion, five hundred twenty-one billion, six hundred fourteen million, six hundred six thousand, two hundred eight.

Those are quite a few unique codes, which will be enough for our URL shortener.

Now, let's implement our `UrlShorteningService`, which handles generating unique codes.
This service generates a random string of the specified length using our predefined alphabet.
It checks against the database to ensure uniqueness.

```csharp
public class UrlShorteningService(ApplicationDbContext dbContext)
{
    private readonly Random _random = new();

    public async Task<string> GenerateUniqueCode()
    {
        var codeChars = new char[ShortLinkSettings.Length];
        const int maxValue = ShortLinkSettings.Alphabet.Length;

        while (true)
        {
            for (var i = 0; i < ShortLinkSettings.Length; i++)
            {
                var randomIndex = _random.Next(maxValue);

                codeChars[i] = ShortLinkSettings.Alphabet[randomIndex];
            }

            var code = new string(codeChars);

            if (!await dbContext.ShortenedUrls.AnyAsync(s => s.Code == code))
            {
                return code;
            }
        }
    }
}
```

**Downsides and Improvement Points**

The downside of this implementation is increased latency because we're checking each code we generate against the database.
An improvement point could be generating the unique codes in the database ahead of time.

Another improvement point could be using a fixed number of iterations instead of an infinite loop.
In case of multiple collisions in a row, the current implementation would continue until a unique value is found.
Consider throwing an exception instead after a few collisions in a row.

## URL Shortening

Now that our core business logic is ready, we can expose an endpoint to shorten URLs.
We can use a simple Minimal API endpoint.

This endpoint accepts a URL, validates it, and then uses the `UrlShorteningService` to create a shortened URL, which is then saved to the database.
We return the full shortened URL to the client.

```csharp
public record ShortenUrlRequest(string Url);

app.MapPost("shorten", async (
    ShortenUrlRequest request,
    UrlShorteningService urlShorteningService,
    ApplicationDbContext dbContext,
    HttpContext httpContext) =>
{
    if (!Uri.TryCreate(request.Url, UriKind.Absolute, out _))
    {
        return Results.BadRequest("The specified URL is invalid.");
    }

    var code = await urlShorteningService.GenerateUniqueCode();

    var request = httpContext.Request;

    var shortenedUrl = new ShortenedUrl
    {
        Id = Guid.NewGuid(),
        LongUrl = request.Url,
        Code = code,
        ShortUrl = $"{request.Scheme}://{request.Host}/{code}",
        CreatedOnUtc = DateTime.UtcNow
    };

    dbContext.ShortenedUrls.Add(shortenedUrl);

    await dbContext.SaveChangesAsync();

    return Results.Ok(shortenedUrl.ShortUrl);
});
```

There is a minor [race condition](solving-race-conditions-with-ef-core-optimistic-locking) here, as we generate a unique code first and then insert it into the database.
A concurrent request could generate the same unique code and insert it into the database before we complete our transaction.
However, the chances of this happening are low, so I decided not to handle that case.

Remember that the unique index in the database is still guarding us against duplicate values.

## URL Redirection

The second use case for a URL shortener is redirection when accessing a shortened URL.

We will expose another Minimal API endpoint for this feature.
The endpoint will accept a unique code, find the respective shortened URL, and redirect the user to the original long URL.
You can implement additional validation for the specified code before checking if there's a shortened URL in the database.

```csharp
app.MapGet("{code}", async (string code, ApplicationDbContext dbContext) =>
{
    var shortenedUrl = await dbContext
        .ShortenedUrls
        .SingleOrDefaultAsync(s => s.Code == code);

    if (shortenedUrl is null)
    {
        return Results.NotFound();
    }

    return Results.Redirect(shortenedUrl.LongUrl);
});
```

This endpoint looks up the code in the database and, if found, redirects the user to the original long URL.
The response will have a [302 (Found) status code](https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.3) per the HTTP standards.

## URL Shortener Improvement Points

While our basic URL shortener is functional, there are several areas we can improve:

- **Caching**: Implement caching to reduce database load for frequently accessed URLs.
- **Horizontal Scaling**: Design the system to scale horizontally to handle increased load.
- **Data Sharding**: Implement data sharding to distribute data across multiple databases.
- **Analytics**: Introduce analytics to track URL usage and expose reports to users.
- **User Accounts**: Allow users to create accounts to manage their URLs.

We've covered the key components of building a URL shortener using .NET.
You can take this further and implement the improvements points for a more robust solution.

If you want to see me build this from scratch, here's a [video tutorial on YouTube.](https://youtu.be/2UoA_PoEvuA)

That's all for today.

See you next week.

---



<Sponsorships>

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w73)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Learn more here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w73)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
And it's free.
[**Get Postman here!**](https://www.postman.com/downloads)

</Sponsorships>

Cross-cutting concerns are software aspects that affect the entire application.
These are your common application-level functionalities that span several layers and tiers.
Cross-cutting concerns should be centralized in one location.
This prevents code duplication and tight coupling between components.

A few examples of cross-cutting concerns are:

- Authentication & Authorization
- Logging and tracing
- Exception handling
- Validation
- Caching

In today's newsletter, I'll show you how to integrate cross-cutting concerns in Clean Architecture.

## Cross-Cutting Concerns in Clean Architecture

In Clean Architecture, [cross-cutting concerns](https://en.wikipedia.org/wiki/Cross-cutting_concern)
play an essential role in ensuring the maintainability and scalability of your system.
Ideally, these concerns should be handled separately from the core business logic.
This aligns with Clean Architecture's principles, emphasizing the decoupling of concerns and modularity.
Your core business rules remain uncluttered, and the architecture stays clean and adaptable.

Ideally, you want to implement cross-cutting concerns in the Infrastructure layer.
You can use ASP.NET Core middleware, [decorators](decorator-pattern-in-asp-net-core), or MediatR pipeline behaviors.
Whichever approach you decide to use, the guiding idea remains the same.

<Image
  src="blogs/mnw_073/clean_architecture_cross_cutting_concerns.png"
  width={2673}
  height={1947}
  alt="Cross-cutting concerns in Clean Architecture"
/>

Let's see how to implement logging, validation, and caching as cross-cutting concerns.

## Cross-Cutting Concern #1 - Logging

Logging is a fundamental aspect of software development, allowing you to look into an application's behavior.
It's vital for debugging, monitoring application health, and tracking user activities and system anomalies.
In the context of Clean Architecture, logging must be implemented in a way that maintains the separation of concerns.

An elegant way to achieve this is with [MediatR's](cqrs-pattern-with-mediatr) `IPipelineBehavior`.
By encapsulating the logging logic inside a pipeline behavior, we ensure that logging is treated as a distinct concern, separate from business logic.
This approach enables us to capture detailed information about requests flowing through the application.

Effective logging should be consistent, context-rich, and non-intrusive.
Using [Serilog's](5-serilog-best-practices-for-better-structured-logging) structured logging
capabilities, we can create logs that are not only informative but also easily queryable.
This is essential for understanding the state of the application at any given moment.

When done correctly, [structured logging](structured-logging-in-asp-net-core-with-serilog)
provides invaluable insights into your application without cluttering the core logic.
It's a balance of granularity and clarity, ensuring that your logs are a helpful tool rather than a source of noise.

```csharp
using Serilog.Context;

internal sealed class RequestLoggingPipelineBehavior<TRequest, TResponse>(
    ILogger<RequestLoggingPipelineBehavior<TRequest, TResponse>> logger)
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : class
    where TResponse : Result
{
    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        string requestName = typeof(TRequest).Name;

        logger.LogInformation(
            "Processing request {RequestName}",
            requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            logger.LogInformation(
                "Completed request {RequestName}",
                requestName);
        }
        else
        {
            using (LogContext.PushProperty("Error", result.Error, true))
            {
                logger.LogError(
                    "Completed request {RequestName} with error",
                    requestName);
            }
        }

        return result;
    }
}
```

## Cross-Cutting Concern #2 - Validation

Validation is a critical cross-cutting concern in software engineering.
It serves as the first line of defense against incorrect data entering your system.
Validation guards the application against inconsistent data states and potential security vulnerabilities.

In the example below, I'm creating a [validation pipeline behavior](cqrs-validation-with-mediatr-pipeline-and-fluentvalidation).
This setup allows for a clean separation of validation logic from business logic.
The pipeline behavior ensures that each request is validated before it reaches the core processing logic.

In approaching validation, it's crucial to distinguish between two types:

- Input validation
- Business rule validation

Input validation checks for the correctness and format of the data (like string length, number ranges, and date formats), ensuring it meets the basic criteria before processing.

On the other hand, business rule validation is more about ensuring that the data adheres to your domain's specific rules and logic.

Effective validation practices significantly contribute to the resilience and reliability of an application.
By enforcing validation rules, you can maintain a high data quality standard and ensure a better user experience.

```csharp
internal sealed class ValidationPipelineBehavior<TRequest, TResponse>(
    IEnumerable<IValidator<TRequest>> validators)
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : class
{
    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        ValidationFailure[] validationFailures = await ValidateAsync(request);

        if (validationFailures.Length != 0)
        {
            throw new ValidationException(validationFailures);
        }

        return await next();
    }

    private async Task<ValidationFailure[]> ValidateAsync(TRequest request)
    {
        if (!validators.Any())
        {
            return [];
        }

        var context = new ValidationContext<TRequest>(request);

        ValidationResult[] validationResults = await Task.WhenAll(
            validators.Select(validator => validator.ValidateAsync(context)));

        ValidationFailure[] validationFailures = validationResults
            .Where(validationResult => !validationResult.IsValid)
            .SelectMany(validationResult => validationResult.Errors)
            .ToArray();

        return validationFailures;
    }
}
```

## Cross-Cutting Concern #3: Caching

Caching is an essential cross-cutting concern in software development.
It's primarily aimed at enhancing performance and scalability.
Caching involves temporarily storing data in a fast-access layer.
This reduces the need to fetch or calculate the same information repeatedly.

The caching pipeline behavior, which you see below, implements the [Cache Aside pattern.](https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside)
This pattern involves checking the cache before processing the request and updating the cache with new data as needed.
It's a popular caching strategy due to its simplicity and effectiveness.
Here's a [video tutorial](https://youtu.be/LOEYZRE72wE) if you want to see how I implemented this.

When implementing caching, it's crucial to consider:

- **What to Cache:** Identify data that is expensive to compute or retrieve and stable enough to be cached.
- **Cache Invalidations:** Determine when and how cached data should be invalidated.
- **Cache Configuration:** Configure cache settings like expiration and size appropriately.

Effective caching improves response times and reduces the load on your system, making it a critical strategy for building scalable .NET applications.

```csharp
internal sealed class QueryCachingPipelineBehavior<TRequest, TResponse>(
    ICacheService cacheService,
    ILogger<QueryCachingPipelineBehavior<TRequest, TResponse>> logger)
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : ICachedQuery
    where TResponse : Result
{
    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        TResponse? cachedResult = await cacheService.GetAsync<TResponse>(
            request.CacheKey,
            cancellationToken);

        string requestName = typeof(TRequest).Name;
        if (cachedResult is not null)
        {
            logger.LogInformation("Cache hit for {RequestName}", requestName);

            return cachedResult;
        }

        logger.LogInformation("Cache miss for {RequestName}", requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            await cacheService.SetAsync(
                request.CacheKey,
                result,
                request.Expiration,
                cancellationToken);
        }

        return result;
    }
}
```

## What To Do Next

Managing cross-cutting concerns such as logging, caching, validation, and exception handling is not just about technical implementation.
It's about aligning these aspects with the core principles of [Clean Architecture.](clean-architecture-and-the-benefits-of-structured-software-design)
By adopting the decoupling techniques we discussed, you can ensure that your .NET projects are robust and maintainable.

Each step you take towards refining your handling of cross-cutting concerns is a step towards a better software architecture.
I encourage you to experiment with these strategies in your own .NET projects.
If you want a structured guide covering these aspects in-depth, take a look at [Pragmatic Clean Architecture.](/pragmatic-clean-architecture)

Remember, the beauty of software development lies in the continuous evolution and relentless pursuit of improvement.

Hope this was helpful.

See you next week.

---



<Sponsorships>

Is [**Blazor the future**](https://www.telerik.com/blogs/is-blazor-future-everything-web?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)
of everything Web?
With the .NET 8 release, a collection of new features take Blazor from a "niche .NET framework" to a "modern web framework - and beyond."
[**Learn more here.**](https://www.telerik.com/blogs/is-blazor-future-everything-web?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)

[**Case Study: Discover Agorus'**](https://ironsoftware.com/customers/case-studies/agorus-build-uses-iron-suite/?utm_source=milan&utm_medium=email&utm_campaign=milan)
use of IronPDF, IronOCR, and IronBarcode in C#.
Tackle complex data effortlessly with automated PDF generation, text extraction from blueprints, and barcode processing.
[**Get started here.**](https://ironsoftware.com/suite/#trial-license?utm_source=milan&utm_medium=email&utm_campaign=milan)

</Sponsorships>

[Delegating handlers](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.delegatinghandler?view=net-8.0) are like [ASP.NET Core middleware](3-ways-to-create-middleware-in-asp-net-core).
Except they work with the [`HttpClient`](the-right-way-to-use-httpclient-in-dotnet).
The ASP.NET Core request pipeline allows you to introduce custom behavior with [middleware.](3-ways-to-create-middleware-in-asp-net-core)
You can solve many cross-cutting concerns using middleware — logging, tracing, validation, authentication, authorization, etc.

But, an important aspect here is that middleware works with incoming HTTP requests to your API.
Delegating handlers work with outgoing requests.

[`HttpClient`](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-8.0) is my preferred way of sending HTTP requests in ASP.NET Core.
It's straightforward to use and solves most of my use cases.
You can use delegating handlers to extend the `HttpClient` with behavior before or after sending an HTTP request.

Today, I want to show you how to use a [`DelegatingHandler`](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.delegatinghandler?view=net-8.0) to introduce:

- Logging
- Resiliency
- Authentication

## Configuring an HttpClient

Here's a very simple application that:

- Configures the `GitHubService` class as a typed HTTP client
- Sets the `HttpClient.BaseAddress` to point to the GitHub API
- Exposes an endpoint that retrieves a GitHub user by their username

We're going to extend the `GitHubService` behavior using delegating handlers.

```csharp
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpClient<GitHubService>(httpClient =>
{
    httpClient.BaseAddress = new Uri("https://api.github.com");
});

var app = builder.Build();

app.MapGet("api/users/{username}", async (
    string username,
    GitHubService gitHubService) =>
{
    var content = await gitHubService.GetByUsernameAsync(username);

    return Results.Ok(content);
});

app.Run();
```

The `GitHubService` class is a [typed client](the-right-way-to-use-httpclient-in-dotnet#replacing-named-clients-with-typed-clients) implementation.
Typed clients allow you to expose a strongly typed API and hide the `HttpClient`.
The runtime takes care of providing a configured `HttpClient` instance through dependency injection.
You also don't have to think about disposing of the `HttpClient`.
It's resolved from an underlying `IHttpClientFactory` that manages the `HttpClient` lifetime.

```csharp
public class GitHubService(HttpClient client)
{
    public async Task<GitHubUser?> GetByUsernameAsync(string username)
    {
        var url = $"users/{username}";

        return await client.GetFromJsonAsync<GitHubUser>(url);
    }
}
```

## Logging HTTP Requests Using Delegating Handlers

Let's start with a simple example.
We will add logging before and after sending an HTTP request.
For this, we will to create a custom delegating handler - `LoggingDelegatingHandler`.

The custom delegating handler implements the `DelegatingHandler` base class.
Then, you can override the `SendAsync` method to introduce additional behavior.

```csharp
public class LoggingDelegatingHandler(ILogger<LoggingDelegatingHandler> logger)
    : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        try
        {
            logger.LogInformation("Before HTTP request");

            var result = await base.SendAsync(request, cancellationToken);

            result.EnsureSuccessStatusCode();

            logger.LogInformation("After HTTP request");

            return result;
        }
        catch (Exception e)
        {
            logger.LogError(e, "HTTP request failed");

            throw;
        }
    }
}
```

You also need to register the `LoggingDelegatingHandler` with dependency injection.
Delegating handlers must be registered as **transient** services.

The `AddHttpMessageHandler` method adds the `LoggingDelegatingHandler` as a delegating handler for the `GitHubService`.
Any HTTP request sent using the `GitHubService` will first go through the `LoggingDelegatingHandler`.

```csharp {1,7}
builder.Services.AddTransient<LoggingDelegatingHandler>();

builder.Services.AddHttpClient<GitHubService>(httpClient =>
{
    httpClient.BaseAddress = new Uri("https://api.github.com");
})
.AddHttpMessageHandler<LoggingDelegatingHandler>();
```

Let's see what else we can do.

## Adding Resiliency With Delegating Handlers

Building [resilient](https://learn.microsoft.com/en-us/dotnet/core/resilience/?tabs=dotnet-cli) applications is an important requirement for cloud development.

The `RetryDelegatingHandler` class uses [Polly](https://github.com/App-vNext/Polly) to create an `AsyncRetryPolicy`.
The retry policy wraps the HTTP request and retries it in case of a transient failure.

```csharp
public class RetryDelegatingHandler : DelegatingHandler
{
    private readonly AsyncRetryPolicy<HttpResponseMessage> _retryPolicy =
        Policy<HttpResponseMessage>
            .Handle<HttpRequestException>()
            .RetryAsync(2);

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var policyResult = await _retryPolicy.ExecuteAndCaptureAsync(
            () => base.SendAsync(request, cancellationToken));

        if (policyResult.Outcome == OutcomeType.Failure)
        {
            throw new HttpRequestException(
                "Something went wrong",
                policyResult.FinalException);
        }

        return policyResult.Result;
    }
}
```

You also need to register the `RetryDelegatingHandler` with dependency injection.
Also, remember to configure it as a message handler.
In this example, I'm chaining two delegating handlers together, and they will run one after another.

```csharp {1,8}
builder.Services.AddTransient<RetryDelegatingHandler>();

builder.Services.AddHttpClient<GitHubService>(httpClient =>
{
    httpClient.BaseAddress = new Uri("https://api.github.com");
})
.AddHttpMessageHandler<LoggingDelegatingHandler>()
.AddHttpMessageHandler<RetryDelegatingHandler>();
```

## Solving Authentication With Delegating Handlers

Authentication is a cross-cutting concern you will have to solve in any microservices application.
A common use case for delegating handlers is adding the `Authorization` header before sending an HTTP request.

For example, the GitHub API requires an access token to be present for authenticating incoming requests.
The `AuthenticationDelegatingHandler` class adds the `Authorization` header value from the `GitHubOptions`.
Another requirement is specifying the `User-Agent` header, which is set from the app configuration.

```csharp
public class AuthenticationDelegatingHandler(IOptions<GitHubOptions> options)
    : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        request.Headers.Add("Authorization", options.Value.AccessToken);
        request.Headers.Add("User-Agent", options.Value.UserAgent);

        return base.SendAsync(request, cancellationToken);
    }
}
```

Don't forget to configure the `AuthenticationDelegatingHandler` with the `GitHubService`:

```csharp {1,9}
builder.Services.AddTransient<AuthenticationDelegatingHandler>();

builder.Services.AddHttpClient<GitHubService>(httpClient =>
{
    httpClient.BaseAddress = new Uri("https://api.github.com");
})
.AddHttpMessageHandler<LoggingDelegatingHandler>()
.AddHttpMessageHandler<RetryDelegatingHandler>()
.AddHttpMessageHandler<AuthenticationDelegatingHandler>();
```

Here's a more involved authentication example using the `KeyCloakAuthorizationDelegatingHandler`.
This is a delegating handler that acquires the access token from [Keycloak](https://www.keycloak.org/).
Keycloak is an open-source identity and access management service.

I used Keycloak as the identity provider in my [Pragmatic Clean Architecture](/pragmatic-clean-architecture) course.

The delegating handler in this example uses an [OAuth 2.0](https://oauth.net/2/) [client credentials](https://www.oauth.com/oauth2-servers/access-tokens/client-credentials/) grant flow to obtain an access token.
This grant is used when applications request an access token to access their own resources, not on behalf of a user.

```csharp
public class KeyCloakAuthorizationDelegatingHandler(
    IOptions<KeycloakOptions> keycloakOptions)
    : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var authToken = await GetAccessTokenAsync();

        request.Headers.Authorization = new AuthenticationHeaderValue(
            JwtBearerDefaults.AuthenticationScheme,
            authToken.AccessToken);

        var httpResponseMessage = await base.SendAsync(
            request,
            cancellationToken);

        httpResponseMessage.EnsureSuccessStatusCode();

        return httpResponseMessage;
    }

    private async Task<AuthToken> GetAccessTokenAsync()
    {
        var params = new KeyValuePair<string, string>[]
        {
            new("client_id", _keycloakOptions.Value.AdminClientId),
            new("client_secret", _keycloakOptions.Value.AdminClientSecret),
            new("scope", "openid email"),
            new("grant_type", "client_credentials")
        };

        var content = new FormUrlEncodedContent(params);

        var authRequest = new HttpRequestMessage(
            HttpMethod.Post,
            new Uri(_keycloakOptions.TokenUrl))
        {
            Content = content
        };

        var response = await base.SendAsync(authRequest, cancellationToken);

        response.EnsureSuccessStatusCode();

        return await response.Content.ReadFromJsonAsync<AuthToken>() ??
               throw new ApplicationException();
    }
}
```

## Takeaway

Delegating handlers give you a powerful mechanism to extend the behavior when sending requests with an `HttpClient`.
You can use delegating handlers to solve cross-cutting concerns, similar to how you would use middleware.

Here are a few ideas on how you could use delegating handlers:

- Logging before and after sending HTTP requests
- Introducing resilience policies (retry, fallback)
- Validating the HTTP request content
- Authenticating with an external API

I'm sure you can come up with a few use cases yourself.

I made a video showing how to [implement delegating handlers](https://youtu.be/_u6v4D6qgDI), and you can [watch it here.](https://youtu.be/_u6v4D6qgDI)

Thanks for reading, and stay awesome!

---



<Sponsorships>

Learn about Postman with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Whether you're a developer or a tester, there's something for you.
Collaborate on API development with workspaces.
Sync your data across devices.
And it's free.
[**Get Postman here!**](https://www.postman.com/downloads)

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240106), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Find out more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240106)

</Sponsorships>

[MassTransit](https://masstransit.io/) is an open-source distributed application framework for .NET.
It provides a messaging abstraction on top of the supported message transports.
MassTransit lets you focus on adding business value instead of worrying about messaging complexity.

MassTransit supports many message transport technologies.
Here are a few that are popular:

- RabbitMQ
- Azure Service Bus
- Amazon SQS
- Kafka

In today's newsletter, I'll show you how to install and configure MassTransit in .NET.
We'll connect MassTransit to a few message brokers - RabbitMQ and Azure Service Bus.
And we will also cover how to publish and consume messages with MassTransit.

## Why Use MassTransit?

MassTransit solves many challenges of building distributed applications.
You (almost) don't have to think about the underlying message transport.
This allows you to focus on providing business value.

Here are a few things MassTransit does for you:

- **Message routing** - Type-based publish/subscribe, automatic broker topology configuration
- **Exception handling** - Messages can be retried or moved to an error queue
- **Dependency injection** - Service collection configuration and scope service provider
- **Request-Response** - Handle requests with automatic response routing
- **Observability** - Native [Open Telemetry (OTEL)](https://opentelemetry.io/) support
- **Scheduling** - Schedule message delivery using transport delay, Quartz.NET, or Hangfire
- **Sagas** - Reliable, durable, event-driven workflow orchestration

Let's see how to start using MassTransit.

## Installing and Configuring MassTransit with RabbitMQ

You need to install the `MassTransit` library.
If you already have a message transport, you can install the respective transport library.
Let's add the `MassTransit.RabbitMQ` library to configure [RabbitMQ](https://www.rabbitmq.com/) as the transport mechanism.

```powershell
Install-Package MassTransit

Install-Package MassTransit.RabbitMQ
```

Then, you can configure the required services for `MassTransit`.
The `AddMassTransit` method accepts a delegate where you can configure many settings.
For example, you can set the messaging endpoints to use kebab case naming by calling `SetKebabCaseEndpointNameFormatter`.
This is also where you configure the transport mechanism.
Calling `UsingRabbitMq` allows you to connect RabbitMQ as the transport mechanism.

```csharp
builder.Services.AddMassTransit(busConfigurator =>
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingRabbitMq((context, configurator) =>
    {
        configurator.Host("localhost", "/", h =>
        {
            h.Username("guest");
            h.Password("guest");
        });

        configurator.ConfigureEndpoints(context);
    });
});
```

MassTransit takes care of setting up the required broker topology.
RabbitMQ supports exchanges and queues, so messages are sent or published to exchanges.
RabbitMQ routes those messages through exchanges to the appropriate queues.

You can start RabbitMQ locally inside a Docker container:

```csharp
docker run -d --name rabbitmq -p 5672:5672
```

## Configuring MassTransit with Azure Service Bus

[Azure Service Bus](https://azure.microsoft.com/en-us/products/service-bus)
is a cloud-based message broker with support for queues and topics.
MassTransit fully supports Azure Service Bus, including many advanced features and capabilities.
However, you must be on the Standard or Premium tier of the Microsoft Azure Service Bus service.

To configure MassTransit to work with Azure Service Bus, you need to install the required transport library:

```powershell
Install-Package MassTransit.Azure.ServiceBus.Core
```

Then, you can connect to Azure Service Bus by calling `UsingAzureServiceBus` and providing the connection string.
Everything else remains the same.
MassTransit takes care of configuring the broker topology.
MassTransit sends messages to topics, and Azure Service Bus routes those messages to the appropriate queues.

```csharp
builder.Services.AddMassTransit(busConfigurator =>
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingAzureServiceBus((context, configurator) =>
    {
        configurator.Host("<CONNECTION_STRING>");

        configurator.ConfigureEndpoints(context);
    });
});
```

## Using the MassTransit In Memory Transport

You can also configure MassTransit to use an in-memory transport.
It's useful for testing, because it doesn't require a message broker to be running.
Another advantage is that it's fast.

However, there's a big problem with the in-memory transport - **it's not durable.**

If the message bus is stopped, all messages are lost.
Don't use the in-memory transport for production systems.

It will only work on a single machine.
So, it doesn't make sense for distributed applications.

```csharp
builder.Services.AddMassTransit(busConfigurator =>
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingInMemory((context, configurator) =>
    {
        configurator.ConfigureEndpoints(context);
    });
});
```

## Message Types

MassTransit requires message types to be [reference types.](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/reference-types)
So, you can use a `class`, `record` or `interface` to define a message.

You'll often create two types of messages: commands and events.

A command is an instruction to perform some action.
Commands are intended to have exactly one consumer.
Commands are expressed using verbs first: `CreateArticle`, `PublishArticle`, `ShareArticle`.

An event represents that something of significance happened.
Events can have one or many consumers.
Events should have a name in the past tense: `ArticleCreated`, `ArticlePublished`, `ArticleShared`.

Here's an example `ArticleCreated` message containing information about an article:

```csharp
public record ArticleCreated
{
    public Guid Id { get; init; }
    public string Title { get; init; }
    public string Content { get; init; }
    public DateTime CreatedOnUtc { get; init; }
}
```

Using `public set` or `public init` properties is recommended to avoid serialization problems with `System.Text.Json`.

## Publishing and Consuming Messages

You can use the `IPublishEndpoint` service to publish messages with MassTransit.
The framework routes the message to the appropriate queue or topic based on the message type.

```csharp {3,5}
app.MapPost("article", async (
    CreateArticleRequest request,
    IPublishEndpoint publishEndpoint) =>
{
    await publishEndpoint.Publish(new ArticleCreated
    {
        Id = Guid.NewGuid(),
        Title = request.Title,
        Content = request.Content,
        CreatedOnUtc = DateTime.UtcNow
    });

    return Results.Accepted();
});
```

To consume an `ArticleCreated` message, you need to implement the `IConsumer` interface.
The `IConsumer` has one `Consume` method where you place your business logic.
The consumer also gives you access to the `ConsumeContext`, which you can use to send further messages.

```csharp
public class ArticleCreatedConsumer(ApplicationDbContext dbContext)
    : IConsumer<ArticleCreatedEvent>
{
    public async Task Consume(ConsumeContext<ArticleCreatedEvent> context)
    {
        ArticleCreated message = context.Message;

        var article = new Article
        {
            Id = message.Id,
            Title = message.Title,
            Content = message.Content,
            CreatedOnUtc = message.CreatedOnUtc
        };

        dbContext.Add(article);

        await dbContext.SaveChangesAsync();
    }
}
```

MassTransit doesn't automatically know that the `ArticleCreatedConsumer` exists.
You have to configure the consumer when calling `AddMassTransit`.
The `AddConsumer` method registers the consumer type with the bus.

```csharp {5}
builder.Services.AddMassTransit(busConfigurator =>
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.AddConsumer<ArticleCreatedConsumer>();

    // ...
});
```

## Next Steps

**MassTransit** is an excellent messaging library I often use when building distributed applications.
The setup is straightforward, and there are only a few important abstractions.
You need to know abstraction for publishing messages (`IPublishEndpoint`) and consuming messages (`IConsumer`).
MassTransit takes care of doing the heavy lifting for you.

If you aren't already using it, I highly recommend adding MassTransit to your toolbox.

Here are some more practical **MassTransit learning resources**:

- [Request-Response pattern with MassTransit](https://youtu.be/NjsoykEOkrk)
- [RabbitMQ and MassTransit tutorial](https://youtu.be/CTKWFMZVIWA)
- [Microservices with RabbitMQ](https://youtu.be/MzC0PgYocmk)
- [Saga pattern with RabbitMQ](implementing-the-saga-pattern-with-rebus-and-rabbitmq)
- [Orchestration vs Choreography](orchestration-vs-choreography)
- [Messaging with Azure Service Bus](messaging-made-easy-with-azure-service-bus)

That's all for today.

See you next week.

---



<Sponsorships>

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w70)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Get started here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w70)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a C4 model diagramming tool for complex software systems.
Align your software engineering & product teams on technical decisions across the business.
[**Learn more here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

In the past year, I built and maintained a large public API.
The API has dozens of integrations, serving mainly mobile applications.

When your API is serving so many clients, breaking changes are expensive.

So, everything I implemented on the public API had to be planned.

Adding a new field? Forget about it.

Renaming existing fields? Forget about it.

If I wanted to introduce breaking changes, I had to version the API.

Today, I'll show you how to implement API versioning in ASP.NET Core.

## Why You Need API Versioning

API versioning allows your API to evolve independently from the clients using it.

Introducing breaking changes to your API is a bad user experience.
API versioning gives you a mechanism to avoid exposing breaking changes to clients.
Instead of making a breaking change, you introduce a new API version.

What's the definition of a breaking change?

This isn't an exhaustive list, but a few examples of breaking changes are:

- Removing or renaming APIs or API parameters
- Changing the behavior of existing APIs
- Changing the API response contract
- Changing the API error codes

You can decide what a breaking change means for your API.
For example, adding a new field to the response doesn't have to be a breaking change.

Let's see how to implement API versioning.

## Implementing API Versioning in ASP.NET Core

Let's start by installing two NuGet packages that we'll need to implement API versioning:

- `Asp.Versioning.Http`
- `Asp.Versioning.Mvc.ApiExplorer`

```powershell
Install-Package Asp.Versioning.Http
Install-Package Asp.Versioning.Mvc.ApiExplorer
```

This allows us to call `AddApiVersioning` and provide a delegate to configure the `ApiVersioningOptions`.

```csharp
builder.Services.AddApiVersioning(options =>
{
    options.DefaultApiVersion = new ApiVersion(1);
    options.ReportApiVersions = true;
    options.AssumeDefaultVersionWhenUnspecified = true;
    options.ApiVersionReader = ApiVersionReader.Combine(
        new UrlSegmentApiVersionReader(),
        new HeaderApiVersionReader("X-Api-Version"));
}).AddApiExplorer(options =>
{
    options.GroupNameFormat = "'v'V";
    options.SubstituteApiVersionInUrl = true;
});
```

Here's the explanation for the `ApiVersioningOptions` properties:

- `DefaultApiVersion` - Sets the default API version. Typically, this will be `v1.0`.
- `ReportApiVersions` - Reports the supported API versions in the `api-supported-versions` response header.
- `AssumeDefaultVersionWhenUnspecified` - Uses the `DefaultApiVersion` when the client didn't provide an explicit version.
- `ApiVersionReader` - Configures how to read the API version specified by the client. The default value is `QueryStringApiVersionReader`.

The `AddApiExplorer` method is helpful if you are using Swagger.
It will fix the endpoint routes and substitute the API version route parameter.

## Different Types of API Versioning

The most common ways to implement API versioning are:

- URL versioning: `https://localhost:5001/api/v1/workouts`
- Header versioning: `https://localhost:5001/api/workouts -H 'X-Api-Version: 1'`
- Query parameter versioning: `https://localhost:5001/api/workouts?api-version=1`

There are a few other ways to implement API versioning.
For example, using the `accept` or `content-type` headers.
But they aren't used often.

The `Asp.Versioning.Http` library has a few `IApiVersionReader` implementations:

- `UrlSegmentApiVersionReader`
- `HeaderApiVersionReader`
- `QueryStringApiVersionReader`
- `MediaTypeApiVersionReader`

[Microsoft's API versioning guidelines](https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md#12-versioning)
suggest using URL or query string parameter versioning.

I use URL versioning almost exclusively in the applications I'm developing.

## Versioning Controllers

To implement API versioning in ASP.NET controllers, you have to decorate the controller with the `ApiVersion` attribute.

The `ApiVersion` attribute allows you to specify which API versions that `WorkoutsController` supports.
In this case, the controller supports both `v1` and `v2`.
You use the `MapToApiVersion` attribute on the endpoints to specify the concrete API version.

The route parameter `v{v:apiVersion}` lets you specify the API version using `v1` or `v2` in the URL.

```csharp {1,2,6,13}
[ApiVersion(1)]
[ApiVersion(2)]
[Route("api/v{v:apiVersion}/workouts")]
public class WorkoutsController : ControllerBase
{
    [MapToApiVersion(1)]
    [HttpGet("{workoutId}")]
    public IActionResult GetWorkoutV1(Guid workoutId)
    {
        return Ok(new GetWorkoutByIdQuery(workoutId).Handle());
    }

    [MapToApiVersion(2)]
    [HttpGet("{workoutId}")]
    public IActionResult GetWorkoutV2(Guid workoutId)
    {
        return Ok(new GetWorkoutByIdQuery(workoutId).Handle());
    }
}
```

## Deprecating API Versions

If you want to deprecate an old API version, you can set the `Deprecated` property on the `ApiVersion` attribute.
The deprecated API versions will be reported using the `api-deprecated-versions` response header.

```csharp {1}
[ApiVersion(1, Deprecated = true)]
[ApiVersion(2)]
[Route("api/v{v:apiVersion}/workouts")]
public class WorkoutsController : ControllerBase
{
}
```

## Versioning Minimal APIs

Versioning Minimal APIs requires you to define an `ApiVersionSet`, which you'll pass to the endpoints.

- `NewApiVersionSet` - Creates a new `ApiVersionSetBuilder` that you can use to configure the `ApiVersionSet`.
- `HasApiVersion` - Indicates that the `ApiVersionSet` supports the specified `ApiVersion`.
- `ReportApiVersions`- Indicates that all APIs in the `ApiVersionSet` will report their versions.

After creating the `ApiVersionSet`, you must pass it to a Minimal API endpoint by calling `WithApiVersionSet`.
You can map to an explicit API version by calling `MapToApiVersion`.

```csharp {1,18,19}
ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .HasApiVersion(new ApiVersion(2))
    .ReportApiVersions()
    .Build();

app.MapGet("api/v{version:apiVersion}/workouts/{workoutId}", async (
    Guid workoutId,
    ISender sender,
    CancellationToken ct) =>
{
    var query = new GetWorkoutByIdQuery(workoutId);

    Result<WorkoutResponse> result = await sender.Send(query, ct);

    return result.Match(Results.Ok, CustomResults.Problem);
})
.WithApiVersionSet(apiVersionSet)
.MapToApiVersion(1);
```

Specifying the `ApiVersionSet` for each Minimal API endpoint can be cumbersome.
So you can define a route group and set the `ApiVersionSet` only once.
Route groups are also practical because they allow you to specify the route prefix.

```csharp {7,8}
ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .ReportApiVersions()
    .Build();

RouteGroupBuilder group = app
    .MapGroup("api/v{version:apiVersion}")
    .WithApiVersionSet(apiVersionSet);

group.MapGet("workouts", ...);
group.MapGet("workouts/{workoutId}", ...);
```

## Takeaway

API versioning is one of the best practices for designing modern APIs.
Consider implementing API versioning from the first release.
This makes it easier for clients to support future API versions.
And it gets your team used to managing breaking changes and versioning the API.

You can use the `Asp.Versioning.Http` library to add API versioning in ASP.NET Core.
Define the supported API versions, and start using them in your endpoints.

Remember to agree as a team what represents a breaking change.
This should be well documented in the team's API design guidelines.

My preferred way to implement API versioning is using URL versioning. It's simple and explicit.

And since this is the last issue for the year, I wish you a happy and prosperous new year.

Thanks for reading, and stay awesome!

---



<Sponsorships>

[**Telerik UI for Blazor**](https://www.telerik.com/blogs/net-8-support-aspnet-core-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)
and [**Telerik UI ASP.NET Core**](https://www.telerik.com/blogs/net-8-support-aspnet-core-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)
libraries allow developers to immediately utilize the advanced features and performance benefits of .NET 8.
[**Check out the key .NET 8 new features here.**](https://www.telerik.com/blogs/net-8-support-aspnet-core-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a C4 model diagramming tool for complex software systems.
Align your software engineering & product teams on technical decisions across the business.
[**Learn more here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

**Value Objects** are one of the building blocks of Domain-Driven Design.
DDD is a software development approach for solving problems in complex domains.

Value objects encapsulate a set of primitive values and related invariants.
A few examples of value objects are money and date range objects.
Money consists of an amount and currency.
A date range consists of start and end dates.

Today, I'll show you some best practices for implementing Value Objects.

## What are Value Objects?

Let's start with the definition from the Domain-Driven Design book:

> An object that represents a descriptive aspect of the domain with no conceptual identity is called a Value Object.
> Value Objects are instantiated to represent elements of the design that we care about only for what they are, not who or which they are.

_&mdash; [Eric Evans](http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215)_

Value objects are different from entities - they don't have a concept of identity.
They encapsulate primitive types in the domain and solve [primitive obsession.](https://refactoring.guru/smells/primitive-obsession)

There are two main qualities of Value Objects:

- They are immutable
- They have no identity

Another quality of value objects is structural equality.
Two value objects are equal if their values are the same.
This quality is the least important in practice.
However, there are cases where you want only some values to determine equality.

## Implementing Value Objects

The most important quality of value objects is immutability.
The values of a value object can't change once an object is created.
If you want to change an individual value, you need to replace the entire value object.

Here's a `Booking` entity with primitive values representing an address and the start and end dates of the booking.

```csharp
public class Booking
{
    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    public DateOnly StartDate { get; init; }
    public DateOnly EndDate { get; init; }
}
```

You can replace these primitive values with `Address` and `DateRange` value objects.

```csharp
public class Booking
{
    public Address Address { get; init; }

    public DateRange Period { get; init; }
}
```

But how do you implement value objects?

### C# Records

You can use C# [records](records-anonymous-types-non-destructive-mutation) to represent value objects.
Records are immutable by design, and they have structural equality.
We want both of these qualities for our value objects.

For example, you can represent an `Address` value object using a `record` with a [primary constructor.](https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/instance-constructors#primary-constructors)
The advantage of this approach is conciseness.

```csharp
public record Address(
    string Street,
    string City,
    string State,
    string Country,
    string ZipCode);
```

However, you lose this advantage when defining a private constructor.
This will happen when you want to enforce invariants while creating the value object.
Another issue with using records is avoiding value object invariants using the `with` expression.

```csharp
public record Address
{
    private Address(
        string street,
        string city,
        string state,
        string country,
        string zipCode)
    {
        Street = street;
        City = city;
        State = state;
        Country = country;
        ZipCode = zipCode;
    }

    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    public static Result<Address> Create(
        string street,
        string city,
        string state,
        string country,
        string zipCode)
    {
        // Check if the address is valid

        return new Address(street, city, state, country, zipCode);
    }
}
```

### Base Class

The alternative way to implement value objects is with a `ValueObject` base class.
The base class handles structural equality with the `GetAtomicValues` abstract method.
`ValueObject` implementations have to implement this method and define the equality components.

The advantage of using a `ValueObject` base class is that it's explicit.
It's clear which classes in your domain represent value objects.
Another advantage is being able to control the equality components.

Here's a `ValueObject` base class I use in my projects:

```csharp
public abstract class ValueObject : IEquatable<ValueObject>
{
    public static bool operator ==(ValueObject? a, ValueObject? b)
    {
        if (a is null && b is null)
        {
            return true;
        }

        if (a is null || b is null)
        {
            return false;
        }

        return a.Equals(b);
    }

    public static bool operator !=(ValueObject? a, ValueObject? b) =>
        !(a == b);

    public virtual bool Equals(ValueObject? other) =>
        other is not null && ValuesAreEqual(other);

    public override bool Equals(object? obj) =>
        obj is ValueObject valueObject && ValuesAreEqual(valueObject);

    public override int GetHashCode() =>
        GetAtomicValues().Aggregate(
            default(int),
            (hashcode, value) =>
                HashCode.Combine(hashcode, value.GetHashCode()));

    protected abstract IEnumerable<object> GetAtomicValues();

    private bool ValuesAreEqual(ValueObject valueObject) =>
        GetAtomicValues().SequenceEqual(valueObject.GetAtomicValues());
}
```

The `Address` value object implementation would look like this:

```csharp
public sealed class Address : ValueObject
{
    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    protected override IEnumerable<object> GetAtomicValues()
    {
        yield return Street;
        yield return City;
        yield return State;
        yield return Country;
        yield return ZipCode;
    }
}
```

## When To Use Value Objects?

I use value objects to solve primitive obsession and encapsulate domain invariants.
Encapsulation is an important aspect of any domain model.
You shouldn't be able to create a value object in an invalid state.

Value objects also give you type safety.
Take a look at this method signature:

```csharp
public interface IPricingService
{
    decimal Calculate(Apartment apartment, DateOnly start, DateOnly end);
}
```

Then, compare it to this method signature, where we added value objects.
You can see how the `IPricingService` with value objects is much more explicit.
You also get the benefit of type safety.
When compiling the code, value objects reduce the chance of errors creeping in.

```csharp
public interface IPricingService
{
    PricingDetails Calculate(Apartment apartment, DateRange period);
}
```

Here are a few more things you should consider to decide if you need value objects:

- **Complexity of invariants** - If enforcing complex invariants, consider using value objects
- **Number of primitives** - Value objects make sense when encapsulating many primitive values
- **Number of duplications** - If you need to enforce invariants only in a few places in the code, you can manage without value objects

## Persisting Value Objects With EF Core

Value objects are part of domain entities, and you need to save them in the database.

I'll show you how to use EF [Owned Types](https://learn.microsoft.com/en-us/ef/core/modeling/owned-entities)
and [Complex Types](https://devblogs.microsoft.com/dotnet/announcing-ef8-rc1/#complex-types-as-value-objects)
to persist value objects.

### Owned Types

[Owned Types](https://learn.microsoft.com/en-us/ef/core/modeling/owned-entities) can be configured by calling the `OwnsOne` method when configuring the entity.
This tells EF to persist the `Address` and `Price` value objects to the same table as the `Apartment` entity.
The value objects are represented with additional columns in the `apartments` table.

```csharp {5,7-13}
public void Configure(EntityTypeBuilder<Apartment> builder)
{
    builder.ToTable("apartments");

    builder.OwnsOne(property => property.Address);

    builder.OwnsOne(property => property.Price, priceBuilder =>
    {
        priceBuilder.Property(money => money.Currency)
            .HasConversion(
                currency => currency.Code,
                code => Currency.FromCode(code));
    });
}
```

A few more remarks about owned types:

- Owned types have a hidden key value
- No support for optional (nullable) owned types
- Owned collections are supported with `OwnsMany`
- Table splitting allows you to persist owned types separately

### Complex Types

[Complex Types](https://devblogs.microsoft.com/dotnet/announcing-ef8-rc1/#complex-types-as-value-objects) are a new EF feature available in .NET 8.
They aren't identified or tracked by a key value.
Complex types have to be part of an entity type.

Complex types are more appropriate for representing value objects with EF.

Here's how you can configure an `Address` value object as a complex type:

```csharp {5}
public void Configure(EntityTypeBuilder<Apartment> builder)
{
    builder.ToTable("apartments");

    builder.ComplexProperty(property => property.Address);
}
```

A few limitations for complex types:

- No support for collections
- No support for nullable values

## Takeaway

Value objects help design a rich domain model.
You can use them to solve primitive obsession and encapsulate domain invariants.
Value objects can reduce errors by preventing the instantiation of invalid domain objects.

You can use a `record` or `ValueObject` base class to represent value objects.
This should depend on your specific requirements and the complexity of your domain.
I use records by default unless I need some qualities of a `ValueObject` base class.
For example, a base class is practical when you want to control equality components.

More learning material about value objects:

- [Solving primitive obsession with value objects](https://youtu.be/P5CRea21R2E)
- [Using EF 8 Complex types for value objects](https://youtu.be/LhCD5CUSP6g)

Hope this was helpful.

See you next week.

---



<Sponsorships>

Do you need to [**export PDF/UA documents in C#?**](https://ironpdf.com/how-to/pdfa/?utm_source=milan&utm_medium=email&utm_campaign=milan)
PDF compliance in highly regulated industries can be extremely time-consuming.
Solution: [**IronPDF free trial.**](https://ironpdf.com/how-to/pdfa/?utm_source=milan&utm_medium=email&utm_campaign=milan)
Test your use case with full IronPDF functionality and access to documentation.

[**Product for Engineers**](https://newsletter.posthog.com/) from PostHog is a newsletter helping engineers improve their product skills.
You will get practical advice on building great products, lessons (and mistakes) from PostHog, and deep dives on top startups.
[**Subscribe here for FREE.**](https://newsletter.posthog.com/)

</Sponsorships>

[Serilog](https://serilog.net/) is a [structured logging](structured-logging-in-asp-net-core-with-serilog) library for .NET.

It's also my preferred logging library in the projects I'm developing.

Serilog supports many logging destinations called [Sinks.](https://github.com/serilog/serilog/wiki/Provided-Sinks)
The log destinations range from console and file sinks to managed logging services such as [Application Insights.](https://github.com/serilog-contrib/serilog-sinks-applicationinsights)

Today, I want to share 5 practical tips for better structured logging with Serilog.

## Use The Configuration System

There are two ways you can configure Serilog in ASP.NET Core:

- Fluent API
- Configuration system

The Fluent API allows you to write code and easily configure Serilog.
The downside is you are hardcoding your configuration.
Any configuration changes require deploying a new version.

I prefer using the ASP.NET configuration system to set up Serilog.
The benefit is you can change the logging configuration without redeploying your application.

You'll need to install the `Serilog.Settings.Configuration` library.

This allows you to configure Serilog using the configuration system:

```csharp
builder.Host.UseSerilog((context, loggerConfig) =>
    loggerConfig.ReadFrom.Configuration(context.Configuration));
```

Here's a Serilog configuration with [Console](https://github.com/serilog/serilog-sinks-console) and [Seq](https://github.com/datalust/serilog-sinks-seq) sinks.
We also configure a few [Serilog enrichers](https://github.com/serilog/serilog/wiki/Enrichment) to enrich application logs with extra information.

```json
{
  "Serilog": {
    "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.Seq"],
    "MinimiumLevel": {
      "Default": "Information",
      "Override": {
        "Micrsoft": "Information"
      }
    },
    "WriteTo": [
      { "Name": "Console" },
      {
        "Name": "Seq",
        "Args": { "serverUrl": "http://localhost:5341" }
      }
    ],
    "Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"]
  }
}
```

## Use Serilog Request Logging

You can install the `Serilog.AspNetCore` library to add Serilog logging for the ASP.NET Core request pipeline.
It adds ASP.NET's internal operations to the same Serilog sinks as your application events.

All you need to do is call the `UseSerilogRequestLogging` method:

```csharp
app.UseSerilogRequestLogging();
```

The `SourceContext` for these structured logs is `Serilog.AspNetCore.RequestLoggingMiddleware`.

Here's an example structured log produced by this middleware:

```json
{
  "@t": "2023-12-16T00:00:00.0000000Z",
  "@mt": "HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms",
  "@m": "HTTP POST /api/users responded 409 in 24.7928 ms",
  "@i": "37aa1435",
  "@r": ["24.7928"],
  "@tr": "61a449a8606fdb64e88d6c64b7b7354e",
  "@sp": "163ed90674cb12f6",
  "ConnectionId": "0HMVSP0L8FVEN",
  "CorrelationId": "0HMVSP0L8FVEN:0000000B",
  "Elapsed": 24.792778,
  "RequestId": "0HMVSP0L8FVEN:0000000B",
  "RequestMethod": "POST",
  "RequestPath": "/api/users",
  "SourceContext": "Serilog.AspNetCore.RequestLoggingMiddleware",
  "StatusCode": 409
}
```

## Enrich Your Logs With CorrelationId

How can you track all the logs belonging to the same request?

You can add a `CorrelationId` property to your structured logs.

This also works across multiple applications.
You need to pass the `CorrelationId` using an HTTP header.
For example, you could use a custom `X-Correlation-Id` header.

In the `RequestContextLoggingMiddleware`, I'm adding the `CorrelationId` to the Serilog [`LogContext`](https://github.com/serilog/serilog/wiki/Enrichment#the-logcontext).
This will make it available to all logs created during this application request.

```csharp
public class RequestContextLoggingMiddleware
{
    private const string CorrelationIdHeaderName = "X-Correlation-Id";
    private readonly RequestDelegate _next;

    public RequestContextLoggingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public Task Invoke(HttpContext context)
    {
        string correlationId = GetCorrelationId(context);

        using (LogContext.PushProperty("CorrelationId", correlationId))
        {
            return _next.Invoke(context);
        }
    }

    private static string GetCorrelationId(HttpContext context)
    {
        context.Request.Headers.TryGetValue(
            CorrelationIdHeaderName, out StringValues correlationId);

        return correlationId.FirstOrDefault() ?? context.TraceIdentifier;
    }
}
```

I like to create an extension method for adding the [middleware.](3-ways-to-create-middleware-in-asp-net-core)
The `UseRequestContextLogging` method will add the `RequestContextLoggingMiddleware` to the request pipeline.
Note that the order of registering middleware is important.
If you want the `CorrelationId` in all your logs, you want to place this middleware at the start.

```csharp
public static IApplicationBuilder UseRequestContextLogging(
    this IApplicationBuilder app)
{
    app.UseMiddleware<RequestContextLoggingMiddleware>();

    return app;
}
```

## Log Important Application Events

In general, I try to log important events in my application.
This includes current request information, errors, failures, unexpected values, branching points, etc.

I'm a proponent of using the [Result pattern](functional-error-handling-in-dotnet-with-the-result-pattern) to express application failures.
So, having a custom middleware to log request processing results is important.

Some developers prefer using exceptions to achieve the same functionality.
I disagree with this.
Using exceptions for flow control is a bad practice.
But still, don't forget to add a [global exception handler](global-error-handling-in-aspnetcore-8) for unhandled exceptions.

If you're using the [CQRS pattern with MediatR](cqrs-pattern-with-mediatr), you can easily add logging for all application requests.

In the `RequestLoggingPipelineBehavior` I'm pushing the `Error` property to the `LogContext`.
The error object is deconstructed into a JSON value in the structured log.
This lets me filter my logs based on the error details.

```csharp
internal sealed class RequestLoggingPipelineBehavior<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : class
    where TResponse : Result
{
    private readonly ILogger _logger;

    public RequestLoggingPipelineBehavior(ILogger logger)
    {
        _logger = logger;
    }

    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        string requestName = typeof(TRequest).Name;

        _logger.LogInformation(
            "Processing request {RequestName}", requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            _logger.LogInformation(
                "Completed request {RequestName}", requestName);
        }
        else
        {
            using (LogContext.PushProperty("Error", result.Error, true))
            {
                _logger.LogError(
                    "Completed request {RequestName} with error", requestName);
            }
        }

        return result;
    }
}
```

## Use Seq for Local Development

[Seq](https://datalust.co/seq) is a self-hosted search, analysis, and alerting server built for structured log data.
It's free to use for local development.
It offers advanced search and filtering capabilities on the structured log data.

You can spin up a Seq instance in a [Docker](https://www.docker.com/) container:

```yaml
version: '3.4'

services:
  seq:
    image: datalust/seq:latest
    container_name: seq
    environment:
      - ACCEPT_EULA=Y
    ports:
      - 5341:5341
      - 8081:80
```

You can start filtering data when you configure Serilog to write application logs to the Seq instance.

<Image src="blogs/mnw_068/seq.png" width={1920} height={910} />

## Summary

Structured logs follow follow the same structure.
And since structured logs are machine-readable, you can search them for specific information.
Structured logs provide more context and details about application errors.
They make it easier to identify and fix problems.

You can use Serilog's powerful `LogContext` to enrich your logs with a `CorrelationId`.
This lets you easily track all logs related to a single application request.

When you have structured logging set up, you'll want to search and analyze your logs.
Seq is an excellent tool for this that you can use for local development.

If you want to get started with Seq, check out my [beginner Seq tutorial.](https://youtu.be/mT8ZkXafuZk)

Thanks for reading, and stay awesome!

---



<Sponsorships>

Every [**Blazor application**](https://www.telerik.com/blogs/how-to-toggle-between-light-dark-modes-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)
needs dark mode.
But how do you toggle between light and dark modes?
Find out how to install a dark/light mode switch easily with [**Telerik UI for Blazor.**](https://www.telerik.com/blogs/how-to-toggle-between-light-dark-modes-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-general-awareness)

[**ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w67)
is a platform for building enterprise-grade ASP.NET Core applications.
It comes with production-ready components, modular architecture, and Domain-Driven Design.
[**Get started here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w67)

</Sponsorships>

Modular monoliths are an architectural approach that's becoming very popular.
They combine the benefits of modularity and monolithic design.

Modular monoliths try to solve the shortcomings of monolithic and microservice architectures.

One problem I often see with monolithic architectures is tight coupling between components.

This leads to dependencies between different parts of the system.

Modular monoliths enforce better architectural practices with well-defined module boundaries and [**communication patterns.**](modular-monolith-communication-patterns)

But one aspect you can't overlook is data isolation between modules.

Data isolation ensures that modules are independent and loosely coupled.

Today, I will show you four data isolation approaches for modular monoliths:

- Separate table
- Separate schema
- Separate database
- Different persistence

## Why Is Data Isolation Important?

Let's first understand why data isolation is important in a modular monolith architecture.

A modular monolith has strict rules for data integrity:

- Each module can only access its own tables
- No sharing of tables or objects between modules
- Joins are only allowed between tables of the same module

Modules inside a modular monolith should be self-contained.
Each module handles its own data.
Other modules can access that data using the module's public API.

What are the benefits of this design?

Keeping modules isolated from each other promotes modularity and loose coupling.
It makes it easier to introduce new changes to the system.
There are fewer unintended side effects when components are loosely coupled.

If you are using a relational database, you can still maintain referential integrity.
Removing the foreign keys when extracting tables is not a problem.

<Image src="blogs/mnw_067/monolith_components.png" width={1748} height={991} />

## Level 1 - Separate Table

The simplest solution is to have no isolation at the database level.
Tables for all modules live inside one database.
It's not easy to determine which tables belong to which module.

I'm only mentioning this approach for the sake of completeness.

But this approach works fine up to a particular application size.

However, the more tables you have, the harder it becomes to keep them isolated between modules.

You can improve this by adding logical isolation between tables.

<Image src="blogs/mnw_067/separate_table.png" width={1758} height={848} />

## Level 2 - Separate Schema

Grouping related tables in the database is a way to introduce logical isolation.
You can implement this using database schemas.
Each module has a unique schema containing the module's tables.

Now, it becomes easy to distinguish which module contains which tables.

An easy way to implement this is using [**multiple EF Core database contexts.**](using-multiple-ef-core-dbcontext-in-single-application)

You can also introduce rules to prevent querying data from other modules.
For example, you could implement this using [**architecture tests.**](enforcing-software-architecture-with-architecture-tests)

I always start with logical data isolation when building a modular monolith.

But what if this isn't enough?

<Image src="blogs/mnw_067/separate_schema.png" width={1748} height={848} />

## Level 3 - Separate Database

The next data isolation level is moving each module's data into separate databases.
This approach has more constraints than data isolation using schemas.

This is the way to go if you need strict data isolation rules between modules.
But, the downside is more operational complexity.
You have to manage infrastructure for multiple databases.

However, this is an excellent step toward [**extracting modules.**](monolith-to-microservices-how-a-modular-monolith-helps)

First, you move the tables of the module you want to extract into a separate database.
This also forces you to solve any database coupling problems between your modules.
You're ready to extract the module once you move the tables into a separate database.

Can we take the module data isolation further?

<Image src="blogs/mnw_067/separate_db.png" width={1742} height={848} />

## Level 4 - Different Persistence

Who says you have to use the same database type for all modules?

I work with relational (SQL) databases most of the time.
Relational databases are amazing and solve a wide range of problems.
But sometimes, a document or graph database is a much better solution.

The idea here is similar: you're doing data isolation using separate databases.

However, you can introduce a different database type to solve specific problems.
For example, you can use a relational database for one module.
And a graph or column-store database for another module.
You also have to maintain different persistence models in your application.

This could be a worthwhile tradeoff for your use case.
But it takes careful planning.

<Image src="blogs/mnw_067/separate_db_type.png" width={1742} height={848} />

## Summary

Modular monoliths are excellent if you don't need microservices right away.
You develop your application as a monolith with distinct boundaries inside the system.
You still have the flexibility to [**extract modules and move to microservices.**](monolith-to-microservices-how-a-modular-monolith-helps)
But you have faster development speed with a modular monolith.

Modules have to comply with a few rules.
They can only access their own tables.
They can't share tables with other modules.
And they can't directly query tables of other modules.
These rules help to enforce data isolation between modules.

But you still have to implement data isolation at the database level.

There are four options you can choose from:

- Separate table
- Separate schema
- Separate database
- Different persistence

I always go for logical isolation using schemas.
It's easy to implement and helps me understand my boundaries better.
Depending on the requirements, I can introduce separate databases later.

Hope this was helpful.

See you next week.

---



<Sponsorships>

[**Product for Engineers**](https://newsletter.posthog.com/) from PostHog is a newsletter helping engineers improve their product skills.
[**Subscribe for free**](https://newsletter.posthog.com/) to get curated advice on building great products, lessons (and mistakes) from PostHog, and deep dives on top startups.

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

Exceptions are for exceptional situations.
I even wrote about [**avoiding exceptions entirely.**](functional-error-handling-in-dotnet-with-the-result-pattern)

But they will inevitably happen in your applications, and you need to handle them.

You can implement a global exception handling mechanism or handle only specific exceptions.

ASP.NET Core gives you a few options on how to implement this. So which one should you choose?

Today, I want to show you an _old_ and _new_ way to handle exceptions in ASP.NET Core 8.

## Old Way: Exception Handling Midleware

The standard to implement exception handling in ASP.NET Core is using middleware.
Middleware allows you to introduce logic before or after executing HTTP requests.
You can easily extend this to implement exception handling.
Add a `try-catch` statement in the middleware and return an error HTTP response.

There are [**3 ways to create middleware**](3-ways-to-create-middleware-in-asp-net-core) in ASP.NET Core:

- Using [**request delegates**](3-ways-to-create-middleware-in-asp-net-core#adding-middleware-with-request-delegates)
- By [**convention**](3-ways-to-create-middleware-in-asp-net-core#adding-middleware-by-convention)
- [`IMiddleware`](3-ways-to-create-middleware-in-asp-net-core#adding-factory-based-middleware)

The convention-based approach requires you to define an `InvokeAsync` method.

Here's an `ExceptionHandlingMiddleware` defined by convention:

```csharp
public class ExceptionHandlingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<ExceptionHandlingMiddleware> _logger;

    public ExceptionHandlingMiddleware(
        RequestDelegate next,
        ILogger<ExceptionHandlingMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (Exception exception)
        {
            _logger.LogError(
                exception, "Exception occurred: {Message}", exception.Message);

            var problemDetails = new ProblemDetails
            {
                Status = StatusCodes.Status500InternalServerError,
                Title = "Server Error"
            };

            context.Response.StatusCode =
                StatusCodes.Status500InternalServerError;

            await context.Response.WriteAsJsonAsync(problemDetails);
        }
    }
}
```

The `ExceptionHandlingMiddleware` will catch any unhandled exception and return a [Problem Details](https://www.rfc-editor.org/rfc/rfc7807.html) response.
You can decide how much information you want to return to the caller.
In this example, I'm hiding the exception details.

You also need to add this middleware to the ASP.NET Core request pipeline:

```csharp
app.UseMiddleware<ExceptionHandlingMiddleware>();
```

## New Way: IExceptionHandler

[ASP.NET Core 8](https://learn.microsoft.com/en-us/aspnet/core/introduction-to-aspnet-core?view=aspnetcore-8.0)
introduces a new [`IExceptionHandler`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.diagnostics.iexceptionhandler?view=aspnetcore-8.0)
abstraction for managing exceptions.
The built-in exception handler middleware uses `IExceptionHandler` implementations to handle exceptions.

This interface has only one `TryHandleAsync` method.

`TryHandleAsync` attempts to handle the specified exception within the ASP.NET Core pipeline.
If the exception can be handled, it should return `true`.
If the exception can't be handled, it should return `false`.
This allows you to implement custom exception-handling logic for different scenarios.

Here's a `GlobalExceptionHandler` implementation:

```csharp
internal sealed class GlobalExceptionHandler : IExceptionHandler
{
    private readonly ILogger<GlobalExceptionHandler> _logger;

    public GlobalExceptionHandler(ILogger<GlobalExceptionHandler> logger)
    {
        _logger = logger;
    }

    public async ValueTask<bool> TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        _logger.LogError(
            exception, "Exception occurred: {Message}", exception.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status500InternalServerError,
            Title = "Server error"
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
```

## Configuring IExceptionHandler Implementations

You need two things to add an `IExceptionHandler` implementation to the ASP.NET Core request pipeline:

1. Register the `IExceptionHandler` service with dependency injection
2. Register the [`ExceptionHandlerMiddleware`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.diagnostics.exceptionhandlermiddleware?view=aspnetcore-8.0)
   with the request pipeline

You call the `AddExceptionHandler` method to register the `GlobalExceptionHandler` as a service.
It's registered with a [**singleton lifetime**](improving-aspnetcore-dependency-injection-with-scrutor).
So be careful about injecting services with a different lifetime.

I'm also calling `AddProblemDetails` to generate a [Problem Details](https://www.rfc-editor.org/rfc/rfc7807.html) response for common exceptions.

```csharp
builder.Services.AddExceptionHandler<GlobalExceptionHandler>();
builder.Services.AddProblemDetails();
```

You also need to call `UseExceptionHandler` to add the `ExceptionHandlerMiddleware` to the request pipeline:

```csharp
app.UseExceptionHandler();
```

## Chaining Exception Handlers

You can add multiple `IExceptionHandler` implementations, and they're called in the order they are registered.
A possible use case for this is using exceptions for flow control.

You can define custom exceptions like `BadRequestException` and `NotFoundException`.
They correspond with the HTTP status code you would return from the API.

Here's a `BadRequestExceptionHandler` implementation:

```csharp
internal sealed class BadRequestExceptionHandler : IExceptionHandler
{
    private readonly ILogger<BadRequestExceptionHandler> _logger;

    public GlobalExceptionHandler(ILogger<BadRequestExceptionHandler> logger)
    {
        _logger = logger;
    }

    public async ValueTask<bool> TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        if (exception is not BadRequestException badRequestException)
        {
            return false;
        }

        _logger.LogError(
            badRequestException,
            "Exception occurred: {Message}",
            badRequestException.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status400BadRequest,
            Title = "Bad Request",
            Detail = badRequestException.Message
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
```

And here's a `NotFoundExceptionHandler` implementation:

```csharp
internal sealed class NotFoundExceptionHandler : IExceptionHandler
{
    private readonly ILogger<NotFoundExceptionHandler> _logger;

    public GlobalExceptionHandler(ILogger<NotFoundExceptionHandler> logger)
    {
        _logger = logger;
    }

    public async ValueTask<bool> TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        if (exception is not NotFoundException notFoundException)
        {
            return false;
        }

        _logger.LogError(
            notFoundException,
            "Exception occurred: {Message}",
            notFoundException.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status404NotFound,
            Title = "Not Found",
            Detail = notFoundException.Message
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
```

You also need to register both exception handlers by calling `AddExceptionHandler`:

```csharp
builder.Services.AddExceptionHandler<BadRequestExceptionHandler>();
builder.Services.AddExceptionHandler<NotFoundExceptionHandler>();
```

The `BadRequestExceptionHandler` will execute first and try to handle the exception.
If the exception isn't handled, `NotFoundExceptionHandler` will execute next and attempt to handle the exception.

## Takeaway

Using middleware for exception handling is an excellent solution in ASP.NET Core.
However, it's great that we have new options using the `IExceptionHandler` interface.
I will use the new approach in ASP.NET Core 8 projects.

I'm very much against using exceptions for flow control.
Exceptions are a last resort when you can't continue normal application execution.
The [**Result pattern**](functional-error-handling-in-dotnet-with-the-result-pattern) is a better alternative.

Exceptions are also [extremely expensive](https://github.com/dotnet/aspnetcore/issues/46280#issuecomment-1527898867),
as David Fowler noted:

<Image src="blogs/mnw_066/fowler_comment.png" width={1360} height={500} />

If you want to get rid of exceptions in your code, [**check out this video.**](https://youtu.be/WCCkEe_Hy2Y)

Thanks for reading, and stay awesome!

---



<Sponsorships>

Are you seeking a more user-friendly Git experience?
Try [**Tower,**](https://www.git-tower.com/windows?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)
the most powerful Git client for Windows.
Currently on sale for Black Friday at a 50% discount!
[**Get Tower here.**](https://www.git-tower.com/windows?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)

[**Iron Software**](https://buzz.ironsoftware.com/cyber-rally-2023?utm_source=milan&utm_medium=email&utm_campaign=milan)
are hosting a Cyber Rally.
If you buy a Plus (or above) single product license you might walk away with a full Iron Suite at no extra cost.
[**Find out more here.**](https://buzz.ironsoftware.com/cyber-rally-2023?utm_source=milan&utm_medium=email&utm_campaign=milan)

</Sponsorships>

Refactoring is a technique for restructuring existing code without changing its behavior.
You can think of refactoring as a series of small code transformations.

One change (refactoring) does little.
But a sequence of refactors produces a significant transformation.

There's no better way to learn refactoring than practicing.

So I prepared a refactoring exercise for you.

Today I'm going to refactor some poorly written code.

And I'll show you 5 awesome refactoring techniques along the way:

- Extract method
- Extract interface
- Extract class
- Functional code
- Pushing logic down

## Starting Point

We will refactor the `CustomerService` below to try to improve the code.

I want to achieve three goals with this refactor.

Or rather, I want to improve three qualities of the `CustomerService`:

- Testability
- Readability
- Maintainability

To improve these qualities, we need to figure out what's preventing us from attaining them.

So, let's first understand what the `CustomerService` is doing on a high level:

- Validation of the input arguments
- Fetching the `Company` and creating a new `Customer`
- Calculating if the `Customer` has a credit limit and the amount
- Saving the `Customer` to the database if they meet a specific condition

You can see quite a few things are happening inside the `AddCustomer` method.

It's almost 100 lines of code, which reduces readability.

It's difficult to test because we can't control any external dependencies.

It's impossible to extend the behavior of the `CustomerService` without changing the code.

But we can fix all these problems.
Let me show you how.

```csharp
public class CustomerService
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (string.IsNullOrEmpty(firstName) || string.IsNullOrEmpty(lastName))
        {
            return false;
        }

        if (!email.Contains('@') && !email.Contains('.'))
        {
            return false;
        }

        var now = DateTime.Now;
        var age = now.Year - dateOfBirth.Year;
        if (dateOfBirth.Date > now.AddYears(-age))
        {
            age -= 1;
        }

        if (age < 21)
        {
            return false;
        }

        var companyRepository = new CompanyRepository();
        var company = companyRepository.GetById(companyId);

        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            Firstname = firstName,
            Surname = lastName
        };

        if (company.Type == "VeryImportantClient")
        {
            // Skip credit check
            customer.HasCreditLimit = false;
        }
        else if (company.Type == "ImportantClient")
        {
            // Do credit check and double credit limit
            customer.HasCreditLimit = true;
            using var creditService = new CustomerCreditServiceClient();

            var creditLimit = creditService.GetCreditLimit(
                customer.Firstname,
                customer.Surname,
                customer.DateOfBirth);

            creditLimit *= 2;
            customer.CreditLimit = creditLimit;
        }
        else
        {
            // Do credit check
            customer.HasCreditLimit = true;
            using var creditService = new CustomerCreditServiceClient();

            var creditLimit = creditService.GetCreditLimit(
                customer.Firstname,
                customer.Surname,
                customer.DateOfBirth);

            customer.CreditLimit = creditLimit;
        }

        if (customer.HasCreditLimit && customer.CreditLimit < 500)
        {
            return false;
        }

        var customerRepository = new CustomerRepository();
        customerRepository.AddCustomer(customer);

        return true;
    }
}
```

## Refactoring the Validation

The first part of the code validating input parameters is pretty concise.
It also follows the early return principle.

The validation consists of simple input validation and a calculation for the customer's age.

I would start with an **extract method** refactor to move the validation into one place.

```csharp
if (string.IsNullOrEmpty(firstName) || string.IsNullOrEmpty(lastName))
{
    return false;
}

if (!email.Contains('@') && !email.Contains('.'))
{
    return false;
}

var now = DateTime.Now;
var age = now.Year - dateOfBirth.Year;
if (dateOfBirth.Date > now.AddYears(-age))
{
    age -= 1;
}

if (age < 21)
{
    return false;
}
```

Calculating the age isn't part of the validation flow, so I'll extract that into the `CalculateAge` method.

```csharp
int CalculateAge(DateTime dateOfBirth, DateTime now)
{
    var age = now.Year - dateOfBirth.Year;
    if (dateOfBirth.Date > now.AddYears(-age))
    {
        age -= 1;
    }

    return age;
}
```

Then, I'll create the `IsValid` method to encapsulate all the validation rules.
Instead of writing many `if-else` statements, I can write a single `bool` expression.

I also introduced a `minimumAge` constant to improve readability.

You can see how the `CalculateAge` method helps simplify the validation check.

```csharp
bool IsValid(
    string firstName,
    string lastName,
    string email,
    DateTime dateOfBirth)
{
    const int minimumAge = 21;

    return !string.IsNullOrEmpty(firstName) &&
           !string.IsNullOrEmpty(lastName) &&
           (email.Contains('@') || email.Contains('.')) &&
           CalculateAge(dateOfBirth, DateTime.Now) >= minimumAge;
}
```

This simplifies the validation code in `AddCustomer` to:

```csharp
if (!IsValid(firstName, lastName, email, dateOfBirth))
{
    return false;
}
```

## Refactoring Towards Dependency Injection

The next problem I want to solve is introducing [dependency injection](improving-aspnetcore-dependency-injection-with-scrutor) to the `CustomerService`.

Dependency injection allows us to achieve **Inversion of Control (IoC).**
We depend only on interfaces at compile time and on implementations at run time.

The dependency injection pattern has a few important benefits.

You don't have to know how to initialize or dispose of external dependencies.

It also improves testability since you now depend on interfaces.
Interfaces can be mocked to make unit testing easier.

So let's update the `CustomerService` not to initialize the dependencies directly:

```csharp
var companyRepository = new CompanyRepository();

using var creditService = new CustomerCreditServiceClient();

var customerRepository = new CustomerRepository();
```

Instead, we will inject them as constructor arguments.
You can even use the [C# 12 primary constructor](https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/primary-constructors) feature.

```csharp {2-4}
public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CustomerCreditServiceClient creditService)
{
    // ...
}
```

The next step would be introducing interfaces for these dependencies.
This comes down to an **extract interface** refactor.

## Refactoring the Credit Limit Calculation

The credit limit calculation is the most complicated part of the code.
There are different business rules based on the company type.

I try to notice patterns in the code before refactoring.
So here are a few of my observations.

Multiple `if-else` statements based on the `Type` property make me wonder if I'll need to extend this in the future.
Adding a new rule would mean another `if-else` check.
The [strategy pattern](https://refactoring.guru/design-patterns/strategy) could be an alternative, but a `switch` statement will also work fine.

Another thing that stands out is the **code duplication** in the last two blocks.
This usually means I can do an **extract method** refactoring to reduce code duplication.

```csharp
if (company.Type == "VeryImportantClient")
{
    // Skip credit check
    customer.HasCreditLimit = false;
}
else if (company.Type == "ImportantClient")
{
    // Do credit check and double credit limit
    customer.HasCreditLimit = true;
    using var creditService = new CustomerCreditServiceClient();

    var creditLimit = creditService.GetCreditLimit(
        customer.Firstname,
        customer.Surname,
        customer.DateOfBirth);

    creditLimit *= 2;
    customer.CreditLimit = creditLimit;
}
else
{
    // Do credit check
    customer.HasCreditLimit = true;
    using var creditService = new CustomerCreditServiceClient();

    var creditLimit = creditService.GetCreditLimit(
        customer.Firstname,
        customer.Surname,
        customer.DateOfBirth);

    customer.CreditLimit = creditLimit;
}
```

The first thing I want to do is introduce an `enum` for the `CompanyType`.

This is a [clean coding principle](8-tips-to-write-clean-code) I often use.
It improves the readability and extensibility of the code.

```csharp
public enum CompanyType
{
    Regular = 0,
    ImportantClient = 1,
    VeryImportantClient = 2
}
```

The next thing that bothers me is that the credit limit calculation doesn't belong to the `CustomerService`.
It violates the [single responsibility principle](https://en.wikipedia.org/wiki/Single-responsibility_principle).

So I want to introduce a dedicated `CreditLimitCalculator` using an **extract class** refactoring.
I replaced the `if-else` statements with a [switch expression](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/operators/switch-expression) that I can easily extend in the future.

```csharp
public class CreditLimitCalculator(
    CustomerCreditServiceClient customerCreditServiceClient)
{
    public (bool HasCreditLimit, decimal? CreditLimit) Calculate(
        Customer customer,
        Company company)
    {
        return company.Type switch
        {
            CompanyType.VeryImportantClient => (false, null),
            CompanyType.ImportantClient => (true, GetCreditLimit(customer) * 2),
            _ => (true, GetCreditLimit(customer))
        };
    }

    private decimal GetCreditLimit(Customer customer)
    {
        return customerCreditServiceClient.GetCreditLimit(
            customer.FirstName,
            customer.LastName,
            customer.DateOfBirth);
    }
}
```

## Reviewing the Refactoring (So Far)

Let's pause momentarily and review the refactored version of the `CustomerService`.
I'm confident you will find it more readable and easier to understand.
We can easily test this class and verify that the behavior is correct.

I would usually stop the refactoring at this point, since I'm happy with the results.

But can we take this further?

```csharp
public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CreditLimitCalculator creditLimitCalculator)
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (!IsValid(firstName, lastName, email, dateOfBirth))
        {
            return false;
        }

        var company = companyRepository.GetById(companyId);

        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            FirstName = firstName,
            LastName = lastName
        };

        (customer.HasCreditLimit, customer.CreditLimit) =
            creditLimitCalculator.Calculate(customer, company);

        if (customer is { HasCreditLimit: true, CreditLimit: < 500 })
        {
            return false;
        }

        customerRepository.AddCustomer(customer);

        return true;
    }
}
```

## Taking It Further - Pushing Logic Down

This part is optional, but I want to show you how to simplify the `CustomerService` by pushing logic into the domain.

What if we moved the responsibility of creating a `Customer` into the class?

I often use the **static factory** pattern to implement this.
The caveat is I have to take a dependency on `CreditLimitCalculator`.
I'm trading off domain model purity to get business rules completeness.

I also added the `IsUnderCreditLimit` method to wrap the credit limit check.

```csharp
public class Customer
{
    // Properties omited

    public static Customer Create(
        Company company,
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        CreditLimitCalculator creditLimitCalculator)
    {
        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            FirstName = firstName,
            LastName = lastName
        };

        (customer.HasCreditLimit, customer.CreditLimit) =
            creditLimitCalculator.Calculate(customer, company);

        return customer;
    }

    public bool IsUnderCreditLimit() => HasCreditLimit && CreditLimit < 500;
}
```

This is what the `CustomerService` looks like now:

```csharp
public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CreditLimitCalculator creditLimitCalculator)
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (!IsValid(firstName, lastName, email, dateOfBirth))
        {
            return false;
        }

        var company = companyRepository.GetById(companyId);

        var customer = Customer.Create(
            company,
            firstName,
            lastName,
            email,
            dateOfBirth,
            creditLimitCalculator);

        if (customer.IsUnderCreditLimit())
        {
            return false;
        }

        customerRepository.AddCustomer(customer);

        return true;
    }
}
```

What do you think about this implementation?

## Next Steps

First of all, congrats on making it to the end.
This was a much longer newsletter issue than usual.
What do you think of this format?

Writing unit tests before starting the refactoring would be a great idea.
Unit tests will help detect any changes in behavior.

Remember, refactoring is transforming the existing code without changing the behavior.

Here are a few ideas on how you could further refactor the code:

- [**Strategy pattern**](https://refactoring.guru/design-patterns/strategy) for the credit limit calculation
- [**Result object**](functional-error-handling-in-dotnet-with-the-result-pattern) to represent the method status

If you want to try this refactoring exercise, you can find the complete [**source code here.**](https://github.com/m-jovanovic/refactoring-katas)

Hope this was helpful.

See you next week.

---



<Sponsorships>

[**Telerik UI for Blazor**](https://www.telerik.com/blogs/basic-themes-customization-telerik-ui-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-content-exposure)
makes it easy to tailor your app to your design specifications.
Its wide range of high-quality components, such as Grid, Charts, Forms, and Scheduler, speed up development significantly.
[**Learn more in this post.**](https://www.telerik.com/blogs/basic-themes-customization-telerik-ui-blazor?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-content-exposure)

[**ABP Commercial**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w64)
is a complete platform built on ABP Framework for enterprise-grade ASP.NET Core based web applications.
It comes with pre-built application modules, advanced startup templates, and premium support.
[**Check it out here.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w64)

</Sponsorships>

EF Core is my favorite ORM for .NET applications.
Yet, its many fantastic features sometimes go unnoticed.
For example, [**query splitting**](how-to-improve-performance-with-ef-core-query-splitting),
[**query filters**](how-to-use-global-query-filters-in-ef-core),
and interceptors.

EF interceptors are interesting because you can do powerful things with them.
For example, you can hook into materialization, handle optimistic concurrency errors, or add query hints.

The most practical use case is adding behavior when saving changes to the database.

Today I want to show you three unique use cases for EF Core interceptors:

- Audit logging
- Publishing domain events
- Persisting Outbox messages

## What are EF Interceptors?

[EF Core interceptors](https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/interceptors)
allow you to intercept, change, or suppress EF Core operations.
Every interceptor implements the `IInterceptor` interface.
A few common derived interfaces include `IDbCommandInterceptor`, `IDbConnectionInterceptor`, and `IDbTransactionInterceptor`.

The most popular one is the `ISaveChangesInterceptor`. It allows you to add behavior before or after saving changes.

Interceptors are registered for each `DbContext` instance when configuring the context.

```csharp
public interface IInterceptor
{
}
```

You don't have to implement these interfaces directly.
It's better to use concrete implementations and override the needed methods.

For example, I'll show you how to use the `SaveChangesInterceptor`.

## Audit Logging With EF Interceptors

An audit log of entity changes is a valuable feature in some applications.
You write additional audit information every time an entity is created or modified.
The audit log could also contain the complete before/after values, depending on your requirements.

However, let's use a simple example to make it easy to understand.

I have an `IAuditable` interface with two properties representing when an entity was created or modified.

```csharp
public interface IAuditable
{
    DateTime CreatedOnUtc { get; }

    DateTime? ModifiedOnUtc { get; }
}
```

Next, I'll implement an `UpdateAuditableInterceptor` interceptor to write the audit values.
It uses the `ChangeTracker` to find all `IAuditable` instances and sets the respective property value.

I want to highlight that I'm overriding the `SavingChangesAsync` method here.
`SavingChangesAsync` runs before the changes are saved in the database and any updates applied inside the `UpdateAuditableInterceptor`
are also part of the current database transaction.

This implementation can be easily extended to include the information about the current user.

```csharp
internal sealed class UpdateAuditableInterceptor : SaveChangesInterceptor
{
    public override ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            UpdateAuditableEntities(eventData.Context);
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private static void UpdateAuditableEntities(DbContext context)
    {
        DateTime utcNow = DateTime.UtcNow;
        var entities = context.ChangeTracker.Entries<IAuditable>().ToList();

        foreach (EntityEntry<IAuditable> entry in entities)
        {
            if (entry.State == EntityState.Added)
            {
                SetCurrentPropertyValue(
                    entry, nameof(IAuditable.CreatedOnUtc), utcNow);
            }

            if (entry.State == EntityState.Modified)
            {
                SetCurrentPropertyValue(
                    entry, nameof(IAuditable.ModifiedOnUtc), utcNow);
            }
        }

        static void SetCurrentPropertyValue(
            EntityEntry entry,
            string propertyName,
            DateTime utcNow) =>
            entry.Property(propertyName).CurrentValue = utcNow;
    }
}
```

## Publish Domain Events With EF Interceptors

Another use case for EF interceptors is [**publishing domain events.**](how-to-use-domain-events-to-build-loosely-coupled-systems)
Domain events are a DDD tactical pattern to create loosely coupled systems.

Domain events allow you to express side effects explicitly and provide a better separation of concerns in the domain.

You can create an `IDomainEvent` interface, which derives from `MediatR.INotification`.
This allows you to use the `IPublisher` to publish domain events and handle them asynchronously.

```csharp
using MediatR;

public interface IDomainEvent : INotification
{
}
```

Then, I'll create a `PublishDomainEventsInterceptor` that also inherits from `SaveChangesInterceptor`.
However, this time, we're using the `SavedChangesAsync` to publish the domain events _after_ saving changes in the database.

This has two important implications:

1. The entire workflow is now eventually consistent. Domain event handlers will save changes to the database after the original transaction.
2. If any domain event handlers fail, we risk failing the request even though the initial transaction was completed successfully.

You can make this process more reliable by using an [**Outbox.**](outbox-pattern-for-reliable-microservices-messaging)

```csharp
internal sealed class PublishDomainEventsInterceptor : SaveChangesInterceptor
{
    private readonly IPublisher _publisher;

    public PublishDomainEventsInterceptor(IPublisher publisher)
    {
        _publisher = publisher;
    }

    public override async ValueTask<int> SavedChangesAsync(
        SaveChangesCompletedEventData eventData,
        int result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            await PublishDomainEventsAsync(eventData.Context);
        }

        return result;
    }

    private async Task PublishDomainEventsAsync(DbContext context)
    {
        var domainEvents = context
            .ChangeTracker
            .Entries<Entity>()
            .Select(entry => entry.Entity)
            .SelectMany(entity =>
            {
                List<IDomainEvent> domainEvents = entity.DomainEvents;

                entity.ClearDomainEvents();

                return domainEvents;
            })
            .ToList();

        foreach (IDomainEvent domainEvent in domainEvents)
        {
            await _publisher.Publish(domainEvent);
        }
    }
}
```

## Store Outbox Messages With EF Interceptors

Instead of [**publishing domain events**](how-to-use-domain-events-to-build-loosely-coupled-systems) as part of the EF transaction, you can convert them to Outbox messages.

Here's an `InsertOutboxMessagesInterceptor` that does precisely this.

It overrides the `SavingChangesAsync` method.
Which means it runs inside the current EF transaction before saving changes.

The `InsertOutboxMessagesInterceptor` converts any domain events into an `OutboxMessage` and adds it to the respective `DbSet<OutboxMessage>`.
This means they will be saved to the database with any existing changes inside the same transaction.

This is an atomic operation.

Either everything succeeds or everything fails.

There's no in-between state like in the `PublishDomainEventsInterceptor`.

You can then create a background worker that will process the Outbox messages.

And this is how you implement the [**Outbox pattern**](outbox-pattern-for-reliable-microservices-messaging) with EF Core.

```csharp
using Newtonsoft.Json;

public sealed class InsertOutboxMessagesInterceptor : SaveChangesInterceptor
{
    private static readonly JsonSerializerSettings Serializer = new()
    {
        TypeNameHandling = TypeNameHandling.All
    };

    public override ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            InsertOutboxMessages(eventData.Context);
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private static void InsertOutboxMessages(DbContext context)
    {
        context
            .ChangeTracker
            .Entries<Entity>()
            .Select(entry => entry.Entity)
            .SelectMany(entity =>
            {
                List<IDomainEvent> domainEvents = entity.DomainEvents;

                entity.ClearDomainEvents();

                return domainEvents;
            })
            .Select(domainEvent => new OutboxMessage
            {
                Id = domainEvent.Id,
                OccurredOnUtc = domainEvent.OccurredOnUtc,
                Type = domainEvent.GetType().Name,
                Content = JsonConvert.SerializeObject(domainEvent, Serializer)
            })
            .ToList();

        context.Set<OutboxMessage>().AddRange(outboxMessages);
    }
}
```

## Configuring EF Interceptors Using Dependency Injection

EF interceptors should be lightweight and stateless.
You can add them to the `DbContext` by calling `AddInterceptors` and passing in the interceptor instances.

I like to configure the interceptors with Dependency Injection for two reasons:

- It allows me also to use DI in the interceptors (be mindful that they are singletons)
- To simplify adding the interceptors to the `DbContext` using `AddDbContext`

Here's how you can configure the `UpdateAuditableInterceptor` and `InsertOutboxMessagesInterceptor` with the `ApplicationDbContext`:

```csharp
services.AddSingleton<UpdateAuditableInterceptor>();
services.AddSingleton<InsertOutboxMessagesInterceptor>();

services.AddDbContext<IApplicationDbContext, ApplicationDbContext>(
    (sp, options) => options
        .UseSqlServer(connectionString)
        .AddInterceptors(
            sp.GetRequiredService<UpdateAuditableInterceptor>(),
            sp.GetRequiredService<InsertOutboxMessagesInterceptor>()));
```

## Closing Thoughts

Interceptors allow you to do almost anything with an EF Core operation.
But with great power comes great responsibility.
You should be mindful that interceptors have an impact on performance.
Calls to external services or handling events will slow down the operation.

Remember, you don't necessarily have to use EF interceptors.
You can achieve the same behavior by overriding the `SaveChangesAsync` method on the `DbContext` and adding your custom logic there.

I showed you a few practical use cases for EF interceptors in this week's issue.

But, if you want to see more examples, I have a few videos about:

- [**Auditing logging**](https://youtu.be/mAlO3OuoQvo)
- [**Publishing domain events**](https://youtu.be/AHzWJ_SMqLo)
- [**Implementing the Outbox pattern**](https://youtu.be/XALvnX7MPeo)

Thanks for reading, and stay awesome!

---



<Sponsorships>

[**Progress**](https://www.telerik.com/campaigns/discover-the-magic-of-dot-net-8-and-beyond-webinar?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=devcraft-webinar-standardization-dot-net-8)
is holding a free webinar: [**Discover the Magic of .NET 8 and Beyond!**](https://www.telerik.com/campaigns/discover-the-magic-of-dot-net-8-and-beyond-webinar?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=devcraft-webinar-standardization-dot-net-8)
Join the webinar and find out all the hot news in .NET 8.
[**Register today for FREE!**](https://www.telerik.com/campaigns/discover-the-magic-of-dot-net-8-and-beyond-webinar?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=devcraft-webinar-standardization-dot-net-8)

How do you [**choose the right PDF library?**](https://ironpdf.com/blog/case-studies/jeff-fritz-pdf-library-comparison/?utm_source=milan&utm_medium=email&utm_campaign=milan)
Microsoft MVP Jeff Fritz compared the most popular HTML-to-PDF libraries: iTextSharp, Syncfusion, Aspose, and IronPDF.
Take a look at his findings and [**find out which PDF library is the best.**](https://ironpdf.com/blog/case-studies/jeff-fritz-pdf-library-comparison/?utm_source=milan&utm_medium=email&utm_campaign=milan)

</Sponsorships>

Reporting is essential for business applications like e-commerce, shipping, fintech, etc.

One of the most popular document formats for reporting purposes is [PDF.](https://en.wikipedia.org/wiki/PDF)

PDF stands for Portable Document Format.
It's a file format to present documents (including text formatting and images) independently of application software, hardware, and operating systems.

Some common problems .NET developers will face when working with PDF files:

- Creating dynamic PDF documents
- Designing a consistent page layout
- Customizing fonts on printed documents

Today I want to show you a few interesting ways to generate PDF files in .NET.

## Creating PDF Files With QuestPDF

[QuestPDF](https://www.questpdf.com/) is an open-source .NET library for generating PDF documents.
It exposes a fluent API you can use to compose together many simple elements to create complex documents.
Unlike other libraries, it does not rely on HTML-to-PDF conversion.

Let's install the QuestPDF NuGet package:

```powershell
Install-Package QuestPDF
```

Here's how you can generate a simplified invoice with QuestPDF:

```csharp
using QuestPDF.Fluent;
using QuestPDF.Helpers;
using QuestPDF.Infrastructure;

Document.Create(container =>
{
    container.Page(page =>
    {
        page.Margin(50);
        page.Size(PageSizes.A4);
        page.PageColor(Colors.White);
        page.DefaultTextStyle(x => x.FontSize(16));

        page.Header()
            .AlignCenter()
            .Text("Invoice #: 2023-77")
            .SemiBold().FontSize(24).FontColor(Colors.Grey.Darken4);

        page.Content()
            .Table(table =>
            {
                table.ColumnsDefinition(columns =>
                {
                    columns.ConstantColumn(20);
                    columns.RelativeColumn();
                    columns.RelativeColumn();
                });

                table.Header(header =>
                {
                    header.Cell().Text("#");
                    header.Cell().Text("Product");
                    header.Cell().AlignRight().Text("Price");
                });

                foreach (var lineItem in lineItems)
                {
                    table.Cell().Text(lineItem.Index.ToString());
                    table.Cell().Text(lineItem.Name);
                    table.Cell().Text($"${lineItem.Price}");
                }
            });
    });
})
.GeneratePdf("invoice.pdf");;
```

What I like about QuestPDF:

- Fluent API
- Easy to use
- Good [documentation](https://www.questpdf.com/introduction.html)

What I don't like about QuestPDF:

- Having to write a lot of code to create documents
- Limited scope of features
- No HTML-to-PDF support

**Licensing**

QuestPDF is free for small companies and development use.
There's also a commercial license for larger companies.
You can check out the licensing details [here.](https://www.questpdf.com/license/)

## HTML to PDF Conversion With IronPDF

The more common approach for generating PDF files is using an HTML template.

My favorite library that supports this is [IronPDF.](https://ironpdf.com/)

IronPDF is a C# PDF library that allows for fast and efficient manipulation of PDF files.
It also has many valuable features, like [exporting to PDF/A format](https://ironpdf.com/how-to/pdfa/)
and [digitally signing PDF documents.](https://ironpdf.com/how-to/signing/)

But what's the idea behind using an HTML template?

First of all, you have more control over formatting the document.
You can use CSS to style the HTML markup, which will be applied when exporting to a PDF document.

An interesting implementation approach is using [ASP.NET Core MVC views](https://learn.microsoft.com/en-us/aspnet/core/mvc/views/overview?view=aspnetcore-7.0)
and the [Razor syntax.](https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0)
You can pass an object to the view at runtime to render dynamic HTML content.

I've used this approach with MVC views on a few projects with excellent results.

Let's start by installing the IronPDF NuGet package:

```powershell
Install-Package IronPdf
```

I'm using a strongly typed Razor view to define my markup.
The `InoviceViewModel` class is the model, and it's used to create dynamic content.

```tsx
@model ViewModels.InoviceViewModel

<div>Invoice number: @Model.InvoiceNumber</div>
<div>Invoice date: @Model.InvoiceDate</div>
<br/>
<span>Line items:</span>
<ul>
    @foreach(var lineItem in Model.LineItems)
    {
        <li>@lineItem.Name | @lineItem.Price</li>
    }
</ul>
```

Now, you need to use the IronPDF `ChromePdfRenderer` to convert the HTML to a PDF document.

```csharp
var html = ConvertRazorViewToHtml(invoice);

var renderer = new ChromePdfRenderer();

var pdf = renderer.RenderHtmlAsPdf(html);

pdf.SaveAs($"invoice-{invoice.InvoiceNumber}.pdf");
```

It really is that simple.

**Licensing**

IronPDF is free for development use and has multiple pricing tiers for commercial use that you can check out [here.](https://ironpdf.com/licensing/)

## Merging Multiple PDF Files

Another common requirement I've seen is merging multiple PDF files.
For example, you could implement a feature to merge the monthly receipts for the accounting department.

You can use the `PdfDocument.Merge` method to implement this.
It accepts a `PdfDocument` collection as the argument.
You'll first have to load the PDF documents into memory before merging them.

Here's an example:

```csharp
var pdfs = new List<PdfDocument>();

pdfs.Add(PdfDocument.FromFile("google-invoice.pdf"));
pdfs.Add(PdfDocument.FromFile("google-ads-invoice.pdf"));
pdfs.Add(PdfDocument.FromFile("converkit-invoice.pdf"));

PdfDocument mergedPdfDocument = PdfDocument.Merge(pdfs);

mergedPdfDocument.SaveAs("merged-invoices.pdf");
```

## Exporting PDF Files From an API

It's pretty straightforward to return a PDF file from an API endpoint in ASP.NET Core.

Minimal APIs have the `Results.File` method accepting either a file path, stream, or byte array.
You also need to specify the content type and an optional file name.
The [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types)
for PDF files is `application/pdf`.

Here's how you can return a PDF file from a byte array:

```csharp
app.MapGet("newsletter/download", () =>
{
    var renderer = new ChromePdfRenderer();

    var pdf = renderer.RenderHtmlAsPdf("<h1>The .NET Weekly</h1>");

    return Results.File(pdf.BinaryData, "application/pdf", "newsletter.pdf");
});
```

## Takeaway

Choosing which PDF library you will use in .NET is an important consideration to make.
And while pricing is a significant factor, the features you want to use will also dictate your choice.

QuestPDF is an excellent choice if you're looking for a (mostly) free option with rich features.
The library is constantly improved, and new features are being added.
However, it doesn't support HTML-to-PDF conversion and modifying existing documents.

IronPDF is the library I've used most often on commercial projects.
It has fantastic features for working with PDF files, with many customization options.
The HTML-to-PDF conversion works like a charm.

The hardest part is picking the right tool for the job.

So I hope this is helpful.

See you next week.

---



<Sponsorships>

Stop wasting time and focus on the skills employers are actually looking for today with [**Julio's .NET Backend Developer Roadmap.**](https://juliocasal.com/roadmap?utm_campaign=Milan-Sponsorship&utm_source=Milan-Newsletter&utm_medium=email)
A fully interactive guide with 110+ topics that link to the best videos or articles on the web.
[**Get it for free here.**](https://juliocasal.com/roadmap?utm_campaign=Milan-Sponsorship&utm_source=Milan-Newsletter&utm_medium=email)

[**Acorn**](https://www.acorn.io/) offers a free cloud sandbox environment where you can deploy applications with the power of Kubernetes.
Try it out by launching your own [**.NET App**](https://acorn.io/run/ghcr.io/infracloudio/dotnetcore-acorn:v7.%23.%23?ref=smw355&name=.Net+Development+Application&_gl=1*6tonkz*_ga*MTUzMDk5NjAwNC4xNjkwMzk2NTI2*_ga_JEWPGB0DW9*MTY5ODk0MDgxMy42OS4xLjE2OTg5NDI5NjIuMC4wLjA.)
in the cloud with Acorn.
Check out their [**.NET tutorial**](https://www.acorn.io/tutorials/developing-a-net-app-in-the-cloud-with-acorn)
and see where your first Acorn takes you!

</Sponsorships>

Layered architectures are the foundation of many software systems.
However, layered architectures organize the system around technical layers.
And the cohesion between layers is low.

What if you wanted to organize the system around features instead?

Minimize coupling between unrelated features and maximize coupling in a single feature.

Today I want to talk about **Vertical Slice Architecture**, which does precisely that.

## The Problem With Layered Architectures

Layered architectures organize the software system into layers or tiers.
Each of the layers is typically one project in your solution.
Some of the popular implementations are N-tier architecture or Clean architecture.

Layered architectures focus on separating the concerns of the various components.
This makes it easier to understand and maintain the software system.
And there are many benefits of [**structured software design,**](clean-architecture-and-the-benefits-of-structured-software-design)
such as maintainability, flexibility, and loose coupling.

<Image src="blogs/mnw_062/clean_architecture.png" width={3268} height={2006} />

However, layered architectures also impose constraints or rigid rules on your system.
The direction of dependencies between layers is pre-determined.

For example, in Clean Architecture:

- Domain should have no dependencies
- Application layer can reference the Domain
- Infrastructure can reference both Application and Domain
- Presentation can reference both Application and Domain

You end up having high coupling inside a layer and low coupling between layers.
This doesn't mean layered architectures are bad.
But it does mean you will have many abstractions between individual layers.
And more abstractions mean increased complexity because there are more components to maintain.

## What is Vertical Slice Architecture?

I first heard about [Vertical Slice Architecture](https://www.jimmybogard.com/vertical-slice-architecture) from Jimmy Bogard.
He's also the creator of some popular open-source libraries like [MediatR](https://github.com/jbogard/MediatR) and [Automapper.](https://github.com/AutoMapper/AutoMapper)

Vertical Slice Architecture was born from the pain of working with layered architectures.
They force you to make changes in many different layers to implement a feature.

Let's imagine what adding a new feature looks like in a layered architecture:

- Updating the domain model
- Modifying validation logic
- Creating a use case with MediatR
- Exposing an API endpoint from a controller

The cohesion is low because you are creating many files in different layers.

Vertical slices take a different approach:

> Minimize coupling between slices, and maximize coupling in a slice.

Here's how you can visualize vertical slices:

<Image
  src="blogs/mnw_062/vertical_slice_architecture.png"
  width={1920}
  height={1080}
/>

All the files for a single use case are grouped inside one folder.
So, the cohesion for a single use case is very high.
This simplifies the development experience.
It's easy to find all the relevant components for each feature since they are close together.

## Implementing Vertical Slices

If you're building an API, the system already breaks down into commands (POST/PUT/DELETE) and queries (GET).
By splitting the requests into commands and queries, you're getting the benefits of the [**CQRS pattern.**](cqrs-pattern-with-mediatr)

Vertical slices narrowly focus on a single feature.
This allows you to treat each use case separately and tailor the implementation to the specific requirements.
One vertical slice can use EF Core to implement a GET request.
Another vertical slice can use Dapper with raw SQL queries.

<Image src="blogs/mnw_062/vertical_slices.png" width={1920} height={1080} />

Another benefit of implementing vertical slices like this is:

> New features only add code, you're not changing shared code and worrying about side effects.

However, vertical slices have their own set of challenges.
Because you are implementing much of the business logic inside a single use case, you need to be able to spot code smells.
As the use case grows, it can end up doing too much.
You will have to refactor the code by [**pushing logic to the domain.**](refactoring-from-an-anemic-domain-model-to-a-rich-domain-model)

## Solution Structure With REPR Pattern

Layered architectures, such as Clean architecture, organize the solution across layers.
This results in a [**folder structure grouped by technical concerns.**](clean-architecture-folder-structure)

Vertical slice architecture, on the other hand, organizes the code around features or use cases.

An interesting approach to structuring APIs around features is using the [REPR pattern.](https://deviq.com/design-patterns/repr-design-pattern)
It stands for Request-EndPoint-Response.
This aligns perfectly with the idea of vertical slices.
You can achieve this with the MediatR library, for example.

The REPR pattern defines that web API endpoints should have three components:

- Request
- Endpoint
- Response

Here's an example solution structure in .NET.
You'll notice the `Features` folder, which contains the vertical slices.
Each vertical slice implements one API request (or use case).

```
🔗 RunTracker.API
|__ 📁 Database
|__ 📁 Entities
    |__ #️⃣ Activity.cs
    |__ #️⃣ Workout.cs
    |__ #️⃣ ...
|__ 📁 Features
    |__ 📁 Activities
        |__ 📁 GetActivity
            |__ #️⃣ ActivityResponse.cs
            |__ #️⃣ GetActivityEndpoint.cs
            |__ #️⃣ GetActivityQuery.cs
            |__ #️⃣ GetActivityQueryHandler.cs
        |__ 📁 CreateActivity
            |__ #️⃣ CreateActivity.cs
                |__ #️⃣ CreateActivity.Command.cs
                |__ #️⃣ CreateActivity.Endpoint.cs
                |__ #️⃣ CreateActivity.Handler.cs
                |__ #️⃣ CreateActivity.Validator.cs
    |__ 📁 Workouts
    |__ 📁 ...
|__ 📁 Middleware
|__ 📄 appsettings.json
|__ 📄 appsettings.Development.json
|__ #️⃣ Program.cs
```

A few more libraries for implementing the REPR pattern:

- [FastEndpoints](https://github.com/FastEndpoints/FastEndpoints)
- [ApiEndpoints](https://github.com/ardalis/ApiEndpoints)

## Next Steps

Some of you may not like the idea of grouping all the files related to a feature in a single folder.

However, there's a lot of value in grouping by features in general.
You don't have to implement vertical slices.
But you can apply this concept to your domain by grouping files around aggregates, for example.
This is the approach I show in [**Pragmatic Clean Architecture.**](pragmatic-clean-architecture)

I made a video about [**Vertical Slice Architecture,**](https://youtu.be/msjnfdeDCmo)
showing how to implement the concepts discussed in today's issue. Check it out [**here.**](https://youtu.be/msjnfdeDCmo)

Thanks for reading, and stay awesome!

---



<Sponsorships>

[**Progress**](https://www.telerik.com/campaigns/maui/migration-to-dot-net-maui?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-ebook-mnm-na)
prepared a free eBook for you - [**Migration to .NET MAUI.**](https://www.telerik.com/campaigns/maui/migration-to-dot-net-maui?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-ebook-mnm-na)
Do you plan to start your first cross-platform app with .NET MAUI?
You're in the right place.
This eBook will help you examine the benefits of using .NET MAUI for the dev experience,
the app and its end users, plus any downsides.
[**Grab the free MAUI eBook here.**](https://www.telerik.com/campaigns/maui/migration-to-dot-net-maui?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-ebook-mnm-na)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

How should you handle errors in your code?

This has been a topic of many discussions, and I want to share my opinion.

One school of thought suggests using exceptions for flow control.
This is not a good approach because it makes the code harder to reason about.
The caller must know the implementation details and which exceptions to handle.

Exceptions are for exceptional situations.

Today, I want to show you how to implement error handling using the **Result pattern.**

It's a functional approach to error handling, making your code more expressive.

## Exceptions For Flow Control

Using exceptions for flow control is an approach to implement the **fail-fast** principle.

As soon as you encounter an error in the code, you throw an exception —
effectively terminating the method, and making the caller responsible for handling the exception.

The problem is the caller must know which exceptions to handle.
And this isn't obvious from the method signature alone.

Another common use case is throwing exceptions for validation errors.

Here's an example in the `FollowerService`:

```csharp
public sealed class FollowerService
{
    private readonly IFollowerRepository _followerRepository;

    public FollowerService(IFollowerRepository followerRepository)
    {
        _followerRepository = followerRepository;
    }

    public async Task StartFollowingAsync(
        User user,
        User followed,
        DateTime createdOnUtc,
        CancellationToken cancellationToken = default)
    {
        if (user.Id == followed.Id)
        {
            throw new DomainException("Can't follow yourself");
        }

        if (!followed.HasPublicProfile)
        {
            throw new DomainException("Can't follow non-public profile");
        }

        if (await _followerRepository.IsAlreadyFollowingAsync(
                user.Id,
                followed.Id,
                cancellationToken))
        {
            throw new DomainException("Already following");
        }

        var follower = Follower.Create(user.Id, followed.Id, createdOnUtc);

        _followerRepository.Insert(follower);
    }
}
```

## Use Exceptions for Exceptional Situations

A rule of thumb I follow is to use exceptions for exceptional situations.
Since you already expect potential errors, why not make it explicit?

You can group all application errors into two groups:

- Errors you know how to handle
- Errors you don't know how to handle

Exceptions are an excellent solution for the errors you don't know how to handle.
And you should catch and handle them at the lowest level possible.

What about the errors you know how to handle?

You can handle them in a functional way with the **Result pattern.**
It's explicit and clearly expresses the intent that the method can fail.
The drawback is the caller has to manually check if the operation failed.

## Expressing Errors Using the Result Pattern

The first thing you will need is an `Error` class to represent application errors.

- `Code` - unique name for the error in the application
- `Description` - contains developer-friendly details about the error

```csharp
public sealed record Error(string Code, string Description)
{
    public static readonly Error None = new(string.Empty, string.Empty);
}
```

Then, you can implement the `Result` class using the `Error` to describe the failure.
This implementation is very bare-bones, and you could add many more features.
In most cases, you also need a generic `Result<T>` class, which will wrap a value inside.

Here's what the `Result` class looks like:

```csharp
public class Result
{
    private Result(bool isSuccess, Error error)
    {
        if (isSuccess && error != Error.None ||
            !isSuccess && error == Error.None)
        {
            throw new ArgumentException("Invalid error", nameof(error));
        }

        IsSuccess = isSuccess;
        Error = error;
    }

    public bool IsSuccess { get; }

    public bool IsFailure => !IsSuccess;

    public Error Error { get; }

    public static Result Success() => new(true, Error.None);

    public static Result Failure(Error error) => new(false, error);
}
```

The only way to create a `Result` instance is by using static methods:

- `Success` - creates a success result
- `Failure` - creates a failure result with the specified `Error`

If you want to avoid building your own `Result` class, take a look at the [FluentResults](https://github.com/altmann/FluentResults) library.

## Applying the Result Pattern

Now that we have the `Result` class let's see how to apply it in practice.

Here's a refactored version of the `FollowerService`.
Notice a few things:

- No more throwing exceptions
- The `Result` return type is explicit
- It's clear which errors the method returns

Another benefit of error handling using the **Result pattern** is that it's easier to test.

```csharp {10,18,23,31,38}
public sealed class FollowerService
{
    private readonly IFollowerRepository _followerRepository;

    public FollowerService(IFollowerRepository followerRepository)
    {
        _followerRepository = followerRepository;
    }

    public async Task<Result> StartFollowingAsync(
        User user,
        User followed,
        DateTime utcNow,
        CancellationToken cancellationToken = default)
    {
        if (user.Id == followed.Id)
        {
            return Result.Failure(FollowerErrors.SameUser);
        }

        if (!followed.HasPublicProfile)
        {
            return Result.Failure(FollowerErrors.NonPublicProfile);
        }

        if (await _followerRepository.IsAlreadyFollowingAsync(
                user.Id,
                followed.Id,
                cancellationToken))
        {
            return Result.Failure(FollowerErrors.AlreadyFollowing);
        }

        var follower = Follower.Create(user.Id, followed.Id, utcNow);

        _followerRepository.Insert(follower);

        return Result.Success();
    }
}
```

## Documenting Application Errors

You can use the `Error` class to document all possible errors in your application.

One approach is to create a static class called `Errors`.
It will have nested classes inside containing the specific errors.
The usage would look like `Errors.Followers.NonPublicProfile`.

However, the approach I like to use is to create a specific class containing the errors.

Here's the `FollowerErrors` class documenting the possible errors for the `Follower` entity:

```csharp
public static class FollowerErrors
{
    public static readonly Error SameUser = new Error(
        "Followers.SameUser", "Can't follow yourself");

    public static readonly Error NonPublicProfile = new Error(
        "Followers.NonPublicProfile", "Can't follow non-public profiles");

    public static readonly Error AlreadyFollowing = new Error(
        "Followers.AlreadyFollowing", "Already following");
}
```

Instead of static fields, you can also use static methods returning an error.
You would call this method with a concrete argument to get an `Error` instance.

```csharp
public static class FollowerErrors
{
    public static Error NotFound(Guid id) => new Error(
        "Followers.NotFound", $"The follower with Id '{id}' was not found");
}
```

## Converting Results Into API Responses

The `Result` object will eventually reach the Minimal API (or controller) endpoint in ASP.NET Core.
Minimal APIs return an `IResult` response, and controllers return an `IActionResult` response.
Regardless, you must convert the `Result` instance into a valid API response.

The straightforward approach is checking the `Result` state and returning an HTTP response.
Here's an example where we check the `Result.IsFailure` flag:

```csharp
app.MapPost(
    "users/{userId}/follow/{followedId}",
    (Guid userId, Guid followedId, FollowerService followerService) =>
    {
        var result = await followerService.StartFollowingAsync(
            userId,
            followedId,
            DateTime.UtcNow);

        if (result.IsFailure)
        {
            return Results.BadRequest(result.Error);
        }

        return Results.NoContent();
    });
```

However, this is an excellent opportunity for a more functional approach.
You can implement the `Match` extension method to provide a callback for each `Result` state.
The `Match` method will execute the respective callback and return the result.

Here's the implementation of `Match`:

```csharp
public static class ResultExtensions
{
    public static T Match<T>(
        this Result result,
        Func<T> onSuccess,
        Func<Error, T> onFailure)
    {
        return result.IsSuccess ? onSuccess() : onFailure(result.Error);
    }
}
```

And this is how you would use the `Match` method in a Minimal API endpoint:

```csharp
app.MapPost(
    "users/{userId}/follow/{followedId}",
    (Guid userId, Guid followedId, FollowerService followerService) =>
    {
        var result = await followerService.StartFollowingAsync(
            userId,
            followedId,
            DateTime.UtcNow);

        return result.Match(
            onSuccess: () => Results.NoContent(),
            onFailure: error => Results.BadRequest(error));
    });
```

Much more concise. Don't you think so?

## Summary

If you take one thing with you from this week's issue, it should be this: exceptions are for exceptional situations.
Moreover, you should only use exceptions for errors you don't know how to handle.
In all other cases, expressing the error clearly with the **Result pattern** is more valuable.

Using the `Result` class allows you to:

- Express the intent that a method _could_ fail
- Encapsulate an application error inside
- Provide a functional way to handle errors

Additionally, you can document all application errors with the `Error` class.
This is helpful for developers to know which errors they need to handle.

You can even convert this to actual _documentation_.
For example, I wrote a simple program that scans the project for all `Error` fields.
It then converts this into a table format and uploads it to a Confluence page.

So I encourage you to try the **Result pattern** and see how it can improve your code.

See you next week.

---



<Sponsorships>

Will .NET become #1 for cross-platform desktop development?
With [**DotNetBrowser's**](https://links.teamdev.com/avalonia-case-study-mj) recent update, Avalonia UI finally gets the much-awaited cross-platform web view control.
Check out the story and [**try it for free now.**](https://links.teamdev.com/avalonia-case-study-mj)

Get all the latest .NET news once a month in the [**.NET Annotated newsletter.**](https://www.jetbrains.com/lp/dotnet-annotated/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetannotated)
It is hosted by Rachel Appel, a Developer Advocate at JetBrains, and guest curators from the wider .NET community.
Feel free to suggest an article too because sharing is caring! [**Join free here.**](https://www.jetbrains.com/lp/dotnet-annotated/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetannotated)

</Sponsorships>

Today I want to show you how to use the **CQRS** pattern to build fast and scalable applications.

The CQRS pattern separates the writes and reads in the application.

This separation can be logical or physical and has many benefits:

- Complexity management
- Improved performance
- Scalability
- Flexibility
- Security

I'm also going to show you how to implement CQRS in your application using MediatR.

But first, we have to understand what CQRS is.

## What Exactly is CQRS?

[CQRS](https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs) stands for **Command Query Responsibility Segregation**.
The CQRS pattern uses separate models for reading and updating data.
The benefits of using CQRS are complexity management, improved performance, scalability, and security.

The standard approach for working with a database is using the same model to query and update data.
This is simple and works great for most CRUD operations.
However, in more complex applications, it becomes difficult to maintain.
On the write side, you could have complex business logic and validation in the model.
On the read side, you may need to perform many different queries.

Also, consider how we create the data model.
Applying SQL data modeling best practices will give you a normalized database.
This is generally fine, but it's optimized for writing.

Having separate models for commands and queries allows you to scale them independently.
The separation could be logical while using the same database.
You could split the subsystems for commands and queries into separate services.
And you can even have multiple databases optimized for writing or reading data.

## How Is It Different From CQS?

[CQS](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation) stands for **Command Query Separation**.
It's a term coined by Bertrand Meyer in his book [Object-Oriented Software Construction.](https://en.wikipedia.org/wiki/Object-Oriented_Software_Construction)

The basic premise of CQS is splitting an object's methods into **Commands** and **Queries**.

- **Commands**: Change the state of a system but don't return a value
- **Queries**: Return a value and don't change the state of the system (no side effects)

This doesn't mean a command can never return a value.
A typical example is popping a value from a stack.
It returns a value and changes the state of the system.
But the intent is what matters here.

CQS is a _principle._
You can follow this principle if it makes sense, but be pragmatic.

CQRS is the evolution of CQS.
CQRS works on the architectural level.
At the same time, CQS works on the method (or class) level.

## Many Flavors of CQRS

Here's a high-level overview of a CQRS system using multiple databases.
Commands update the write database.
Then, you need to synchronize the updates with the read database.
This introduces eventual consistency to CQRS systems.

Eventual consistency significantly increases the complexity of your application.
You must consider what happens if the synchronization process fails, and have a fault tolerance strategy.

<Image
  src="blogs/mnw_060/cqrs.png"
  width={1207}
  height={708}
  alt="Diagram of a system using CQRS with two databases."
/>

There are many flavors of this approach:

- SQL database on the write side and NoSQL database (for example, [RavenDB](https://ravendb.net/)) on the read side
- Event sourcing on the write side and NoSQL database on the read side
- Using Redis or some other distributed cache on the read side

Separating the models for updating and reading data allows you to choose the best database for your requirements.

## Logical CQRS Architecture

How do you apply the CQRS pattern to your system?
I prefer using [MediatR.](https://github.com/jbogard/MediatR)

MediatR implements the [mediator pattern](https://refactoring.guru/design-patterns/mediator) to solve a simple problem - decoupling the in-process sending of messages from handling messages.

You can extend MediatR's `IRequest` interface with a custom `ICommand` and `IQuery` abstraction.
This allows you to define commands and queries in your system explicitly.

On the write side, I typically use [EF Core](https://learn.microsoft.com/en-us/ef/core/) and a rich domain model to encapsulate business logic.
The command flow uses EF to load an entity into memory, execute the domain logic, and save the changes to the database.

On the read side, I want as little indirection as possible.
Using [Dapper](https://github.com/DapperLib/Dapper) with raw SQL queries is an excellent choice.
You can also create views in the database and query them.
Alternatively, you could use EF Core to execute queries with projections.

<Image
  src="blogs/mnw_060/cqrs_application.png"
  width={1578}
  height={1182}
  alt="Diagram of an application using CQRS on the architectural level."
/>

## Implementing CQRS With MediatR

Implementing CQRS with MediatR has two components:

- Defining your command or query class
- Implementing the respective command or query handler

I made an in-depth video explaining this process, and you can [watch it here.](https://youtu.be/vdi-p9StmG0)

You use the `ISender` interface to `Send` the command or query.
MediatR takes care of routing the command or query to the respective handler.

The request will pass through the _request pipeline_.
It's a wrapper around each request, and you can use it to solve cross-cutting concerns with `IPipelineBehavior`.
For example, you can implement [validation for commands with FluentValidation.](cqrs-validation-with-mediatr-pipeline-and-fluentvalidation)

```csharp
[ApiController]
[Route("api/bookings")]
public class BookingsController : ControllerBase
{
    private readonly ISender _sender;

    public BookingsController(ISender sender)
    {
        _sender = sender;
    }

    [HttpPut("{id}/confirm")]
    public async Task<IActionResult> ConfirmBooking(
        Guid id,
        CancellationToken cancellationToken)
    {
        var command = new ConfirmBookingCommand(id);

        var result = await _sender.Send(command, cancellationToken);

        if (result.IsFailure)
        {
            return BadRequest(result.Error);
        }

        return NoContent();
    }
}

```

Here's an example of a command handler with repositories and a rich domain model:

```csharp
internal sealed class ConfirmBookingCommandHandler
    : ICommandHandler<ConfirmBookingCommand>
{
    private readonly IDateTimeProvider _dateTimeProvider;
    private readonly IBookingRepository _bookingRepository;
    private readonly IUnitOfWork _unitOfWork;

    public ConfirmBookingCommandHandler(
        IDateTimeProvider dateTimeProvider,
        IBookingRepository bookingRepository,
        IUnitOfWork unitOfWork)
    {
        _dateTimeProvider = dateTimeProvider;
        _bookingRepository = bookingRepository;
        _unitOfWork = unitOfWork;
    }

    public async Task<Result> Handle(
        ConfirmBookingCommand request,
        CancellationToken cancellationToken)
    {
        var booking = await _bookingRepository.GetByIdAsync(
            request.BookingId,
            cancellationToken);

        if (booking is null)
        {
            return Result.Failure(BookingErrors.NotFound);
        }

        var result = booking.Confirm(_dateTimeProvider.UtcNow);

        if (result.IsFailure)
        {
            return result;
        }

        await _unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
```

Here's an example of a query handler that uses Dapper and raw SQL:

```csharp
internal sealed class SearchApartmentsQueryHandler
    : IQueryHandler<SearchApartmentsQuery, IReadOnlyList<ApartmentResponse>>
{
    private static readonly int[] ActiveBookingStatuses =
    {
        (int)BookingStatus.Reserved,
        (int)BookingStatus.Confirmed,
        (int)BookingStatus.Completed
    };

    private readonly ISqlConnectionFactory _sqlConnectionFactory;

    public SearchApartmentsQueryHandler(
        ISqlConnectionFactory sqlConnectionFactory)
    {
        _sqlConnectionFactory = sqlConnectionFactory;
    }

    public async Task<Result<IReadOnlyList<ApartmentResponse>>> Handle(
        SearchApartmentsQuery request,
        CancellationToken cancellationToken)
    {
        if (request.StartDate > request.EndDate)
        {
            return new List<ApartmentResponse>();
        }

        using var connection = _sqlConnectionFactory.CreateConnection();

        const string sql = """
            SELECT
                a.id AS Id,
                a.name AS Name,
                a.description AS Description,
                a.price_amount AS Price,
                a.price_currency AS Currency,
                a.address_country AS Country,
                a.address_state AS State,
                a.address_zip_code AS ZipCode,
                a.address_city AS City,
                a.address_street AS Street
            FROM apartments AS a
            WHERE NOT EXISTS
            (
                SELECT 1
                FROM bookings AS b
                WHERE
                    b.apartment_id = a.id AND
                    b.duration_start <= @EndDate AND
                    b.duration_end >= @StartDate AND
                    b.status = ANY(@ActiveBookingStatuses)
            )
            """;

        var apartments = await connection
            .QueryAsync<ApartmentResponse, AddressResponse, ApartmentResponse>(
                sql,
                (apartment, address) =>
                {
                    apartment.Address = address;

                    return apartment;
                },
                new
                {
                    request.StartDate,
                    request.EndDate,
                    ActiveBookingStatuses
                },
                splitOn: "Country");

        return apartments.ToList();
    }
}
```

## Closing Thoughts

Separating commands and queries can improve performance and scalability in the long run.
You can optimize commands and queries differently based on your requirements.

Commands encapsulate complex business logic and validation.
Using EF Core and a rich domain model is an excellent solution.

Queries are all about performance, so you want to use what's fastest.
This could be raw SQL queries with Dapper, EF Core projections, or Redis.

If you want the system I use to build scalable applications with CQRS and MediatR, check out [**Pragmatic Clean Architecture.**](/pragmatic-clean-architecture)

Stay awesome!

---



<Sponsorships>

[**Progress**](https://www.telerik.com/campaigns/blazor/planning-a-blazor-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-ebook-mnm-na)
prepared a free eBook for you - [**Planning a Blazor Application.**](https://www.telerik.com/campaigns/blazor/planning-a-blazor-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-ebook-mnm-na)
This eBook documents a high-level outline of what developers need to consider when choosing Blazor: migration strategies, target platforms, tooling, testing and UI.
[**Get the Blazor eBook here.**](https://www.telerik.com/campaigns/blazor/planning-a-blazor-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-ebook-mnm-na)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

Dependency injection (DI) is one of the most exciting features of ASP.NET Core.
It helps us build more testable and maintainable applications.
However, ASP.NET Core's built-in DI system sometimes needs a little help to achieve more advanced scenarios.

So I want to introduce you to a powerful library for enhancing your ASP.NET Core DI - [Scrutor.](https://www.nuget.org/packages/Scrutor)

If you're an ASP.NET Core developer, you're already familiar with Dependency Injection.
It's a fundamental part of building modular and maintainable applications.

Let's explore how Scrutor can simplify and enhance your DI setup.

## What is Dependency Injection?

[Dependency Injection](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection?view=aspnetcore-7.0)
is a software design pattern used in ASP.NET Core to achieve the [Inversion of Control (IOC)](https://learn.microsoft.com/en-us/dotnet/architecture/modern-web-apps-azure/architectural-principles#dependency-inversion)
principle.
This promotes loose coupling and makes your code more testable, maintainable, and extensible.

DI allows you to inject dependencies into your classes rather than create them within the class.
The framework takes care of providing the required instances at runtime.
It also manages the disposal of these dependencies based on the service lifetime.

Here's an example of combining constructor and method injection in a controller:

```csharp
[ApiController]
[Route("api/activities")]
public class ActivitiesController : ControllerBase
{
    private readonly ILogger<ActivitiesController> _logger;

    // Constructor injection
    public ActivitiesController(ILogger<ActivitiesController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public async Task<IActionResult> Get(ISender sender) // Method injection
    {
        var activities = await sender.Send(new GetActivitiesQuery());

        return Ok(activities);
    }
}
```

## Service Lifetimes in ASP.NET Core

Before we dive into Scrutor, let's briefly discuss [service lifetimes in ASP.NET Core.](https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes)
When you register a service in the DI container, you specify its lifetime.
The service lifetime defines how long the DI container should maintain the service.

ASP.NET Core provides three main lifetimes:

- **Singleton**: A single instance of the service is created and reused throughout the application's lifetime.
- **Scoped**: A new instance is created for each scope (usually a web request). Services created in the same scope share the same instance.
- **Transient**: A new instance is created every time the service is requested.

Understanding service lifetimes is crucial when designing your application's architecture.

## What is Scrutor?

The [Scrutor library](https://github.com/khellang/Scrutor) improves your dependency injection code by extending the existing features from `Microsoft.Extensions.DependencyInjection`.

These extensions add support for advanced assembly scanning and service decoration.

To get started using Scrutor, you need to install the NuGet package:

```powershell
Install-Package Scrutor
```

## Assembly Scanning With Scrutor

One of the most powerful features of Scrutor is its ability to perform assembly scanning.
Rather than manually registering each service, Scrutor allows you to scan your assemblies for types that should be registered with the DI container.
This can significantly reduce the boilerplate code required for service registration, making your code cleaner and more maintainable.

The entry point for assembly scanning is the `Scan` method, which accepts a delegate to define the DI setup.

Here's an example of scanning two assemblies and registering the classes inside as scoped services:

```csharp
builder.Services.Scan(selector => selector
    .FromAssemblies(
        typeof(PersistenceAssembly).Assembly,
        typeof(InfrastructureAssembly).Assembly)
    .AddClasses(publicOnly: false)
    .UsingRegistrationStrategy(RegistrationStrategy.Skip)
    .AsMatchingInterface()
    .WithScopedLifetime());
```

Let's unpack what's happening here:

- `FromAssemblies` - allows you to specify which assemblies to scan
- `AddClasses` - adds the classes from the selected assemblies
- `UsingRegistrationStrategy` - defines which `RegistrationStrategy` to use
- `AsMatchingInterface` - registers the types as matching interfaces (`ClassName` → `IClassName`)
- `WithScopedLifetime` - registers the types with a scoped service lifetime

There are three values for `RegistrationStrategy` you can use:

- `RegistrationStrategy.Skip` - skips registrations if service already exists
- `RegistrationStrategy.Append`- appends a new registration for existing services
- `RegistrationStrategy.Throw`- throws when trying to register an existing service

You can also specify a filter to `AddClasses` to select specific types you want to configure.
Here's an example of registering repository implementations:

```csharp
services.Scan(scan => scan
    .FromAssemblies(typeof(PersistenceAssembly).Assembly)
    .AddClasses(
        filter => filter.Where(x => x.Name.EndsWith("Repository")),
        publicOnly: false)
    .UsingRegistrationStrategy(RegistrationStrategy.Throw)
    .AsMatchingInterface()
    .WithScopedLifetime());
```

## Service Decoration With Scrutor

Service decoration is another valuable feature offered by Scrutor.
It enables you to modify or extend services during registration without changing the original implementation.

This is incredibly useful when adding cross-cutting concerns or other modifications to services without altering their core functionality.
For example, you can implement a [caching decorator for repositories.](decorator-pattern-in-asp-net-core)

Here's how you can configure a decorator with Scrutor's `Decorate` method:

```csharp
services.AddScoped<IActivitiesRepository, ActivitiesRepository>();

services.Decorate<IActivitiesRepository, PermissionActivitiesRepository>();
```

It will decorate the `ActivitiesRepository` service using the `PermissionActivitiesRepository`.
This also means that `PermissionActivitiesRepository` can inject an `IActivitiesRepository` instance, and at runtime, this is resolved as `ActivitiesRepository`.

Here's how you can implement the `PermissionActivitiesRepository`:

```csharp {3,7,21}
public class PermissionActivitiesRepository : IActivitiesRepository
{
    private readonly IActivitiesRepository _decorated;
    private readonly IPermissionChecker _permissionChecker;

    public PermissionActivitiesRepository(
        IActivitiesRepository decorated,
        IPermissionChecker permissionChecker)
    {
        _decorated = decorated;
        _permissionChecker = permissionChecker;
    }

    public List<Activity> Get()
    {
        if (!_permissionChecker.HasPermission(Permissions.FetchActivities))
        {
            return new();
        }

        return _decorated.Get();
    }
}
```

## Takeaway

Scrutor can improve your ASP.NET Core DI by simplifying service registration through assembly scanning and enabling service decoration.
You can use Scrutor's capabilities to write cleaner, more maintainable, and flexible DI code while reducing the complexity of your startup configuration.

Assembly scanning can reduce the boilerplate code required for service registration.
It also allows you to create custom conventions for registering services.

Service decoration has been a real game-changer for me.
It's the simplest way to introduce cross-cutting concerns in your application.
For example, I used to add an idempotency check before handling events.

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

Want to move to [**JetBrains Rider**](https://www.jetbrains.com/dotnet/guide/tutorials/resharper-to-rider/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)
but not sure where to start?
For a smooth experience, check out this guide where we look at the essential functionality you'll find helpful when [**migrating from Visual Studio.**](https://www.jetbrains.com/dotnet/guide/tutorials/resharper-to-rider/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)

Do you build complex software systems?
See how [**NServiceBus**](https://go.particular.net/milanjovanovic) makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

</Sponsorships>

NServiceBus is a feature-rich messaging framework supporting many different message transports.
It's developed and maintained by [Particular Software.](https://particular.net/)
And it simplifies the process of building complex distributed systems across various cloud-based queueing technologies.

The basic building blocks of NServiceBus are messages and endpoints.
A message contains the required information to execute a business operation.
Endpoints are logical entities that send and receive messages.

And now let's see how to get started with NServiceBus, from installation and setup to building your first NServiceBus endpoint.

In this week's newsletter, you will learn how to:

- Configure an endpoint to use Azure Service Bus
- Send and publish messages using `IMessageSession`
- Handle messages with NServiceBus

Let's dive in!

## What is NServiceBus?

[NServiceBus](https://go.particular.net/milanjovanovic) is a messaging framework and platform that simplifies building reliable, scalable, and maintainable distributed systems.
It's designed to address the challenges that arise when building applications that are distributed across multiple servers.

One of NServiceBus's foundational principles is its embrace of a message-driven architecture.
In this model, components communicate by sending and receiving messages.
Messages are the fundamental units of communication, representing commands, events, or data that services exchange.

Why is this significant?

Message-driven architectures offer several advantages:

- Asynchronous communication
- Loose coupling
- Reliability

NServiceBus supports the powerful publish/subscribe (pub/sub) messaging pattern.
This pattern allows services to publish events and subscribe to events of interest.
When a service publishes an event, all interested subscribers receive a copy of the event.
This is a key feature for building event-driven architectures, where services react to and process events in response to various actions or changes in the system.

## Configuring the NServiceBus Endpoint

NServiceBus uses the concept of an _endpoint_ to send and receive messages.
It's a logical component that communicates with other components.
You define your message handlers and sagas inside of an endpoint.

Let's start by installing the `NServiceBus` NuGet package:

```powershell
Install-Package NServiceBus.Extensions.Hosting
```

Now you can configure an _endpoint_ to use [Azure Service Bus](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview)
to send messages:

```csharp
var builder = WebApplication.CreateBuilder();

builder.Host.UseNServiceBus(context =>
{
    var endpointConfiguration = new EndpointConfiguration("Training");

    var transport = endpointConfiguration
        .UseTransport<AzureServiceBusTransport>();

    var connectionString = builder.Configuration
        .GetConnectionString("AzureServiceBusConnectionString");
    transport.ConnectionString(connectionString);

    endpointConfiguration.EnableInstallers();

    return endpointConfiguration;
});

var app = builder.Build();

app.Run();
```

The call to `UseNServiceBus` tells the host to use NServiceBus.
Inside the callback, you can configure the endpoint that will start when the host runs.

One more important aspect is calling `EnableInstallers` to set up the Azure Service Bus topology.
This will tell NServiceBus to create the required queues, so you don't have to do it manually.

## Publishing Messages in NServiceBus

The next building block you need in any messaging system is the messages.
Messages are C# classes or interfaces that contain meaningful data for the business process.

NServiceBus supports three types of messages:

- `ICommand` - sends a request to perform an action
- `IEvent` - communicates that something significant occurred
- `IMessage` - for messages that aren't commands or events (typically for replies in _request-response_)

Events can have more than one handler, while a command should have only one handler.

Let's create our first message contract:

```csharp
using NServiceBus;

public class WorkoutCreated : IEvent
{
    public Guid Id [ get; set; ]
}
```

The `WorkoutCreated` message is an event that we will publish after creating a new `Workout`.

You can use the `IMessageSession` service to send messages from your controllers or Minimal API endpoints.

```csharp {4,10}
app.MapPost("api/workouts", async (
    Workout workout,
    AppDbContext context,
    IMessageSession messageSession) =>
{
    context.Add(workout);

    await context.SaveChangesAsync();

    await messageSession.Publish(new  WorkoutCreated { Id = workout.Id });

    return Results.Ok(workout);
});
```

NServiceBus has some built-in validation when sending messages.
You have to specify an `ICommand` when calling the `Send` method, or you will get an exception.
Similarly, you have to specify an `IEvent` when calling the `Publish` method.

## Handling Messages With NServiceBus

Once you send a message, you need a way to handle it and run some business logic.
To handle a message, you need to implement the `IHandleMessages` interface and specify which message you are handling.

Here's an implementation of the `WorkoutCreatedHandler`:

```csharp
public class WorkoutCreatedHandler : IHandleMessages<WorkoutCreated>
{
    private readonly ILogger<WorkoutCreated> _logger;

    public WorkoutCreatedHandler(ILogger<WorkoutCreated> logger)
    {
        _logger = logger;
    }

    public async Task Handle(
        WorkoutCreated message,
        IMessageHandlerContext context)
    {
        logger.LogInformation("Processing workout - {Id}", message.Id);

        // Continue to process the message.
    }
}
```

Implementing `IHandleMessages<WorkoutCreated>` tells NServiceBus how to process the `WorkoutCreated` message when an endpoint receives it.
This interface defines only one method: `Handle`.

The `Handle` method has an `IMessageHandlerContext` parameter, which allows you to send more messages.
This can be helpful when implementing a choreographed saga.
Processing one message triggers the next step in the chain until the entire process is completed.

## In Summary

In this week's issue, we discussed NServiceBus, a robust messaging framework for building distributed systems in .NET.
You learned how to configure NServiceBus with the Azure Service Bus transport.
We discussed the different message types in NServiceBus and how to publish and handle a message.

Building distributed systems is a complex endeavor, but NServiceBus simplifies many challenges.
By embracing a message-driven architecture and leveraging NServiceBus's features,
you'll be well-equipped to create resilient, scalable, and maintainable applications in the .NET ecosystem.

Further reading:

- [NServiceBus step-by-step tutorial](https://go.particular.net/milanjovanovic/getting-started-with-nservicebus)
- [Live coding an NServiceBus system](https://go.particular.net/milanjovanovic/live-coding-your-first-nservicebus-system)
- [NServiceBus monitoring demo](https://go.particular.net/milanjovanovic/monitoring-demo)
- [Messaging with Azure Service Bus](messaging-made-easy-with-azure-service-bus)
- [Implementing the Saga pattern](implementing-the-saga-pattern-with-rebus-and-rabbitmq)
- [Orchestration vs Choreography](orchestration-vs-choreography)

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**Telerik UI for Blazor**](https://www.telerik.com/blogs/why-modernize-blazor-winning-strategy-next-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-mnm-content-exposure) -
a winning strategy for your next application.
With .NET 8 just around the corner, there's a lot of talk about Blazor becoming a "drop-in" replacement for MVC or Razor Pages.
If you are exploring how to modernize existing applications, moving from MVC, Razor Pages or Web Forms,
[**take a look at what Blazor brings to the table.**](https://www.telerik.com/blogs/why-modernize-blazor-winning-strategy-next-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-mnm-content-exposure)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

Validation is an essential cross-cutting concern that you need to solve in your application.
You want to ensure the request is valid before you consider processing it.

Another important question you need to answer is how you approach different types of validation.
For example, I consider input and business validation differently, and each deserves a specific solution.

I want to show you an elegant solution for validation using [MediatR](https://github.com/jbogard/MediatR)
and [FluentValidation.](https://docs.fluentvalidation.net/en/latest/index.html)

If you aren't using [CQRS](https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs) with MediatR, don't worry.
Everything I explain about validation can easily be adapted to other paradigms.

Here's what I'm going to talk about in this week's newsletter:

- Standard validation approach
- Input vs business validation
- Separating validation logic
- Generic `ValidationBehavior`

Let's dive in.

## The Standard Command Validation Approach

The standard way of implementing validation is right before processing the command.
The validation is tightly coupled to the command handler, which could be problematic.

I find this approach difficult to maintain as the complexity of the validation increases.
Each change to the validation logic also touches the handler, and the handler itself can grow out of control.

It also makes it harder to differentiate between input and _business_ validation.

Here's an example `ShipOrderCommandHandler` that checks if the `ShippingAddress.Country` is one of the supported countries:

```csharp
internal sealed class ShipOrderCommandHandler
    : IRequestHandler<ShipOrderCommand>
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly ShipmentSettings _shipmentSettings;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        if (!_shipmentSettings
                .SupportedCountries
                .Contains(command.ShippingAddress.Country))
        {
            throw new ArgumentException(nameof(ShipOrderCommand.Address));
        }

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
```

What if we can separate command validation and command handling?

## Input Validation and Business Validation

I mentioned input and _business_ validation in the previous section.

Here's how I consider them to be different:

- **Input validation** - We only validate that the command is _processable_.
  These are simple validations, such as checking for `null` values, empty strings, etc.
- **Business validation** - We validate the command to satisfy the business rules.
  This includes checking the system state for required preconditions before processing the command.

Another way to compare them is cheap vs. expensive.
Input validation is usually cheap to execute and can be done in memory.
While business validation involves reading state and is slower.

So, input validation sits at the entry point of the use case before handling the request.
After it completes, we have a _valid_ command.
And this is a rule I always follow - an invalid command should never reach the handler.

## Input Validation With FluentValidation

[FluentValidation](https://docs.fluentvalidation.net/en/latest/index.html) is an excellent validation library for .NET,
which uses a fluent interface and lambda expressions for building strongly typed validation rules.

Here's the `ShipOrderCommand` that we want to validate:

```csharp
public sealed record ShipOrderCommand : IRequest
{
    public Guid OrderId { get; set; }

    public string ShippingMethod { get; set; }

    public Address ShippingAddress { get; set; }
}
```

To implement a validator with [FluentValidation,](https://github.com/FluentValidation/FluentValidation)
you create a class that inherits from the `AbstractValidator<T>` base class.
Then, you can add the validation rules from the constructor using `RuleFor`:

```csharp
public sealed class ShipOrderCommandValidator
    : AbstractValidator<ShipOrderCommand>
{
    public ShipOrderCommandValidator(ShipmentSettings settings)
    {
        RuleFor(command => command.OrderId)
            .NotEmpty()
            .WithMessage("The order identifier can't be empty.");

        RuleFor(command => command.ShippingMethod)
            .NotEmpty()
            .WithMessage("The shipping method can't be empty.");

        RuleFor(command => command.ShippingAddress)
            .NotNull()
            .WithMessage("The shipping address can't be empty.");

        RuleFor(command => command.ShippingAddress.Country)
            .Must(country => settings.SupportedCountries.Contains(country))
            .WithMessage("The shipping country isn't supported.");
    }
}
```

The naming convention I like to use is the name of the command and append _Validator_.
You can also enforce this by writing [architecture tests.](enforcing-software-architecture-with-architecture-tests)

To automatically register all validators from an assembly, you need to call the `AddValidatorsFromAssembly` method:

```csharp
services.AddValidatorsFromAssembly(ApplicationAssembly.Assembly);
```

## Running Validation From the Use Case

To run the `ShipOrderCommandValidator`, you can use the `IValidator<T>` service and inject it from the constructor.

The validator exposes a few methods you can call, like `Validate`, `ValidateAsync`, or `ValidateAndThrow`.

The `Validate` method returns a `ValidationResult` object which contains two properties:

- `IsValid` - a boolean flag saying whether the validation succeeded
- `Errors` - a collection of `ValidationFailure` objects containing any validation failures

Alternatively, calling the `ValidateAndThrow` method throws a `ValidationException` if validation fails.

```csharp
internal sealed class ShipOrderCommandHandler
    : IRequestHandler<ShipOrderCommand>
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly IValidator<ShipOrderCommand> _validator;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        _validator.ValidateAndThrow(command);

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
```

This approach forces you to define an explicit dependency on `IValidator` in every command handler.

What if we can implement this cross-cutting concern in a more generic way?

## MediatR Validation Pipeline

Here's a complete implementation of a `ValidationBehavior` using FluentValidation and MediatR's `IPipelineBehavior`.

The `ValidationBehavior` acts as a middleware for the request pipeline and performs validation.
If the validation fails, it will throw a custom `ValidationException` with a collection of `ValidationError` objects.

I also want to highlight the use of `ValidateAsync`, which allows you to define asynchronous validation rules.
You must call the `ValidateAsync` method if you have asynchronous rules.
Otherwise, the validator will throw an exception.

```csharp
public sealed class ValidationBehavior<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : ICommandBase
{
    private readonly IEnumerable<IValidator<TRequest>> _validators;

    public ValidationBehavior(IEnumerable<IValidator<TRequest>> validators)
    {
        _validators = validators;
    }

    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        var context = new ValidationContext<TRequest>(request);

        var validationFailures = await Task.WhenAll(
            _validators.Select(validator => validator.ValidateAsync(context)));

        var errors = validationFailures
            .Where(validationResult => !validationResult.IsValid)
            .SelectMany(validationResult => validationResult.Errors)
            .Select(validationFailure => new ValidationError(
                validationFailure.PropertyName,
                validationFailure.ErrorMessage))
            .ToList();

        if (errors.Any())
        {
            throw new Exceptions.ValidationException(errors);
        }

        var response = await next();

        return response;
    }
}
```

Don't forget to register the `ValidationBehavior` with MediatR by calling `AddOpenBehavior`:

```csharp
services.AddMediatR(config =>
{
    config.RegisterServicesFromAssemblyContaining<ApplicationAssembly>();

    config.AddOpenBehavior(typeof(ValidationBehavior<,>));
});
```

## Handling Validation Exceptions

Here's a custom `ValidationExceptionHandlingMiddleware` middleware that only handles the custom `ValidationException`.
It converts the exception to a `ProblemDetails` response and includes any validation errors.

You can easily expand this to be a generic global exception handler.

```csharp
public sealed class ValidationExceptionHandlingMiddleware
{
    private readonly RequestDelegate _next;

    public ValidationExceptionHandlingMiddleware(RequestDelegate next)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (Exceptions.ValidationException exception)
        {
            var problemDetails = new ProblemDetails
            {
                Status = StatusCodes.Status400BadRequest,
                Type = "ValidationFailure",
                Title = "Validation error",
                Detail = "One or more validation errors has occurred"
            };

            if (exception.Errors is not null)
            {
                problemDetails.Extensions["errors"] = exception.Errors;
            }

            context.Response.StatusCode = StatusCodes.Status400BadRequest;

            await context.Response.WriteAsJsonAsync(problemDetails);
        }
    }
}
```

You also need to include the middleware in the request pipeline by calling `UseMiddleware`:

```csharp
app.UseMiddleware<ExceptionHandlingMiddleware>();
```

## Takeaway

This implementation of `ValidationBehavior` is something I use in real projects, and it works incredibly well.
If I don't want to throw an exception, I can update the `ValidationBehavior` to return a result object instead.

How do you apply this if you're not using MediatR?

I'm using an `IPipelineBehavior`, which allows me to implement a _middleware_ wrapping each request.

So, all you need is a way to implement middleware and place your validation inside.
And I like having options, so here are [three ways to create middleware in ASP.NET Core.](3-ways-to-create-middleware-in-asp-net-core)

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

Do you build complex software systems?
See how [**NServiceBus**](https://go.particular.net/milanjovanovic) makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

Want to learn [**Blazor?**](https://www.jetbrains.com/dotnet/guide/tutorials/blazor-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)
Check out this [**Blazor Essentials tutorial,**](https://www.jetbrains.com/dotnet/guide/tutorials/blazor-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)
where Rachel Appel, Developer Advocate at JetBrains, explores working with its components, forms, data, JavaScript interoperability, and more.

</Sponsorships>

You start building a beautiful monolith system.
Maybe a modular monolith.

The system grows over time, and requirements are ever-changing. Slowly, cracks begin to appear in the system.

This could be for organizational reasons and distributing the work across a team.
Or it could be because of scaling issues and performance bottlenecks.

You begin evaluating the possible solutions, and the benefits and tradeoffs of each one.
At last, you come to a decision.

It's time to migrate parts of the system to individual (micro)services.

So, how do we approach this migration from monolith to microservices?

That is the topic of this week's newsletter.

Let's dive in!

## Decoupling Using Bounded Contexts

The first step in moving from a monolith to microservices is identifying the bounded contexts.
Because they represent cohesive parts of the domain that are candidates for extraction.

One solution is to identify [bounded contexts](https://martinfowler.com/bliki/BoundedContext.html)
using the domain-driven design strategic modeling.

Bounded contexts define the explicit boundaries between modules and separate the responsibilities.
This is one of the biggest challenges when migrating to microservices.
[Identifying good boundaries](https://learn.microsoft.com/en-us/azure/architecture/microservices/model/domain-analysis)
between modules ensures microservices are narrowly focused on one problem domain.

Defining boundaries is also easier in a monolith because you aren't working with a distributed system.
[Refactoring bad boundaries](https://cloud.google.com/architecture/microservices-architecture-refactoring-monoliths)
is less risky, and you have more freedom to "get it right".

<Image
  src="blogs/mnw_056/bounded_contexts.png"
  width={978}
  height={597}
  alt="Bounded contexts."
/>

And the size of the bounded context shouldn't worry you.
Instead, focus on [service boundaries.](https://go.particular.net/right-sized-services)

The next problem you need to solve is coupling.
Coupling is manifested in two ways:

- Database dependencies
- Communication between modules

You can solve these problems from the start by building a modular monolith.
But I'll also explain the guiding principles you can use to solve coupling.

## How a Modular Monolith Solves Coupling

A [modular monolith](modular-monolith-communication-patterns#what-is-a-modular-monolith) is a catchy name for a monolith system built from a few
bounded contexts (modules) and following a set of principles to control coupling.
Each module contains a cohesive set of functionalities and is isolated from other modules in the system.
The isolation refers to database dependencies and inter-module communication.

<Image
  src="blogs/mnw_056/modular_monolith.png"
  width={978}
  height={597}
  alt="Modular monolith."
/>

You can think of a module as a distinct application within the system.
A module has its own domain, entities, use cases, database tables.
The modules are deployed together as a single executable application.
But they are otherwise independent.

You can apply different architectural approaches to each module, like [Clean Architecture.](/pragmatic-clean-architecture)

So I mentioned that you need to reduce the coupling between modules.

Here are two principles to solve database coupling:

- Modules can't share tables in the database
- Modules can't directly query the database tables of other modules

Sharing database tables leads to a high degree of coupling, and this is exactly what you are trying to avoid.
You can isolate the data for each module on a logical level using schemas or physically with different databases.

A module should expose a public API that other modules can call.
This public API is the entry point into the module.
And this is the only way for modules to communicate.

Communication between modules can be [synchronous](modular-monolith-communication-patterns#synchronous-communication-with-method-calls)
using method calls, or [asynchronous](modular-monolith-communication-patterns#asynchronous-communication-with-messaging)
using a message bus.

My preferred approach is asynchronous communication using messaging.
It's loosely coupled and makes the transition to microservices easier.

## Adding a Message Broker To The System

To implement asynchronous communication between modules, you can introduce a message broker.
But you don't need to introduce a full-blown message broker from the start.

You can implement messaging between modules using an abstraction like [MassTransit](https://masstransit.io) while abstracting away the transport mechanism.

MassTransit has an in-memory transport that works well inside a single process.
It's very fast.
But it isn't durable, and you can lose messages if the bus is stopped.

You only need to configure a different transport mechanism when introducing a real message broker.
But you don't need to change your messaging code.

<Image
  src="blogs/mnw_056/modular_monolith_queue.png"
  width={978}
  height={597}
  alt="Modular monolith with a queue."
/>

What is the purpose of messaging inside a modular monolith?

Designing your system like this makes the modules loosely coupled and independent.
The price you pay in increased complexity at the start is justified as the project matures.

## Extracting Modules to Microservices

We decided to move from a monolith system to microservices.
Since we built our system in a modular way, the migration comes down to extracting a module into a new process.

You should introduce a [reverse proxy](implementing-an-api-gateway-for-microservices-with-yarp) in front of your services to route incoming traffic.
This will hide the implementation details of the microservices system from client applications.

The new microservice needs to connect to the message bus, but we don't need to change anything in our code.
Using messaging for communication between modules simplifies the migration process.
This might remind you of [event-driven architecture.](https://go.particular.net/break-that-big-ball-of-mud)

If you implement inter-module communication using method calls, you must replace that implementation with HTTP calls over the network.
Because you're now building a distributed system, and the previous implementation using method calls will not work.
You also need to consider authentication, fault tolerance...

<Image
  src="blogs/mnw_056/extracting_modules.png"
  width={978}
  height={597}
  alt="Microservices with extracting modules."
/>

Extracting modules from the monolith system leads to replacing all the functionalities of the old system with new microservices.
This process of migrating to microservices follows the [strangler fig pattern.](https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig)

## Closing Thoughts

The biggest blocker for moving from a monolith to microservices is coupling.
Coupling is a change preventer.
So, this is the first thing you need to tackle.

You need to solve coupling at the database level and between components in the code.
Building the system in a modular way can prevent these problems from the start.

Which is why a Modular monolith is an excellent approach.

You can identify bounded contexts in the system and use them as the boundaries in the monolith.
And getting the boundaries right is easier in a monolith.

Migrating to microservices comes down to extracting the modules into individual services.

Of course, you still need to think about security and fault tolerance because you now have a distributed system.

Talking about architecture in abstract terms can be difficult to grasp, but it's important when discussing conceptual solutions.

I'll show you a practical Modular monolith implementation soon to complete the circle.

Until then, I hope this was valuable.

See you next week!

---



<Sponsorships>

[**Progress**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)
prepared a free eBook for you - [**Blazor Hybrid and Web in One Solution.**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)
Dive into Blazor cross-platform apps, potential roadblocks, testing with Razor components, and more.
[**Get the Blazor eBook here.**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

The ability to conditionally turn features on or off in your application without redeploying the code is a powerful tool.

It lets you quickly iterate on new features and frequently integrate your changes with the main branch.

You can use **feature flags** to achieve this.

**Feature flags** are a software development technique that allows you to wrap application features in a conditional statement.
You can then toggle the feature on or off in runtime to control which features are enabled.

We have a lot to cover in this week's newsletter:

- Feature flag fundamentals in .NET
- Feature filters and phased rollouts
- Trunk-based development
- A/B testing

Let's dive in!

## Feature Flags In .NET

[Feature flags](https://github.com/microsoft/FeatureManagement-Dotnet) provide a way for .NET and ASP.NET Core applications to turn features on or off dynamically.

To get started, you need to install the `Microsoft.FeatureManagement` library in your project:

```powershell
Install-Package Microsoft.FeatureManagement
```

This library will allow you to develop and expose application functionality based on features.
It's useful when you have special requirements when a new feature should be enabled and under what conditions.

The next step is to register the required services with dependency injection by calling `AddFeatureManagement`:

```csharp
builder.Services.AddFeatureManagement();
```

And you are ready to create your first feature flag.
Feature flags are built on top of the .NET configuration system.
Any .NET configuration provider can act as the backbone for exposing feature flags.

Let's create a feature flag called `ClipArticleContent` in our `appsettings.json` file:

```json
"FeatureManagement": {
  "ClipArticleContent": false
}
```

By convention, feature flags have to be defined in the `FeatureManagement` configuration section.
But you can change this by providing a different configuration section when calling `AddFeatureManagement`.

Microsoft recommends exposing feature flags using enums and then consuming them with the `nameof` operator.
For example, you would write `nameof(FeatureFlags.ClipArticleContent)`.

However, I prefer defining feature flags as constants in a static class because it simplifies the usage.

```csharp
// Using enums
public enum FeatureFlags
{
    ClipArticleContent = 1
}

// Using constants
public static class FeatureFlags
{
    public const string ClipArticleContent = "ClipArticleContent";
}
```

To check the feature flag state, you can use the `IFeatureManager` service.
In this example, if the `ClipArticleContent` feature flag is turned on, we will return only the first thirty characters of the article's content.

```csharp {8}
app.MapGet("articles/{id}", async (
    Guid id,
    IGetArticle query,
    IFeatureManager featureManager) =>
{
    var article = query.Execute(id);

    if (await featureManager.IsEnabledAsync(FeatureFlags.ClipArticleContent))
    {
        article.Content = article.Content.Substring(0, 50);
    }

    return Results.Ok(article);
});
```

You can also apply feature flags on a controller or endpoint level using the `FeatureGate` attribute:

```csharp
[FeatureGate(FeatureFlags.EnableArticlesApi)]
public class ArticlesController : Controller
{
   // ...
}
```

This covers the fundamentals of using feature flags, and now, let's tackle more advanced topics.

## Feature Filters And Phased Rollouts

The feature flags I showed you in the previous section were like a simple on-off switch.
Although practical, you might want more flexibility from your feature flags.

The `Microsoft.FeatureManagement` package comes with a few built-in feature filters that allow you to create dynamic rules for enabling feature flags.

The available feature filters are
[`Microsoft.Percentage`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.percentagefilter?view=azure-dotnet),
[`Microsoft.TimeWindow`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.timewindowfilter?view=azure-dotnet)
and [`Microsoft.Targeting`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.targetingfilter?view=azure-dotnet).

Here's an example of defining a `ShowArticlePreview` feature flag that uses a percentage filter:

```json {3,6}
"FeatureFlags": {
  "ClipArticleContent": false,
  "ShowArticlePreview": {
    "EnabledFor": [
      {
        "Name": "Percentage",
        "Parameters": {
          "Value": 50
        }
      }
    ]
  }
}
```

This means the feature flag will be randomly turned on 50% of the time.
The downside is the same user might see different behavior on subsequent requests.
A more realistic scenario is to have the feature flag state be cached for the duration of the user's session.

To use the `PercentageFilter`, you need to enable it by calling `AddFeatureFilter`:

```csharp
builder.Services.AddFeatureManagement().AddFeatureFilter<PercentageFilter>();
```

Another interesting feature filter is the `TargetingFilter`, which allows you to target specific users.
Targeting is used in phased rollouts, where you want to introduce a new feature to your users gradually.
You start by enabling the feature for a small percentage of users and slowly increase the rollout percentage while monitoring how the system responds.

## Trunk-based Development and Feature Flags

[Trunk-based development](https://trunkbaseddevelopment.com/) is a Git branching strategy where all developers work in short-lived branches or directly in the trunk, the main codebase.
The _"trunk"_ is the main branch of your repository.
If you're using Git, it will be either the `main` or `master` branch.
Trunk-based development avoids the "merge hell" problem caused by long-lived branches.

So, how do feature flags fit into trunk-based development?

The only way to ensure the trunk is always releasable is to hide incomplete features behind feature flags.
You continue pushing changes to the trunk as you work on the feature while the feature flag remains turned off on the main branch.
When the feature is complete, you turn on the feature flag and release it to production.

<Image
  src="blogs/mnw_055/trunk_based_development.png"
  width={900}
  height={468}
  alt="Trunk based development."
/>

## How I Used Feature Flags for A/B Testing On My Website

A/B testing (split testing) is an experiment where two or more variants of a page (or feature) are shown randomly to users.
Statistical analysis is performed in the background to determine which variation performs better for a given conversion goal.

Here's an example A/B test I performed on my website:

<Image
  src="blogs/mnw_055/split_test.png"
  width={1400}
  height={1700}
  alt="Split test with two variants."
/>

The hypothesis was that removing the image and focusing on the benefits would make more people want to subscribe.
I measure this using the conversion rate, which is the number of people visiting the page divided by the number of people subscribing.

I'm using a platform called [Posthog](https://posthog.com) to run experiments, which automatically calculates the results.

You can see that the _test_ variant has a significantly higher conversion rate, so it becomes the winner of the A/B test.

<Image
  src="blogs/mnw_055/experiment_results.png"
  width={907}
  height={749}
  alt="Split test with two variants experiment results."
/>

## Takeaway

The ability to dynamically turn features on or off without deploying the code is like a superpower.
Feature flags give you this ability with very little work.

You can work with feature flags in .NET by installing the `Microsoft.FeatureManagement` library.
Feature flags build on top of the .NET configuration system, and you can check the feature flag state using the `IFeatureManager` service.

Another use case for feature flags is A/B testing.
I run weekly experiments on my website, testing changes that will improve my conversion rate.
Feature flags help me decide which version of the website to show to the user.
And then, I can measure results based on the user's actions.

I also made a video about [**feature flagging in .NET,**](https://youtu.be/QVEUgIC7Wpo) and you can watch it [**here**](https://youtu.be/QVEUgIC7Wpo) if you want to learn more.

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

[**JetBrains .NET Day Online '23**](https://lp.jetbrains.com/dotnet-day-2023/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetday23)
is a free virtual event that will take place on September 26.
Ten amazing community speakers will share what they're passionate about in the .NET world.
Topics include C#, F#, Blazor, Avalonia, EF Core, xUnit, and more.
[**Register to save the date.**](https://lp.jetbrains.com/dotnet-day-2023/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetday23)

Do you build complex software systems?
See how [**NServiceBus**](https://go.particular.net/milanjovanovic) makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

</Sponsorships>

How often do you think about concurrency conflicts when writing code?

You write the code for a new feature, confirm that it works, and call it a day.

But one week later, you find out you introduced a nasty bug because you didn't think about concurrency.

The most common issue is race conditions with two competing threads executing the same function.
If you don't consider this during development, you introduce the risk of leaving the system in a corrupted state.

In this week's newsletter, I'll challenge you to spot the race condition in a method for reserving a booking.
The business requirement is you can't have two overlapping reservations for the same dates.

And then, I'll show you how to solve this race condition using EF Core optimistic concurrency.

Let's dive in!

## What's Wrong With This Code?

There's a race condition hiding somewhere in this code snippet.

Can you see it?

```csharp {9,18}
public Result<Guid> Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }

    var booking = Booking.Reserve(apartment, user, startDate, endDate);

    dbContext.Add(booking);

    dbContext.SaveChanges();

    return booking.Id;
}
```

The call to `IsOverlapping` is an optimistic check to see if there's an existing booking for the specified dates.

```csharp
if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate)) { }
```

If it returns `true`, we're trying to double-book the apartment.
So we return a failure, and the method completes.

But if it returns `false`, we reserve a booking and call `SaveChanges` to persist the changes in the database.

And there lies the problem.

There's a chance for a concurrent request to pass the `IsOverlapping` check and attempt to reserve the booking.
Without any concurrency control, both requests will succeed, and we will end up with an inconsistent state in the database.

So how can we solve this?

## Optimistic Concurrency With EF Core

The pessimistic concurrency approach acquires a lock for the data before modifying it.
It's slower and causes competing transactions to be blocked until the lock is released.
EF Core doesn't support this approach out of the box.

You can also solve this problem using optimistic concurrency with EF Core.
It doesn't take any locks, but any data modifications will fail to save if the data has changed since it was queried.

To implement optimistic concurrency in EF Core, you need to configure a property as a _concurrency token_.
It's loaded and tracked with the entity.
When you call `SaveChanges`, EF Core will compare the value of the concurrency token to the value in the database.

Let's assume we're using SQL Server, which has a native [`rowversion`](https://learn.microsoft.com/en-us/sql/t-sql/data-types/rowversion-transact-sql?view=sql-server-ver16) column.
The `rowversion` automatically changes when the row is updated, so it's a great option for a concurrency token.

To configure a `byte[]` property as a concurrency token you can decorate it with the `Timestamp` attribute.
It will be mapped to a `rowversion` column in SQL Server.

```csharp {5}
public class Apartment
{
    public Guid Id { get; set; }

    [Timestamp]
    public byte[] Version { get; set; }
}
```

I prefer a different approach because attributes pollute the entity.

You can do the same with the Fluent API.
I will even use a shadow property to hide the concurrency token from the entity class.

```csharp {4,5}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Apartment>()
        .Property<byte[]>("Version")
        .IsRowVersion();
}
```

The exact configuration will differ based on the database you are using, so check the documentation.

## How Optimistic Concurrency Works In Practice

So here's what changes when we configure the concurrency token.

When loading the `Apartment` entity, EF will also load the concurrency token.

```sql
SELECT a.Id, a.Version
FROM Apartments a
WHERE a.Id = @p0
```

And when we call `SaveChanges`, the update statement will compare the concurrency token value with the one in the database:

```sql
UPDATE Apartments a
SET a.LastBookedOnUtc = @p0
WHERE a.Id = @p1 AND a.Version = @p2;
```

If the `rowversion` in the database changes, the number of updated rows will be `0`.

EF Core expects to update `1` row, so it will throw a `DbUpdateConcurrencyException`, which you need to handle.

## Handling Concurrency Exceptions

Now that you know how to use optimistic concurrency with EF Core, you can fix the previous code snippet.

If two concurrent requests pass the `IsOverlapping` check, only one can complete the `SaveChanges` call.
The other concurrent request will run into a `Version` mismatch in the database and throw a `DbUpdateConcurrencyException`.

In case of a concurrency conflict, we need to add a `try-catch` statement to catch the `DbUpdateConcurrencyException`.
How you handle the actual exception depends on your business requirements.
And sometimes, [race conditions](https://go.particular.net/milanjovanovic/raceconditions) might not even exist.

```csharp {24}
public Result<Guid> Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }

    try
    {
        var booking = Booking.Reserve(apartment, user, startDate, endDate);

        dbContext.Add(booking);

        dbContext.SaveChanges();

        return booking.Id;
    }
    catch (DbUpdateConcurrencyException)
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }
}
```

## When Should You Use Optimistic Concurrency?

Optimistic concurrency considers the best scenario is also the most probable one.
It assumes conflicts between transactions will be infrequent and doesn't acquire locks on the data.
This means your system can scale better because there is no blocking slowing down performance.

However, you must still expect concurrency conflicts and implement custom logic to handle them.

Optimistic concurrency is a good choice if your application doesn't expect many conflicts.

Another reason to use optimistic concurrency is when you can't hold an open connection to the database for the length of the transaction.
This is required for pessimistic locking.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**How to Get Started with Telerik Blazor Data Grid:**](https://www.telerik.com/blogs/how-to-get-started-telerik-blazor-data-grid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-content-exposure&utm_content=get-started-telerik-blazor-data-grid)
If there's one thing you're likely to find yourself doing in your Blazor web applications, it's working with data.
The simplest (and most common) way to do this is often via a table (or grid).
[**Learn how the Telerik UI for Blazor DataGrid**](https://www.telerik.com/blogs/how-to-get-started-telerik-blazor-data-grid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-content-exposure&utm_content=get-started-telerik-blazor-data-grid)
can make working with data in your Blazor web applications much easier.

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

Modern software applications rarely work in isolation.
On the contrary, a typical application will talk to several external systems like databases, messaging systems, cache providers, and many 3rd party services.
And it's up to you to ensure everything functions correctly.

Hopefully, I don't have to convince you about the value of writing tests.

You should be writing tests.
Period.

However, I do want to discuss the _value_ of **integration testing**.

**Unit tests** are helpful to test business logic in isolation, without any external services.
They are easy to write and provide almost instant feedback.

But you can't be fully confident in your application without **integration tests**.

So, in this week's newsletter, I'll show you how to use **Docker** for integration testing.

Here's what we will use to write **integration tests**:

- **Testcontainers**
- Docker
- [**xUnit**](creating-data-driven-tests-with-xunit)

Let's dive in!

## What is Testcontainers?

[Testcontainers](https://dotnet.testcontainers.org/) is a library for writing tests with throwaway Docker containers.

Why should you use it?

Integration testing is considered "difficult" because you have to maintain testing infrastructure.
Before running tests, you need to make sure the database is up and running.
You also have to seed any data required for the tests.
If you have tests running in parallel on the same database, they could interfere with each other.

A possible solution could be using in-memory variations of the required services.
But this isn't much different from using mocks.
In-memory services might not have all the features of the production service.

Testcontainers solves this by using Docker to spin up real services for integration testing.

Here's an example of creating a **SQL Server** container:

```csharp
MsSqlContainer dbContainer = new MsSqlBuilder()
    .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
    .WithPassword("Strong_password_123!")
    .Build();
```

You can then use the `MsSqlContainer` instance to get a connection string for the database running inside the container.

Do you see how this is valuable for writing integration tests?

No more need for mocks or fake in-memory databases.
Instead, you can use the real deal.

I won't do a deep dive into this library here, so refer to the documentation for more information.

## Implementing a Custom WebApplicationFactory

**ASP.NET Core** provides an in-memory test server that we can use to spin up an application instance for running tests.
The `Microsoft.AspNetCore.Mvc.Testing` package provides the `WebApplicationFactory` class that we will use as the base for our implementation.

`WebApplicationFactory<TEntryPoint>` is used to create a `TestServer` for the integration tests.

The custom `IntegrationTestWebAppFactory` will do a few things:

- Create and configure a `MsSqlContainer` instance
- Call `ConfigureTestServices` to set up EF Core with the container database
- Start and stop the container instance with `IAsyncLifetime`

`MsSqlContainer` has a `GetConnectionString` method to grab the connection string for the current container.
Note that this can change between tests, as each test class will create a separate container instance.
Test cases inside the same test class will use the same container instance.
So keep that in mind if you need to do a cleanup between tests.

Another thing to keep in mind is **database migrations**.
You will have to run them manually before every test to create the required database structure.

Starting the container instance is done asynchronously using `IAsyncLifetime`.
The container is started inside `StartAsync` before any of the tests run.
And it's stopped inside `StopAsync`.

Here's the complete code for `IntegrationTestWebAppFactory`:

```csharp
public class IntegrationTestWebAppFactory
    : WebApplicationFactory<Program>,
      IAsyncLifetime
{
    private readonly MsSqlContainer _dbContainer = new MsSqlBuilder()
        .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
        .WithPassword("Strong_password_123!")
        .Build();

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureTestServices(services =>
        {
            var descriptorType =
                typeof(DbContextOptions<ApplicationDbContext>);

            var descriptor = services
                .SingleOrDefault(s => s.ServiceType == descriptorType);

            if (descriptor is not null)
            {
                services.Remove(descriptor);
            }

            services.AddDbContext<ApplicationDbContext>(options =>
                options.UseSqlServer(_dbContainer.GetConnectionString()));
        });
    }

    public Task InitializeAsync()
    {
        return _dbContainer.StartAsync();
    }

    public new Task DisposeAsync()
    {
        return _dbContainer.StopAsync();
    }
}
```

## Creating The Base Test Class

The base test class will implement a class fixture interface `IClassFixture`.
It indicates the class contains tests and provides shared object instances across the test cases inside.
This is a good place to instantiate any services that are required for most tests.

For example, I'm creating an `IServiceScope` for resolving scoped services inside the tests.

- `ISender` for sending commands and queries
- `ApplicationDbContext` for database setup or verifying results

```csharp
public abstract class BaseIntegrationTest
    : IClassFixture<IntegrationTestWebAppFactory>,
      IDisposable
{
    private readonly IServiceScope _scope;
    protected readonly ISender Sender;
    protected readonly ApplicationDbContext DbContext;

    protected BaseIntegrationTest(IntegrationTestWebAppFactory factory)
    {
        _scope = factory.Services.CreateScope();

        Sender = _scope.ServiceProvider.GetRequiredService<ISender>();

        DbContext = _scope.ServiceProvider
            .GetRequiredService<ApplicationDbContext>();
    }

    public void Dispose()
    {
        _scope?.Dispose();
        DbContext?.Dispose();
    }
}
```

With all the infrastructure in place, we're finally ready to write the tests.

## Putting It All Together - Writing Integration Tests

Here's a `ProductTests` class with an integration test inside.

I use the _Arrange-Act-Assert_ pattern to structure tests:

- _Arrange_ - create the `CreateProduct.Command` instance
- _Act_ - send the command using `ISender` and store the result
- _Assert_ - use the result from the _Act_ step to verify the database state

The value of writing integration tests like this is that you can use the complete **MediatR** request pipeline.
If you have any `IPipelineBehavior` wrapping the request, it will also be executed.

The same applies if you write your business logic inside service classes.
Instead of resolving the `ISender`, you would resolve the specific services you want to test.

Most importantly, this test uses a real database instance running inside a **Docker container**.

```csharp
public class ProductTests : BaseIntegrationTest
{
    public ProductTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Create_ShouldCreateProduct()
    {
        // Arrange
        var command = new CreateProduct.Command
        {
            Name = "AMD Ryzen 7 7700X",
            Category = "CPU",
            Price = 223.99m
        };

        // Act
        var productId = await Sender.Send(command);

        // Assert
        var product = DbContext
            .Products
            .FirstOrDefault(p => p.Id == productId);

        Assert.NotNull(product);
    }
}
```

## Running Integration Tests In CI/CD Pipelines

You can also run integration tests with **Testcontainers** inside **CI/CD pipelines**.
The only requirement is that it supports Docker.

**GitHub Actions** does support Docker.
If you are hosting your project there, integration tests will work out of the box.

You can learn more about [**building a CI/CD pipeline with GitHub Actions here.**](how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet)

And if you want a plug-in solution, here's a GitHub Actions workflow you can use:

```yaml
name: Run Tests 🚀

on:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  run-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '7.0.x'

      - name: Restore
        run: dotnet restore ./Products.Api.sln

      - name: Build
        run: dotnet build ./Products.Api.sln --no-restore

      - name: Test
        run: dotnet test ./Products.Api.sln --no-build
```

## Takeaway

**Testcontainers** is an excellent solution for writing **integration tests** with Docker.
You can spin up and configure any **Docker** image and use it from your application.
This is far better than using mocks or in-memory variations because they lack many features.

If you have a CI/CD pipeline that supports Docker, Testcontainers will work out of the box.

Only a few integration tests will drastically improve your confidence in the system.

You can grab the [**source code for this newsletter**](https://github.com/m-jovanovic/testcontainers-sample) on my GitHub.\
It's completely free, so what are you waiting for?

And if you prefer video, here's a quick tutorial on [**integration testing with Testcontainers.**](https://youtu.be/tj5ZCtvgXKY)

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

This week's issue is sponsored by [**NServiceBus.**](https://go.particular.net/milanjovanovic)
Do you build complex software systems?
See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

And [**ReSharper,**](https://www.jetbrains.com/resharper/?utm_source=milan&utm_medium=newsletter&utm_campaign=resharper)
which is a productivity extension by JetBrains that makes Visual Studio a much better IDE.
Check out the [**ReSharper Essentials**](https://www.jetbrains.com/dotnet/guide/tutorials/resharper-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=resharper)
video series to learn how to write better code faster by using on-the-fly code analysis,
automated solution-wide refactorings, and instant fixes to eliminate errors and code smells.

</Sponsorships>

More than 63%+ of organizations said in a [Dzone survey](https://dzone.com/articles/new-research-shows-63-percent-of-enterprises-are-a)
that they are adopting **Microservices** for some or all of their applications.

As more businesses adopt the use of Microservice architectures, we as developers have to become more skilled with Microservices communication.

Working with **distributed systems** is both fun and challenging at the same time.
One of those challenges is designing **effective communication** between services.

More centralization or less centralization?
More coupling or less coupling?
More control or less control?

These are only a few questions you need to answer.

In this week's newsletter, we will:

- Break down **Orchestration vs. Choreography**
- Understand key differences and **tradeoffs** between them
- Create a framework for deciding which approach to use

Let's dive in!

## What Are Microservices?

**Microservices** are a **software architecture** style where an application is built from small, autonomous services.
Each microservice serves a distinct purpose and can be independently deployed.

You probably already know this, so this is a quick refresher.

<Image src="blogs/mnw_052/microservices_hell.png" width={900} height={468} />

Here are the key tenets of Microservices:

- Independent development
- Deployment independence
- Technological freedom
- Scalability
- Resilience

A significant challenge is designing effective inter-service communication within this distributed environment.

Inside a Monolith system, communication happens through direct method calls.
This is a straightforward approach that works well when all components live within a single process.
However, this doesn't work with microservices.

## Orchestration - Command-driven Communication

**Orchestration** is a centralized approach to Microservices communication.
One of the services takes on the role of the **orchestrator** and coordinates the communication between services.

Orchestration uses **command-driven** communication.
The command communicates the intent of the action.
The sender wants something to happen, and the recipient doesn't need to know who sent the command.

An example of orchestration can be a [**Saga implemented with RabbitMQ.**](implementing-the-saga-pattern-with-rebus-and-rabbitmq)

<Image src="blogs/mnw_052/orchestration.png" width={1021} height={600} />

It has some nice **benefits**:

- Simple
- Centralized
- Ease of troubleshooting
- Monitoring is straightforward

**Orchestration** is typically **simpler** to implement and maintain than choreography.
Because there is a central coordinator, you can manage and monitor service interactions.
This, in turn, improves troubleshooting, as you know where to look when something goes wrong.

The **drawbacks** of orchestration are:

- Tight coupling
- Single point of failure
- Difficulty adding, removing, or replacing microservices

## Choreography - Event-driven Communication

**Choreography** is a **decentralized** communication approach.
Choreography uses **event-driven** communication - as opposed to orchestration, which uses commands.

An **event** is something that has happened in the past and is a fact.
The sender does not know who will handle the event or what will happen after processing it.

I talked about events in-depth in the newsletter about [**publishing domain events.**](how-to-use-domain-events-to-build-loosely-coupled-systems)

<Image src="blogs/mnw_052/choreography.png" width={1021} height={600} />

The most important **benefits** of choreography are:

- Loose coupling
- Ease of maintenance
- Decentralized control
- Asynchronous communication

**Choreography** allows microservices to be **loosely coupled**, which means they can operate independently and asynchronously.
This makes the system more scalable and resilient.
A failure of one microservice won't necessarily affect microservices.

Of course, there are **downsides** to choreography:

- Complexity
- Monitoring is difficult
- Difficulty troubleshooting

It's more complex to implement and maintain than orchestration.

Effective monitoring is the biggest challenge from my experience.

## The Crossroads: Which One Should You Choose?

So, how do you know which one to pick for your system?

I always start with the requirements of the system I'm building.
And then, I look at the pros and cons of orchestration vs. choreography.
Here's a small framework to help you decide.

**Orchestration** excels when:

- You need to wait for the completion of intermediate steps (such as credit card payment confirmation)
- You need to make a conditional choice of subsequent steps
- The process must be carried out atomically (entirely or not at all)
- The process needs to be centralized in one place for monitoring

This also means you will need a central database managed by the orchestrator to handle all workflow-related state.

**Choreography** works best when:

- The process can rely on the input message without needing additional context
- Steps clearly follow one another
- Progress is made in one direction

You can benefit from increased flexibility (such as modifying individual steps in isolation)
Unfortunately, choreography can make it difficult to trace, debug, or monitor the processes triggered by an event.
And the larger the event stream is, the more challenging it becomes.

So, think carefully before opting for orchestration or choreography.

Both approaches bring their advantages but also downsides.

## Takeaway

**Orchestration** defines a sequence of steps that each microservice must follow.
This is great for identifying and addressing complex service interdependencies.
Another benefit is that business logic can be managed and monitored in one place.

**Choreography**, on the other hand, is a decentralized technique for microservices communication.
Each service can operate independently while still being part of the larger architecture.

To decide which approach to use, you should observe your system and identify what you stand to gain or lose.
Everything is a tradeoff.

There's also an **alternative approach** I want to mention.

You could go for a **hybrid approach** that integrates orchestration and choreography.

In a **hybrid approach**, you decide which communication technique to use for a specific workflow.
Some workflows can benefit from orchestration, and others can benefit more from choreography.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**Treblle**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan) is a lightweight SDK that helps engineering and product teams build,
ship & maintain REST-based APIs faster.
Simple integration for all popular languages & frameworks, including .NET 6.
[**Check it out here.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)

[**ABP Commercial**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w51) is a complete web development platform built on ABP Framework, perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.
[**Get instant access here.**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w51)

</Sponsorships>

**Rate limiting** is about restricting the number of requests to your application.
It's usually applied within a specific time window or based on other criteria.

It's helpful for a few reasons:

- Improves security
- Guards against DDoS attacks
- Prevents overloading of application servers
- Reduces costs by preventing unnecessary resource consumption

**.NET 7** shipped with a **built-in rate limiter**, but you need to know how to implement it correctly.
Or you could grind your system to a halt - and we don't want that.

In this week's newsletter, I'll teach you:

- How to rate limit users by **IP address**
- How to rate limit users by their **identity**
- How to apply **rate limiting** on the **reverse proxy**

So let's dive in!

## Built-In Rate Limiting In .NET 7

Starting with .NET 7, we have access to built-in **rate limiting middleware** in the `Microsoft.AspNetCore.RateLimiting` namespace.
The API is straightforward, and you can create a rate limit policy with a few lines of code.

We can use one of the four **rate limiting algorithms**:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here's how to define a **rate limit policy** by calling the `AddTokenBucketLimiter` method:

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 1000;
        options.ReplenishmentPeriod = TimeSpan.FromHours(1);
        options.TokensPerPeriod = 700;
        options.AutoReplenishment = true;
    });
});
```

Now you can reference the `token` rate limit policy on your endpoint or controller.

You also have to add the `RateLimitingMiddleware` to the request pipeline:

```csharp
app.UseRateLimiter();
```

You can learn more about [**rate limiting in .NET 7 here,**](how-to-use-rate-limiting-in-aspnet-core) so I won't go deeper into the fundamentals.

## Rate Limiting Users By IP Address

The approach I just showed you has a **problem** - the **rate limit policy** is global and **applies to all users**.

Most of the time, you don't want to do this.
Rate limiting should be granular and apply to **individual users**.

Luckily, you can achieve this by creating a `RateLimitPartition`.

The `RateLimitPartition` has two components:

- Partition key
- Rate limiter policy

Here's how to define a rate limiter with a fixed window policy, and the **partition key** is the user's **IP address**.

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-ip", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.Connection.RemoteIpAddress?.ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

Rate limiting by **IP address** can be a good layer of security for **unauthenticated users**.
You don't know who is accessing your system and can't apply more granular rate limiting.
This can help protect your system from malicious users trying to perform a DDoS attack.

You can also [**create chained limiters**](https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-7.0#create-chained-limiters) using the `CreateChained` API.
It allows you to pass in multiple `PartitionedRateLimiter`, which are combined into one `PartitionedRateLimiter`.
The chained limiter runs all the input limiters in sequence (one by one).

If your application is running behind a **reverse proxy**, you need to make sure not to rate limit the proxy IP address.
Reverse proxies usually **forward** the original IP address with the `X-Forwarded-For` header.
So you can use it as the **partition key**:

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-ip", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            httpContext.Request.Headers["X-Forwarded-For"].ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

## Rate Limiting Users By Identity

If you require users to **authenticate** with your API, you can determine who the current is.
Then you can use the user's **identity** as the **partition key** for a `RateLimitPartition`.

Here's how you would create such a rate limit policy:

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-user", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.User.Identity?.Name?.ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

I'm using the `User.Identity` value on the `HttpContext` to get the current user's `Name` claim.
This usually corresponds to the `sub` claim inside a JWT - which is the user identifier.

## Applying Rate Limting On The Reverse Proxy

In a robust implementation, you want to **rate limit** on the **reverse proxy** level before the request hits your API.
And if you have a distributed system, this is a requirement.
Otherwise, your system wouldn't function correctly.

There are many reverse proxy implementations to choose from.

**YARP** is a reverse proxy with excellent .NET integration.
Not surprising since it was written in C#.
You can learn more about [**building an API Gateway with YARP here.**](implementing-an-api-gateway-for-microservices-with-yarp)

To implement rate limiting on the reverse proxy with **YARP** you need to:

- Define a rate limit policy (covered in previous examples)
- Configure the `RateLimiterPolicy` for the route in YARP settings

```json
"products-route": {
  "ClusterId": "products-cluster",
  "RateLimiterPolicy": "sixty-per-minute-fixed",
  "Match": {
    "Path": "/products/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

The built-in rate limiter middleware uses an **in-memory** store to track the number of requests.
If you want to run your reverse proxy in a high-availability setup, you will need to use a **distributed cache**.
A nice option to look into is using a [**Redis backplane for rate limiting.**](https://github.com/cristipufu/aspnetcore-redis-rate-limiting)

## Closing Thoughts

With the `PartitionedRateLimiter` you can easily create granular rate limit policies.

The two common approaches are:

- Rate limiting by **IP address**
- Rate limiting by the **user identifier**

I was really excited to see the .NET team ship rate limiting.
But, the current implementation has its shortcomings.
The main issue is that it only works **in memory**.
For a **distributed** solution, you need to implement something yourself or use an external library.

You can use the **YARP** reverse proxy to build robust and scalable distributed systems.
And it only takes a few lines of code to add **rate limiting** on the reverse proxy level.
I'm using it extensively in my systems.

Thanks for reading.

And stay awesome!

---



<Sponsorships>

[**JetBrains Rider**](https://www.jetbrains.com/rider/?utm_source=milan&utm_medium=newsletter&utm_campaign=rider)
is a fast and powerful cross-platform .NET IDE that helps you develop ASP.NET Core and
.NET desktop apps, services, libraries, Unity and Unreal Engine games, Xamarin and MAUI apps, and more.
Watch the [**Rider Essentials**](https://www.jetbrains.com/dotnet/guide/tutorials/rider-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=rider)
video series with tips and tricks to get more out of the IDE.

[**Workshop: How to Migrate to .NET MAUI? →**](https://www.telerik.com/campaigns/maui-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-workshop-mnm-na-awareness)
Master the basic concepts behind .NET MAUI and learn about the possibilities of sharing code between web and native mobile and desktop applications.
[**Join the workshop here.**](https://www.telerik.com/campaigns/maui-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-workshop-mnm-na-awareness)

</Sponsorships>

**Dapper** is a lightweight **object-relational mapper** in .NET.
It's popular because it's easy to use and fast at the same time.

Dapper extends the `IDbConnection` interface with methods for sending SQL queries to the database.

But, because of the nature of SQL, mapping the result into an object model can be tricky.

So in this week's newsletter, I'll teach you how to map:

- Simple queries
- One-to-one relationships
- One-to-many relationships
- Many-to-many relationships

Let's dive in!

## Simple Mapping

Let's first see how to do a **simple mapping** using Dapper.

Writing a query with Dapper has three parts:

- Creating an `IDbConnection` instance
- Writing the SQL query
- Calling a method that Dapper exposes

We will write a SQL query to load a set of `LineItem` objects for a specific `Order`.

```csharp
public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
```

Here's the SQL query returning the result we need:

```sql
SELECT Id AS LineItemId, OrderId, Price, Currency, Quantity
FROM LineItems
WHERE OrderId = @OrderId
```

I'm parameterizing the `Order` identifier using the `@OrderId` syntax.
This is a Dapper convention.
It's important that you use **parameterized queries** to **avoid SQL injection attacks**.

The mapping is straightforward in this case because we are only returning one type from the database.

We call the `QueryAsync` method and specify `LineItem` as the return type.
Make sure to pass in the arguments for this method, the SQL query, and the `OrderId` parameter.
I prefer creating anonymous objects for Dapper parameters.

```csharp
using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync<LineItem>(
    sql,
    new { OrderId = orderId });
```

That's everything you need for a simple mapping.

## Dapper One To One Relationship Mapping

What if the object we want to return from the SQL query contains a **nested object**?

Here's an updated `LineItem` type that also contains a `Product` inside.

```csharp
public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }

    public Product Product { get; init; }
}

public class Product
{
    public long ProductId { get; init; }

    public string Name { get; init; }
}
```

Now you need to return two types in the same query.

Here's the updated SQL query with a join on the `Products` table:

```sql
SELECT li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity,
       p.Id AS ProductId, p.Name
FROM LineItems li
JOIN Products p ON p.Id = li.ProductId
WHERE li.OrderId = @OrderId
```

This query is more complicated because we need to use Dapper's **multi-mapping** feature.

In the `QueryAsync` method, we specify both `LineItem` and `Product` as return types and `LineItem` as the final return type for the method.

We must also tell Dapper how to map the `LineItem` and `Product` from the result set into a single `LineItem` object.

And we need to specify the `splitOn` argument, which tells Dapper where one object ends and the next begins.

```csharp
using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync<LineItem, Product, LineItem>(
    sql,
    (lineItem, product) =>
    {
        lineItem.Product = product;

        return lineItem;
    },
    new { OrderId = orderId },
    splitOn: "ProductId");
```

We write more code to make this work, but it should be easy to wrap your head around it.

## Dapper One To Many Relationship Mapping

Another frequent situation is mapping a **one-to-many relationship** from SQL into an object model.

Because you are joining two tables, the result set will contain duplicate data on the "one" side of the relationship.

For this example, let's use an `Order` with a list of `LineItem` objects.

```csharp
public class Order
{
    public long OrderId { get; init; }

    public List<LineItem> LineItems { get; init; } = new();
}

public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
```

Here's the SQL query returning the data we need from the database:

```sql
SELECT o.Id AS OrderId,
       li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity
FROM Orders o
JOIN LineItems li ON li.OrderId = o.Id
WHERE o.Id = @OrderId
```

We're going to get back duplicate `Order` data because of the `JOIN`.
But we only want to return one `Order` with all the line items.

The Dapper mapping function only gives us the `Order` and `LineItem` for the current row in the result set.

One way to solve this is to use a `Dictionary` to store the `Order` and reuse it inside the mapping.

- Store the `Order` in the dictionary if it's not there
- If it is there, add the `LineItem` to the existing `Order` instance

```csharp
using var connection = new SqlConnection();

var ordersDictionary = new Dictionary<long, Order>();

await connection.QueryAsync<Order, LineItem, Order>(
    sql,
    (order, lineItem) =>
    {
        if (ordersDictionary.TryGetValue(order.OrderId, out var existingOrder))
        {
            order = existingOrder;
        }
        else
        {
            ordersDictionary.Add(order.OrderId, order);
        }

        order.LineItems.Add(lineItem);

        return order;
    },
    new { OrderId = orderId },
    splitOn: "LineItemId");

var mappedOrder = ordersDictionary[orderId];
```

A **many-to-many relationship** would use the same idea, except you'll need two dictionaries for each side of the relationship.

## In Summary

**Dapper** is a fantastic library for writing fast database queries using SQL.

Because of how SQL works, mapping into an object model is sometimes complicated.

There are four common scenarios:

- Simple mapping - a flat structure mapped directly from SQL to an object
- One-to-one mapping - provide a mapping function to connect two objects
- One-to-many mapping - manage a dictionary for the "one" side of the relationship
- Many-to-many mapping - same as above, except you need a dictionary for both sides of the relationship

Now you have a cheat sheet for mapping relationships with Dapper.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**Workshop: How to migrate a legacy app to Blazor:**](https://www.telerik.com/campaigns/blazor-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-workshop-mnm-na-awareness)
Join Ed Charbeneau for a live application-building workshop, where you'll migrate an existing .NET Web Forms application to Blazor in real time.
Learn how to avoid potential roadblocks and master the basic concepts behind .NET 7 and Blazor.
[**Join the workshop here.**](https://www.telerik.com/campaigns/blazor-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-workshop-mnm-na-awareness)

[**Treblle**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan) is a lightweight SDK that helps engineering and product teams build, ship & maintain REST-based APIs faster.
Simple integration for all popular languages & frameworks, including .NET 6.
[**Check it out here.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)

</Sponsorships>

**Modular monoliths** are becoming more popular in the software engineering community.

The allure of **Microservices** is becoming less compelling.
We also have seasoned veterans of our industry saying you should reconsider:

> You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.

_&mdash; [Martin Fowler](https://martinfowler.com/bliki/MonolithFirst.html)_

Modular monoliths give you the **logical architecture** of Microservices without the operational complexity.
You can safely determine the boundaries between modules.
And refactoring is straightforward and less risky.
They can also be easily migrated into Microservices if you decide to do so.

I've built and maintained several large **Modular monolith** systems in the past years.

In this week's newsletter, I want to focus on the **communication patterns** in the Modular monolith architecture.

But first, let me explain what is a **Modular monolith**.

## What Is a Modular Monolith?

Here's one definition of what a **Modular monolith** is:

> A Modular Monolith is a software design approach in which a monolith is designed with an emphasis on interchangeable (and potentially reusable) modules.

The problem with most monolith systems is that they become **tightly coupled** over time.
Components are deeply intertwined.
Making a change in one component impacts many others.
Introducing new features is difficult and error-prone.

Modular monoliths aim to solve these problems.

<Image
  src="blogs/mnw_049/modular_monolith_diagram.png"
  width={3876}
  height={1880}
/>

A Modular monolith consists of many **loosely coupled** modules.
Modules represent cohesive sets of functionalities.
Modules are also **independent** of each other.

Here are a few examples:

- Payments module
- Shipping module
- Booking module
- Reviews module

If this concept reminds you of Microservices, that's because it should.

Microservices represent self-contained services encapsulating a set of functionalities of the larger system, much like modules in a Modular monolith.

For a Modular monolith to be loosely coupled, you need to solve how modules will communicate.

Modules cannot reference each other directly except through their public APIs.

There are two widely used communication patterns.
Both have pros and cons and a set of tradeoffs that you need to understand.

## Synchronous Communication With Method Calls

The first and easiest communication pattern is simple **method calls** between modules.
Method calls are **synchronous** and very fast because they're in memory.

Module A calls a method declared on the **public API** of Module B and waits until it receives a result.

Each module exposes a **public API**, which can be an `interface` in .NET.

The module will implement this interface internally and hide any implementation details.
You can use the `internal` keyword to make the implementation inaccessible outside the module.

Modules depend on the interfaces at compile-time.
At runtime, **dependency injection** will provide the respective implementation.

<Image
  src="blogs/mnw_049/modular_monolith_sync_communication.png"
  width={3220}
  height={1192}
/>

The benefits of this approach are:

- Speed of in-memory calls
- Easy to implement
- No indirection

But, the drawback of this approach is **strong coupling**.

**Synchronous communication** means that the modules will be tightly coupled.
If one of the modules is unavailable, it will affect any dependent modules.
You can introduce a retry mechanism, but this only goes so far.

## Asynchronous Communication With Messaging

The second communication pattern is asynchronous **messaging** between modules.

Module A sends a message to the message broker in a fire-and-forget fashion.
Module B subscribes to relevant messages and handles them accordingly.

Modules don't need to know about each other, but they do need to know about the message contracts.

**Message contracts** are the **public API** of a module in this scenario.

<Image
  src="blogs/mnw_049/modular_monolith_async_communication.png"
  width={5384}
  height={1569}
/>

The benefits of this approach are:

- High availability
- Loose coupling

**Asynchronous communication** gives us loose coupling because modules communicate using messages.
Module B doesn't need to be available for Module A to send a message.

The obvious drawback of this approach is **increased complexity**.

We're introducing a **message broker** to the system.
This is another infrastructure component we have to manage.
It's also a single point of failure.
If the message broker fails, so does communication between the modules.

You can prevent message loss by storing messages in an [**Outbox**](outbox-pattern-for-reliable-microservices-messaging) before publishing them.
We can always send the messages again from the database in case of a message broker failure.

## Takeaway

**Synchronous communication** between modules is easy to implement, and it's performant. But it comes at the cost of tight coupling between modules.

**Asynchronous communication** using a message broker is loosely coupled. But it's more complex to implement.

So which **communication pattern** should you be using?

It depends.

Asynchronous communication can help you build **loosely coupled** and **independent** modules.
Migrating a **Modular monolith** using messaging into a **distributed system** is much easier.

You extract a module into its own deployment unit.
And the communication between modules remains the same.
Because you are using messaging, you don't need to reimplement anything.

**Synchronous method calls** are an excellent choice to increase development velocity and reduce operational complexity.

I'll let the software architect in you decide.

Thanks for reading.

And stay awesome!

---



<Sponsorships>

[**Software Design Simplified .NET Edition**](https://softwaredesignsimplified.com/?utm_source=newsletter&utm_medium=email&utm_campaign=dotnet-newsletter-29-07-2023)
is a short and practical book for junior and mid-level engineers.
You will learn encapsulation, cohesion, polymorphism, and more in one afternoon.
[**Check it out here.**](https://softwaredesignsimplified.com/?utm_source=newsletter&utm_medium=email&utm_campaign=dotnet-newsletter-29-07-2023)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems**](https://u.icepanel.io/1eec0e6f) with an interactive map that teammates can drill down to the level of detail they need.

</Sponsorships>

I've been using **Clean Architecture** for 6+ years on large-scale applications serving thousands of customers and millions of requests.
Today I want to talk about why it's a great approach for structuring your applications.

I'm aware that Clean Architecture isn't a silver bullet, so I will discuss what types of systems can benefit from this architecture.

Clean architecture isn't revolutionary.

But it's **prescriptive** about how you should structure the code.

It's an evolution of layered architecture, focusing on the core domain and the direction of dependencies.
All dependencies should point inwards, applying **dependency inversion**.

Here are some of the promises of Clean Architecture:

- Maintainability
- Testability
- Loose coupling
- Separation of concerns

It's **independent** of UI, databases, or external services - but you also need to be **pragmatic** (more on this later).

Let's dive in!

## What Is Clean Architecture?

Clean Architecture was created by [**Robert C. Martin**](https://en.wikipedia.org/wiki/Robert_C._Martin), aka Uncle Bob.

Clean Architecture is an approach to organizing a software system to **separate the concerns** of the various components.
Making the system easier to understand and maintain.

You can think about Clean Architecture as a domain-centric approach to organizing dependencies.

There are similar architectures that follow the same domain-centric idea.
You may know them as the Hexagonal and Onion architectures.
They are more or less interchangeable with each other.
And they all place the core domain at the center of the architecture.

This is my high-level interpretation of **Clean Architecture**:

<Image src="blogs/mnw_048/clean_architecture.png" width={3268} height={2006} />

There are four layers inside:

- Domain
- Application
- Infrastructure
- Presentation

Let's see what should live inside each layer.

## Clean Architecture Layers

Here's a breakdown of what should live inside each **layer** of the **Clean Architecture**.

**Domain**

- Contains the core business rules & logic
- It should be independent of other layers in the system
- Should be persistence ignorant - the persistence mechanism shouldn't influence your domain model
- **Examples:** Entities, Value Objects, Domain services, Domain events, Enums, Repository interfaces

**Application**

- Contains the application use cases
- Contains application-specific business rules
- Orchestrates the domain entities to perform business operations
- It should be independent of external concerns (but it doesn't have to be)
- **Examples:** Application services, Commands, Queries, External service interfaces, Exceptions

**Infrastructure**

- Contains anything related to external concerns
- Implements interfaces defined in the layers below
- **Examples:** PostgreSQL, Keycloak, AWS S3, RabbitMQ, Kafka, SendGrid

**Presentation**

- Represents the entry point to the system
- Accepts data from the outside and passes it to the use cases
- Acts as the composition root for dependency injection
- **Examples:** ASP.NET Core, gRPC

[**Here's an example**](clean-architecture-folder-structure) of how to structure the Clean Architecture on a solution level.
You can also group related components together by feature.
It leads to better cohesion and is an excellent option if your project is more complex.

I also made a few videos covering the Clean Architecture project setup:

- [**Clean Architecture walkthrough**](https://youtu.be/tLk4pZZtiDY) (100k+ views)
- [**Clean Architecture project setup from scratch**](https://youtu.be/fe4iuaoxGbA) (40k+ views)

## Where Should You Use Clean Architecture?

**Clean Architecture** is very versatile and applies to various domains and systems.

But, you should play to its strengths and use it only when there's a tangible benefit.

I use the **Clean Architecture** when I want to:

- Apply Domain-Driven Design
- Solve complex business logic
- Build highly testable projects
- Enforce design policies via the architecture

If the above is true for your project, then Clean Architecture is an excellent option.

You should also consider the [**benefits of Clean Architecture.**](clean-architecture-and-the-benefits-of-structured-software-design)

## The Case For Being Pragmatic

I try to be **pragmatic** when using **Clean Architecture**.

Applying what I like and having the freedom of "breaking" Clean Architecture if it will simplify things.

Can this be called Clean Architecture, then?
No, not in the purest sense.

But, I still get **most** of the **benefits** of Clean Architecture.

Here's an example, when I'm applying CQRS in Clean Architecture to implement the use cases.

On the command (write) side, it's valuable to be independent of external concerns, so I will use repositories behind an interface.
I can control the repository contract, and unit testing is straightforward.

But, on the query (read) side, I want to return the response as fast as possible.
Creating an abstraction only adds indirection and reduces performance.

A better approach is to use the EF or Dapper in the handler and query the database.
It's simple, fast, and you can use all the features offered by the ORM.
You don't need a lot of complexity or abstractions on the query side.

I should call this approach [**Pragmatic Clean Architecture**.](/pragmatic-clean-architecture)

## Closing Thoughts

**Clean Architecture** gives you a **standard** for organizing your solution.

You don't have to reinvent the wheel every time at the start of the project.

But, the layered structure and architectural constraints can increase the **complexity** of smaller projects.\
So make sure your project is complex enough to apply Clean Architecture.

Another **caveat** of Clean Architecture is the danger of **over-engineering**.

Don't follow the principles religiously without considering the specific project requirements.\
The overhead of maintaining so many layers and abstractions may not be justified.

Be **pragmatic** and try to make the best decision possible.

Sometimes that means straying from the paradigm.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

Today's issue is sponsored by [**Pluralsight.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)
Pluralsight's [**2023 State of Cloud report**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/) comes with some surprising findings.
Did you know 70% of organizations have more than half of their infrastructure in the cloud?
Or that 44% of organizations adopt the latest cloud products as soon as they're available?
Find out where you stand in the industry - and [**how to get ahead.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)

And by [**ABP Commercial.**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w47&utm_campaign=newsletter-sponsorship)
ABP Commercial is a complete [**web development platform built on ABP Framework**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w47&utm_campaign=newsletter-sponsorship),
perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.

</Sponsorships>

In software engineering, "coupling" means how much different parts of a software system depend on each other.
If they are **tightly coupled**, changes to one part can affect many others.
But if they are **loosely coupled**, changes to one part won't cause big problems in the rest of the system.

**Domain events** are a **Domain-Driven Design (DDD)** tactical pattern that we can use to build **loosely coupled** systems.

You can raise a **domain event** from the domain, which represents a fact that has occurred.
And other components in the system can subscribe to this event and handle it accordingly.

Here's what you will learn in this week's newsletter:

- What are [domain events](#what-are-domain-events)
- How they're [different from integration events](#domain-events-versus-integration-events)
- How to [implement](#implementing-domain-events) & [raise domain events](#raising-domain-events)
- How to [publish domain events](#how-to-publish-domain-events-with-ef-core) with EF Core
- How to [handle domain events](#how-to-handle-domain-events) with MediatR

We have a lot to cover, so let's dive in!

## What Are Domain Events?

An **event** is something that has happened in the past.

It is a fact.

Unchangeable.

A **domain event** is something that happened in the domain, and other parts of the domain should be aware of it.

**Domain events** allow you to express side effects explicitly, and provide a better separation of concerns in the domain.
They're an ideal way to trigger side effects across multiple aggregates inside the domain.

It's your responsibility to ensure that publishing a **domain event** is transactional.
You'll see why this is easier said than done.

<Image src="blogs/mnw_047/domain_events.png" width={2704} height={1515} />

## Domain Events Versus Integration Events

You may have heard of **integration events**, and you're now wondering what's the difference between them and **domain events**.

Semantically, they're the same thing: a representation of something that occurred in the past.

However, their **intent is different** and this is important to understand.

Domain events:

- Published and consumed within a single domain
- Sent using an in-memory message bus
- Can be processed synchronously or asynchronously

Integration events:

- Consumed by other subsystems (microservices, Bounded Contexts)
- Sent with a message broker over a queue
- Processed completely asynchronously

So if you're wondering what type of event you should publish, think about the intent and who should be handling the event.

**Domain events** can also be used to **generate integration events**, which leave the domain boundary.

## Implementing Domain Events

My preferred approach to implement **domain events** is creating an `IDomainEvent` abstraction and implementing MediatR `INotification`.

The benefit is you can use **MediatR's publish-subscribe** support to publish a notification to one or multiple handlers.

```csharp
using MediatR;

public interface IDomainEvent : INotification
{
}
```

Now you can implement a concrete domain event.

Here are a few **constraints** to consider when **designing domain events**:

- Immutability - domain events are facts, and should be immutable
- Fat vs Thin domain events - how much information do you need?
- Use past tense for event naming

```csharp
public class CourseCompletedDomainEvent : IDomainEvent
{
    public Guid CourseId { get; init; }
}
```

## Raising Domain Events

After you create your domain events, you want to raise them from the domain.

My approach is creating an `Entity` base class, because only entities are allowed to raise domain events.
You can further encapsulate raising domain events by making the `RaiseDomainEvent` method `protected`.

We're storing domain events in an internal collection, to prevent anyone else from accessing it.
The `GetDomainEvents` method is there to get a snapshot of the collection, and the `ClearDomainEvents` method to clear the internal collection.

```csharp
public abstract class Entity : IEntity
{
    private readonly List<IDomainEvent> _domainEvents = new();

    public IReadOnlyList<IDomainEvent> GetDomainEvents()
    {
        return _domainEvents.ToList();
    }

    public void ClearDomainEvents()
    {
        _domainEvents.Clear();
    }

    protected void RaiseDomainEvent(IDomainEvent domainEvent)
    {
        _domainEvents.Add(domainEvent);
    }
}
```

Now you're entities can inherit from the `Entity` base class and raise domain events:

```csharp
public class Course : Entity
{
    public Guid Id { get; private set; }

    public CourseStatus Status { get; private set; }

    public DateTime? CompletedOnUtc { get; private set; }

    public void Complete()
    {
        Status = CourseStatus.Completed;
        CompletedOnUtc = DateTime.UtcNow;

        RaiseDomainEvent(new CourseCompletedDomainEvent { CourseId = this.Id });
    }
}
```

And all that's left to do is **publish** the **domain events**.

## How To Publish Domain Events With EF Core

An elegant solution for publishing domain events is using **EF Core**.

Since EF Core acts as a **Unit of Work**, you can use it to gather all **domain events** in the current transaction and publish them.

I don't like to complicate things, and simply override the `SaveChangesAsync` method to publish the domain events after persisting the changes in the database.
But you could also use an interceptor.

```csharp
public class ApplicationDbContext : DbContext
{
    public override async Task<int> SaveChangesAsync(
        CancellationToken cancellationToken = default)
    {
        // When should you publish domain events?
        //
        // 1. BEFORE calling SaveChangesAsync
        //     - domain events are part of the same transaction
        //     - immediate consistency
        // 2. AFTER calling SaveChangesAsync
        //     - domain events are a separate transaction
        //     - eventual consistency
        //     - handlers can fail

        var result = await base.SaveChangesAsync(cancellationToken);

        await PublishDomainEventsAsync();

        return result;
    }
}
```

The most **important decision** you will have to make here is **when to publish the domain events**.

I think it makes the most sense to publish after calling `SaveChangesAsync`.
In other words, after saving changes to the database.

This comes with a few tradeoffs:

- Eventual consistency - because messages are processed after the original transactions
- Database inconsistency risk - because handling domain events can fail

Eventual consistency is something I can live with, so I choose to make this tradeoff.

However, introducing a risk of database inconsistency is a big concern.

You can solve this with the [**Outbox pattern,**](outbox-pattern-for-reliable-microservices-messaging)
where you persist your changes to the database and the domain events (as outbox messages) in a single transaction.
Now you have a guaranteed atomic transaction, and the domain events are processed asynchronously using a background job.

If you're wondering what's inside the `PublishDomainEventsAsync` method:

```csharp
private async Task PublishDomainEventsAsync()
{
    var domainEvents = ChangeTracker
        .Entries<Entity>()
        .Select(entry => entry.Entity)
        .SelectMany(entity =>
        {
            var domainEvents = entity.GetDomainEvents();

            entity.ClearDomainEvents();

            return domainEvents;
        })
        .ToList();

    foreach (var domainEvent in domainEvents)
    {
        await _publisher.Publish(domainEvent);
    }
}
```

## How To Handle Domain Events

With all of the plumbing we created so far, we're ready to implement a handler for the domain events.
Luckily, this is the simplest step in the process.

All you have to do is define a class implementing `INotificationHandler<T>` and specify your domain event type as the generic argument.

Here's a handler for the `CourseCompletedDomainEvent`, which takes the domain event and publishes a `CourseCompletedIntegrationEvent` to notify other systems.

```csharp
public class CourseCompletedDomainEventHandler
    : INotificationHandler<CourseCompletedDomainEvent>
{
    private readonly IBus _bus;

    public CourseCompletedDomainEventHandler(IBus bus)
    {
        _bus = bus;
    }

    public async Task Handle(
        CourseCompletedDomainEvent domainEvent,
        CancellationToken cancellationToken)
    {
        await _bus.Publish(
            new CourseCompletedIntegrationEvent(domainEvent.CourseId),
            cancellationToken);
    }
}
```

## In Summary

**Domain events** can help you build a loosely coupled system.
You can use them to separate the core domain logic from the side effects, which can be handled asynchronously.

There's no need to reinvent the wheel for implementing domain events, and you can use the **EF Core** and **MediatR** libraries to build this.

You will have to make the decision when you want to publish domain events.
Publishing before or after saving changes to the database both have their set of **tradeoffs**.

I prefer **publishing** domain events **after saving changes** to the database, and I use the [**Outbox pattern**](outbox-pattern-for-reliable-microservices-messaging)
to add transactional guarantees.
This approach introduces eventual consistency, but it's also more reliable.

Hope this was helpful.

See you next week!

**Today's action step:**
Take a look at [**this video,**](https://youtu.be/AHzWJ_SMqLo) where I explain how to implement domain events to build a decoupled system that scales.

---



<Sponsorships>

Today's issue is sponsored by [**Pluralsight.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)
Pluralsight's [**2023 State of Cloud report**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/) comes with some surprising findings.
Did you know 70% of organizations have more than half of their infrastructure in the cloud?
Or that 44% of organizations adopt the latest cloud products as soon as they're available?
Find out where you stand in the industry - and [**how to get ahead.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

</Sponsorships>

**Clean code** is code that's easy to read, maintain, and understand.

I consider writing **clean code** a skill.

And it's a **skill** that **you can learn** and improve with deliberate practice.

My favorite approach for practicing **clean coding** is doing **refactoring exercises**.

So I prepared one for you today, and we're going to improve it one step at a time by applying **clean code principles**.

Let's dive in!

## Starting Point

I like starting with a problem when trying to learn new concepts.

And the more illustrative the problem, the better.

So we'll use some poorly written code as the starting point for our refactoring.

And in each step, I will highlight what the current issue is and how we will fix it.

Here's what I see when I look at the `Process` method:

- Deep nesting of code - 4 levels, to be precise
- Precondition checks are applied one after the other
- Throwing exceptions to represent a failure

How can we turn this into **clean code**?

```csharp
public void Process(Order? order)
{
    if (order != null)
    {
        if (order.IsVerified)
        {
            if (order.Items.Count > 0)
            {
                if (order.Items.Count > 15)
                {
                    throw new Exception(
                        "The order " + order.Id + " has too many items");
                }

                if (order.Status != "ReadyToProcess")
                {
                    throw new Exception(
                        "The order " + order.Id + " isn't ready to process");
                }

                order.IsProcessed = true;
            }
        }
    }
}
```

## #1: Early Return Principle

It should be painfully obvious by now that the initial version is deeply nested because of the `if` statements applying precondition checks.

We'll solve this using the **early return principle**, which states that we should return from a method as soon as the conditions for that have been met.

In the case of the `Process` method, this means moving from a deeply nested structure to a set of **guard clauses**.

```csharp {3-6,8-11,13-16}
public void Process(Order? order)
{
    if (order is null)
    {
        return;
    }

    if (!order.IsVerified)
    {
        return;
    }

    if (order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #2: Merge If Statements To Improve Readability

The **early return principle** makes the `Process` method more readable.

But there's no need to have one **guard clause** after another.

So we can merge all of them into one `if` statement.

The behavior of the `Process` method remains unchanged, but we remove a lot of excess code.

```csharp {3-8}
public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #3: Use LINQ For More Concise Code

A quick improvement can be using **LINQ** to make the code more concise and expressive.

Instead of checking for `Items.Count == 0`, I prefer using the LINQ `Any` method.

You could argue that LINQ has worse performance, but I always optimize for readability.

There are far more expensive operations in an application than a simple method call.

```csharp {5}
public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        !order.Items.Any())
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #4: Replace Boolean Expression With Descriptive Method

Merging multiple conditions into one `if` statement means writing less code, but it can **decrease readability** with **complex conditions**.

However, you can fix this and improve readability by using a variable or method with a **descriptive name**.

I prefer using methods, so I will introduce the `IsProcessable` method to represent the precondition check.

```csharp {3, 23-28}
public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #5: Prefer Throwing Custom Exceptions

Now let's talk about throwing exceptions.
I like to use exceptions for _"exceptional"_ situations only, and I don't use them for flow control in my code.

Having said that, if you _do_ want to use exceptions for flow control, it's better to use **custom exceptions**.

You can introduce valuable contextual information and better describe the reason for throwing the exception.

And if you want to handle these exceptions globally, you can create a base class to be able to catch specific exceptions.

```csharp {10,15}
public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #6: Fix Magic Numbers With Constants

A common **code smell** I see is the use of **magic numbers**.

They are usually easy to spot because they're used to check if numeric some condition applies.

The problem with **magic numbers** is that they **carry no meaning**.

The code is harder to reason about, and more error-prone.

Fixing **magic numbers** should be straightforward, and one solution is introducing a constant.

```csharp {1,10}
const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #7: Fix Magic Strings With Enums

Similar to **magic numbers**, we have the **magic strings** **code smell**.

A typical use case for **magic strings** is to represent some sort of state.

You'll notice that we're comparing the `Order.Status` value to a **magic string** to check if the order is ready to process.

A few **problems** with **magic strings**:

- Easy to make mistakes (typo)
- Lack of strong typing
- Not refactoring proof

Let's create an `OrderStatus` `enum` to represent the possible states:

```csharp
enum OrderStatus
{
    Pending = 0,
    ReadyToProcess = 1,
    Processed = 2
}
```

And now we have to use the appropriate `OrderStatus` in the check:

```csharp {15,21}
const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #8: Use The Result Object Pattern

I said I don't prefer using exceptions for flow control. But how can we fix this?

One solution is using the **result object pattern**.

You can use a generic `Result` class to represent all types of results or a specific one like `ProcessOrderResult`.

To make your result objects encapsulated, expose a set of factory methods to create the concrete result type.

```csharp
public class ProcessOrderResult
{
    private ProcessOrderResult(
        ProcessOrderResultType type,
        long orderId,
        string message)
    {
        Type = type;
        OrderId = orderId;
        Message = message;
    }

    public ProcessOrderResultType Type { get; }

    public long OrderId { get; }

    public string? Message { get; }

    public static ProcessOrderResult NotProcessable() =>
      new(ProcessOrderResultType.NotProcessable, default, "Not processable");

    public static ProcessOrderResult TooManyLineItems(long oderId) =>
      new(ProcessOrderResultType.TooManyLineItems, orderId, "Too many items");

    public static ProcessOrderResult NotReadyForProcessing(long oderId) =>
      new(ProcessOrderResultType.NotReadyForProcessing, oderId, "Not ready");

    public static ProcessOrderResult Success(long oderId) =>
      new(ProcessOrderResultType.Success, oderId, "Success");
}
```

Using an `enum` like `ProcessOrderResultType` will make consuming the result object easier with switch expressions.
Here's the `enum` to represent the `ProcessOrderResult.Type`:

```csharp
public enum ProcessOrderResultType
{
    NotProcessable = 0,
    TooManyLineItems = 1,
    NotReadyForProcessing = 2,
    Success = 3
}
```

And now the `Process` method becomes:

```csharp {7,12,17,23}
const int MaxNumberOfLineItems = 15;

public ProcessOrderResult Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return ProcessOrderResult.NotProcessable();
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        return ProcessOrderResult.TooManyLineItems(order);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        return ProcessOrderResult.NotReadyForProcessing(order);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;

    return ProcessOrderResult.Success(order);
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

Here's how using an `enum` for the `ProcessOrderResult.Type` allows you to write a switch expression:

```csharp
var result = Process(order);

result.Type switch
{
    ProcessOrderResultType.TooManyLineItems =>
        Console.WriteLine($"Too many line items: {result.OrderId}"),

    ProcessOrderResultType.NotReadyForProcessing =>
        Console.WriteLine($"Not ready for processing {result.OrderId}"),

    ProcessOrderResultType.Success =>
        Console.WriteLine($"Processed successfully {result.OrderId}"),

    _ => Console.WriteLine("Failed to process: {OrderId}", result.OrderId),
};
```

## Takeaway

That's it, 8 tips to write **clean code**:

- [Early return principle](#1-early-return-principle)
- [Merge multiple if statements](#2-merge-if-statements-to-improve-readability)
- [Use LINQ for conciseness](#3-use-linq-for-more-concise-code)
- [Replace boolean expression with method](#4-replace-boolean-expression-with-descriptive-method)
- [Prefer throwing custom exceptions](#5-prefer-throwing-custom-exceptions)
- [Replace magic numbers with constants](#6-fix-magic-numbers-with-constants)
- [Replace magic string with enums](#7-fix-magic-strings-with-enums)
- [Use the result object pattern](#8-use-the-result-object-pattern)

Writing **clean code** is a matter of deliberate practice and experience.

Most people will read about **clean coding principles**, but few will strive to apply them daily.

This is where you can set yourself apart.

I also made a video about these **clean code** tips, you can watch it [here.](https://youtu.be/McDvyFglkvU)

Hope this was helpful.

See you next week!

**Today's action step:**
Take a look at your project and see if you're making some of the mistakes I highlighted here.
And then fix them using the clean code tips I shared with you.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

</Sponsorships>

Large **Microservice-based** systems can consist of tens or even hundreds of individual services.
A client application needs to have all of this information to be able to make requests to the relevant **microservice** directly.

However, this has numerous issues, such as security concerns, increased complexity, and coupling.

We can solve this by introducing an **API gateway** that acts as a **reverse proxy** to accept API calls from the client application and forward them to the appropriate service.

The **API gateway** also enforces security and ensures scalability and high availability.

In this week's newsletter, I'll show you how to implement an **API gateway** for your **microservices system** using the **YARP reverse proxy**.

Here's what we will cover:

- Difference between **API gateway** and **reverse proxy**
- Installing and configuring **YARP**
- Creating an **API gateway** with **YARP**
- **Authentication** and **rate limiting** on the **API gateway**

Let's dive in.

## What's The Difference Between an API Gateway And a Reverse Proxy?

A **reverse proxy** and an **API gateway** are similar concepts, but they serve different purposes.

A **reverse proxy** acts as an intermediary between clients and servers.
The clients can only call the backend servers through the **reverse proxy**, which forwards the request to the appropriate server.
It hides the implementation details of individual servers inside the internal network.

A **reverse proxy** is commonly used for:

- Load balancing
- Caching
- Security
- SSL termination

<Image src="blogs/mnw_045/reverse_proxy.png" width={2602} height={1502} />

An **API gateway** is a specific type of **reverse proxy** designed for managing APIs.
It acts as a single entry point for API consumers to the various backend services.

The key characteristics of an **API gateway** are:

- Request routing and composition
- Request/response transformations
- Authentication and authorization
- Rate limiting
- Monitoring

Also, note that an **API gateway** can perform **load balancing** and other functionalities mentioned for reverse proxies.

Now let's see how to use a **reverse proxy** to implement an **API gateway**.

## Installing And Configuring YARP

**YARP** (Yet Another Reverse Proxy) is a library developed by Microsoft to address the needs of various teams needing to build a **reverse proxy**.
It's open source and built with .NET, so it integrates nicely with the existing ecosystem.

Let's install `Yarp.ReverseProxy` **NuGet** package to get started:

```powershell
Install-Package Yarp.ReverseProxy
```

Next, we're going to call:

- `AddReverseProxy` to add the required services for **YARP**
- `LoadFromConfig` to load the **reverse proxy** configuration from application settings
- `MapReverseProxy` to introduce the **reverse proxy** middleware

```csharp {3-4,8}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
```

We need to tell the **YARP** reverse proxy how to route the incoming requests to the individual microservices.

**YARP** uses the concept of `Routes` to represent request patterns for the proxy and `Clusters` to represent the services to forward those requests.

```json {3,6}
{
    "ReverseProxy": {
        "Routes": {
            ...
        },
        "Clusters": {
            ...
        }
    }
}
```

Here's an example **YARP configuration** with a `{**catch-all}` pattern that will route all incoming requests to the one destination server.

```json
{
  "ReverseProxy": {
    "Routes": {
      "ROUTE_NAME": {
        "ClusterId": "CLUSTER_NAME",
        "Match": {
          "Path": "{**catch-all}"
        }
      }
    },
    "Clusters": {
      "CLUSTER_NAME": {
        "Destinations": {
          "destination1": {
            "Address": "https://www.milanjovanovic.tech/"
          }
        }
      }
    }
  }
}
```

## Implementing an API Gateway With YARP

We can use **YARP** to build an **API gateway** by providing the configuration for the services we want to route traffic to.

I created a sample [API gateway implementation with YARP](https://github.com/m-jovanovic/yarp-api-gateway-sample) on GitHub, so you can give it a try.
The system has two services, the `Users.Api` and `Products.Api`, which are .NET 7 applications.

If a request comes in matching the `/users-service/{**catch-all}`, for example `/users-service/users`, it will be routed to the `users-cluster`.
The same logic applies for the `products-cluster`. We can apply more advanced transformations through the `Transforms` section.

```json
{
  "ReverseProxy": {
    "Routes": {
      "users-route": {
        "ClusterId": "users-cluster",
        "Match": {
          "Path": "/users-service/{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      },
      "products-route": {
        "ClusterId": "products-cluster",
        "Match": {
          "Path": "/products-service/{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      }
    },
    "Clusters": {
      "users-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5201/"
          }
        }
      },
      "products-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5101/"
          }
        }
      }
    }
  }
}
```

We now have a functioning **API gateway** built with **YARP**, routing requests to two individual services.

But what else can we do with **YARP**?

## Adding Authentication

The **API gateway** can enforce **authentication** and **authorization** at the entry point to the system before letting authenticated requests proceed.

And **YARP** supports integrating with the existing authentication & authorization middleware.

You first need to define an **authorization policy**:

```csharp
builder.Services.AddAuthorization(options =>
{
    options.AddPolicy("authenticated", policy =>
        policy.RequireAuthenticatedUser());
});
```

And call `UseAuthentication` and `UseAuthorization` to add the respective middleware to the request pipeline.
It's important to add them before calling `MapReverseProxy`.

```csharp {1,3}
app.UseAuthentication();

app.UseAuthorization();

app.MapReverseProxy();
```

Now all you have to do is add the `AuthorizationPolicy` section to the reverse proxy configuration:

```json {3}
"users-route": {
  "ClusterId": "users-cluster",
  "AuthorizationPolicy": "authenticated",
  "Match": {
    "Path": "/users-service/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

**YARP** will forward most credentials to the proxied services, such as cookies or bearer tokens because it might be important to identify the user in the individual microservices.

## Adding Rate Limiting

You can also use an **API gateway** to introduce **rate limiting** to your system.
It's a technique to limit the number of requests to your API to **improve security** and reduce the load on the servers.

You can learn more about [how to use rate limiting in .NET here.](how-to-use-rate-limiting-in-aspnet-core)

As you can already guess, **YARP** supports the native **rate limiting** mechanism added in .NET 7.

All you need to do is define a **rate limit policy**:

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.Window = TimeSpan.FromSeconds(10);
        options.PermitLimit = 5;
    });
});
```

Then you need to call `UseRateLimiter` to add the rate limiter middleware to the request pipeline.
It's important to do it before calling `MapReverseProxy`.

```csharp {1}
app.UseRateLimiter();

app.MapReverseProxy();
```

And then, you can apply rate limiting to the desired route using the `RateLimiterPolicy` section in the reverse proxy configuration:

```json {3}
"products-route": {
  "ClusterId": "products-cluster",
  "RateLimiterPolicy": "fixed",
  "Match": {
    "Path": "/products-service/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

## In Summary

An **API gateway** is a critical component for a robust **microservices system** implementation.

And **YARP** is an excellent option if you want to build an **API gateway** with .NET.

I created a sample API gateway implementation with YARP, which you can find [here.](https://github.com/m-jovanovic/yarp-api-gateway-sample)
The system consists of two APIs, and the API gateway is configured to route requests between them.
It also implements:

- [Authentication](https://microsoft.github.io/reverse-proxy/articles/authn-authz.html)
- [Rate limiting](how-to-use-rate-limiting-in-aspnet-core)

In this newsletter, we only scratched the surface of what's possible with **YARP**.

Here are some useful resources if you want to learn more:

- [YARP docs](https://microsoft.github.io/reverse-proxy/articles/index.html)
- [Load balancing](https://microsoft.github.io/reverse-proxy/articles/load-balancing.html)
- [Session affinity](https://microsoft.github.io/reverse-proxy/articles/session-affinity.html)
- [Request/response transformations](https://microsoft.github.io/reverse-proxy/articles/transforms.html)

That's all for today.

Hope it was helpful.

**Today's action step:**
Download the [source code](https://github.com/m-jovanovic/yarp-api-gateway-sample) for the sample application implementing an API gateway with YARP, and take it for a spin.
You can challenge yourself by creating multiple instances of a single service and configuring load balancing with the various load balancing algorithms.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

And by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

</Sponsorships>

Reducing the size of your API responses can noticeably improve the performance of your application.

And since network bandwidth is a limited resource, you should at least consider the benefits of **response compression**.

Here's what you'll learn in this week's newsletter:

- How to configure **response compression** in .NET
- Server-based vs. application-based compression
- Possible **security risks** and **mitigation strategies**
- How to configure the available compression providers
- How much network bandwidth you could be saving

Let's get started!

## Configuring Response Compression

Using **response compression** in an .NET applications is remarkably easy.

You only have to call these two methods:

- `AddResponseCompression` - to configure the default services for response compression
- `UseResponseCompression` - to add the response compression middleware to the request pipeline

The `UseResponseCompression` method should be called before any middleware that compresses responses.

**Response compression** isn't turned on by default for HTTPS, so you have to enable it by setting `EnableForHttps` to `true`.

```csharp {3-6,10}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
});

var app = builder.Build();

app.UseResponseCompression();

app.MapGet("/", () => "This response will be compressed 📦");

app.Run();
```

It really is that simple.

But...

## When Should You Use Response Compression?

Ideally, you want to be using **server-based response compression** if your application server supports it.

Because the middleware performs **response compression** at the **application level**, it will typically have worse performance.

If you are hosting your application and you can't use server-based compression, then using the response compression middleware is justified.

One more concern should be **security**, because using **response compression over HTTPS** can expose you to [CRIME](https://en.wikipedia.org/wiki/CRIME) and [BREACH](https://en.wikipedia.org/wiki/BREACH) attacks

Here's what you can do to improve security:

- You can mitigate CRIME and BREACH attacks by introducing **anti-forgery tokens** in ASP.NET Core
- Don't send application secrets as part of the request body
- Implement a [**rate limiter**](how-to-use-rate-limiting-in-aspnet-core)

## Configuring Compression Providers

There are two compression providers added by default when you call `AddResponseCompression`:

- `BrotliCompressionProvider`
- `GzipCompressionProvider`

You can further customize the available providers by adding custom compression providers if you want to.

Compression will default to **Brotli** compression when it's supported by the client.
Otherwise, it will default to **Gzip** if that is the supported compression format.

```csharp {4,5,6}
builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
    options.Providers.Add<BrotliCompressionProvider>();
    options.Providers.Add<GzipCompressionProvider>();
    options.MimeTypes = ResponseCompressionDefaults.MimeTypes;
});
```

The interesting thing is you can configure the `CompressionLevel` for the **Brotli** and **Gzip** compression providers.

There are four possible values:

- `Optimal` - tries to balance response size and compression speed
- `Fastest` - sacrifices optimal compression for improved speed
- `NoCompression` - self explanatory
- `SmallestSize` - sacrifices compression speed to make the output as small as possible

The default value is `CompressionLevel.Fastest`.

```csharp
builder.Services.Configure<BrotliCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.Optimal;
});

builder.Services.Configure<GzipCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.SmallestSize;
});
```

## How Much Can You Save?

Let's find out how much network bandwidth we can save by using **response compression**.

Here's a minimal API endpoint returning a list of `Message` objects:

```csharp
app.MapGet("/", () => Results.Ok(
    Enumerable
        .Range(1, 100)
        .Select(num => new Message
        {
            Id = num,
            Content = $"This is the message #{num}"
        })));
```

And here are the results with different providers and compression levels:

- No compression - 4.5kB
- **Gzip** + `CompressionLevel.Fastest` - 569B
- **Gzip** + `CompressionLevel.SmallestSize` - 539B
- **Gzip** + `CompressionLevel.Optimal` - 554B
- **Brotli** + `CompressionLevel.Fastest` - 400B
- **Brotli** + `CompressionLevel.SmallestSize` - 296B
- **Brotli** + `CompressionLevel.Optimal` - 319B

**Brotli** is the clear winner, which is why it's the **default compression provider**.

In the best case scenario, you can reduce the response size by ~93.5%.
Multiply this by the number of requests you're serving daily, and then you can begin to estimate the possible network savings.

One more thing I noticed is that using `CompressionLevel.SmallestSize` has a noticeable **negative impact** on response time.
I can't say this was surprising, so I suggest to simply keep using the default compression level.

## In Summary

[Response compression](https://learn.microsoft.com/en-us/aspnet/core/performance/response-compression?view=aspnetcore-7.0)
is an interesting technique to improve API performance and reduce network costs.

Ideally, you'd want to be using **server-based response compression** if it's supported by your application server.
If that's not the case, **application-based compression** is available in .NET with the response compression middleware.

What's the cost of response compression?

It will increase the CPU load, and can expose some security risks over HTTPS, but there are ways to mitigate this.

In my experience, the **default configuration values** for the compression provider and compression level give excellent results.

That's all for this week.

See you next Saturday.

**Today's action step:**
To see the value of response compression, I suggest enabling it in your application and examining the changes in response size.
You can try the different compression providers by varying the `Accept-Encoding` header, and also configure different compression levels in your application.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**Fiverr.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)
Fiverr is your one-stop solution for finding skilled [**Web application developers.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)
Build your custom Web application from scratch with help from skilled freelance coders.
[**Check it out here!**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)

</Sponsorships>

Today's modern applications must deliver the latest information without refreshing the user interface.

If you need to introduce **real-time** functionality to your application in .NET, there's one library you will most likely reach for - **SignalR**.

**SignalR** allows you to push content from your server-side code to any connected clients as changes happen in real-time.

Here's what I'll teach you in this week's newsletter:

- Creating your first **SignalR** `Hub`
- Testing **SignalR** from **Postman**
- Creating strongly typed hubs
- Sending messages to a specific user

Let's see why **SignalR** is so powerful and how easy it is to build **real-time** applications with it.

## Installing And Configuring SignalR

To start using **SignalR** you'll need to:

- Install the NuGet package
- Create the `Hub` class
- Register the SignalR services
- Map and expose the hub endpoint so clients can connect to it

Let's start by installing the `Microsoft.AspNetCore.SignalR.Client` NuGet package:

```powershell
Install-Package Microsoft.AspNetCore.SignalR.Client
```

Then you need a SignalR `Hub`, which is the central component in your application responsible for managing clients and sending messages.

Let's create a `NotificationsHub` by inheriting from the base `Hub` class:

```csharp
public sealed class NotificationsHub : Hub
{
    public async Task SendNotification(string content)
    {
        await Clients.All.SendAsync("ReceiveNotification", content);
    }
}
```

The SignalR `Hub` exposes a few useful properties:

- `Clients` - used to invoke methods on the clients connected to this hub
- `Groups` - an abstraction for adding and removing connections from groups
- `Context` - used for accessing information about the hub caller connection

You can learn more about the `Hub` class [here](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub?view=aspnetcore-7.0).

Lastly, you need to register the SignalR services by calling the `AddSignalR` method.
You also need to call the `MapHub<T>` method, where you specify the `NotificationsHub` class and the path clients will use to connect to the hub.

```csharp {3, 7}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSignalR();

var app = builder.Build();

app.MapHub<NotificationsHub>("notifications-hub");

app.Run();
```

Now let's see how we can test the `NotificationsHub`.

## Connecting To SignalR Hub From Postman

To test SignalR, you need a client that will connect to the `Hub` instance.
You could create a simple application with Blazor or JavaScript, but I will show you a different approach.

We will use Postman's **WebSocket Request** to connect to the `NotificationsHub`.

<Image
  src="blogs/mnw_043/postman_websocket_request.png"
  width={1011}
  height={747}
/>

Here's what we need to do:

- Connect to the `NotificationsHub`
- Set the communication protocol to JSON
- Send messages to call the `NotificationsHub` methods

All messages need to end with a null termination character, which is just the ASCII character `0x1E`.

Let's start off by sending this message to set the communication protocol to JSON:

```json
{
  "protocol": "json",
  "version": 1
}?
```

You'll receive this response from the hub.

<Image
  src="blogs/mnw_043/postman_set_protocol_request.png"
  width={1731}
  height={890}
/>

We need a slightly different message format to call a message on the `Hub`.
The key is specifying the `arguments` and `target`, which is the actual hub method we want to call.

Let's say we want to call the `SendNotification` method on the `NotificationsHub`:

```json
{
  "arguments": ["This is the notification message."],
  "target": "SendNotification",
  "type": 1
}?
```

This will be the response we get back from the `NotificationsHub`:

<Image
  src="blogs/mnw_043/postman_send_notification_request.png"
  width={1726}
  height={816}
/>

## Strongly Typed Hubs

The base `Hub` class uses the `SendAsync` method to send messages to connected clients.
Unfortunately, we have to use strings to specify client-side methods to invoke, and it's easy to make a mistake.
There's also nothing enforcing which parameters are used.

SignalR supports **strongly typed hubs** that aim to solve this.

First, you need to define a client interface, so let's create a simple `INotificationsClient` abstraction:

```csharp
public interface INotificationsClient
{
    Task ReceiveNotification(string content);
}
```

The arguments don't have to be primitive types and can also be objects. SignalR will take care of serialization on the client side.

After that, you need to update the `NotificationsHub` class to inherit from the `Hub<T>` class to make it strongly typed:

```csharp
public sealed class NotificationsHub : Hub<INotificationsClient>
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
```

You will lose access to the `SendAsync` method, and only the methods defined in your client interface will be available.

## Sending Server-Side Messages With `HubContext`

What good is a `NotificationsHub` if we can't send notifications from the backend to connected clients?\
Not much.

You can use the `IHubContext<THub>` interface access to the `Hub` instance in your backend code.

And you can use `IHubContext<THub, TClient>` for a strongly typed hub.

Here's a simple Minimal API endpoint that injects an `IHubContext<NotificationsHub, INotificationsClient>` for our strongly typed hub and uses it
to send a notification to all connected clients:

```csharp {3}
app.MapPost("notifications/all", async (
    string content,
    IHubContext<NotificationsHub, INotificationsClient> context) =>
{
    await context.Clients.All.ReceiveNotification(content);

    return Results.NoContent();
});
```

## Sending Messages To a Specific User

The real value of SignalR is being able to **send messages**, or notifications in this example, to a **specific user**.

I've seen some complicated implementations that manage a dictionary with a user identifier and a map of active connections.
Why would you do that when SignalR already supports this functionality?

You can call the `User` method and pass it the `userId` to scope the `ReceiveNotification` message to that specific user.

```csharp {2,6}
app.MapPost("notifications/user", async (
    string userId,
    string content,
    IHubContext<NotificationsHub, INotificationsClient> context) =>
{
    await context.Clients.User(userId).ReceiveNotification(content);

    return Results.NoContent();
});
```

How does **SignalR** know which user to send the message to?

It uses the `DefaultUserIdProvider` internally to extract the user identifier from the claims.
To be specific, it's using the `ClaimTypes.NameIdentifier` claim.
This also implies that you should be authenticated when connecting to the `Hub`, for example, by passing a JWT.

```csharp
public class DefaultUserIdProvider : IUserIdProvider
{
    public virtual string? GetUserId(HubConnectionContext connection)
    {
        return connection.User?.FindFirst(ClaimTypes.NameIdentifier)?.Value;
    }
}
```

By default, all the methods in a hub can be called by unauthenticated users.
So you need to decorate it with an `Authorize` attribute to only allow authenticated clients to access the hub.

```csharp {1}
[Authorize]
public sealed class NotificationsHub : Hub<INotificationsClient>
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
```

## In Summary

Adding **real-time functionality** to your application creates room for innovation and adds value to your users.

With **SignalR**, you can start building real-time apps in .NET in minutes.

You need to grasp one concept - the `Hub` class.
SignalR abstracts away the message transport mechanism, so you don't have to worry about it.

Make sure to send authenticated requests to **SignalR** hubs and turn on authentication on the `Hub`.
SignalR will internally track the users connecting to your hubs, allowing you to send them messages based on the user identifier.

That's all for today.

Thanks for reading, and have an awesome Saturday.

**Today's action step:**
Look at your project and try to find an opportunity to add real-time functionality.
Commit 30 min. to build a simple proof of concept with SignalR and see if it can improve your project.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

And by [**ABP Commercial.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w42&utm_campaign=newsletter-sponsorship)
ABP Commercial is a complete [**web development platform built on ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w42&utm_campaign=newsletter-sponsorship),
perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.

</Sponsorships>

Is the **anemic domain model** an **antipattern**?

It's a domain model without any behavior and only data properties.

Anemic domain models work great in simple applications, but they are difficult to maintain and evolve if you have rich business logic.

The important parts of your business logic and rules end up being scattered all over the application.
It reduces cohesiveness and reusability, and makes adding new features more difficult.

**Rich domain model** attempts to solve this by encapsulating as much of the business logic as possible.

But how can you design a **rich domain model**?

This is a never-ending process of moving business logic into the domain and refining your domain model.

Let's see how to **refactor** from an **anemic domain model** to a **rich domain model**.

## Working With Anemic Domain Model

To understand what working with an **anemic domain model** looks like, I'll use an example of handling a `SendInvitationCommand`.

I omitted the class and its dependencies so that we can focus on the `Handle` method.
It loads some entities from the database, performs validation, executes the business logic, and finally persists the changes in the database and sends an email.

It already implements some good practices like using repositories and returning result objects.

However, it's working with an **anemic domain model**.

A few things indicating this:

- Parameterless constructors
- Public property setters
- Exposed collections

In other words - the classes representing domain entities contain only data properties and no behavior.

The **problems** of an **anemic domain model** are:

- Discoverability of operations
- Potential code duplication
- Lack of encapsulation

We'll apply a few techniques to push logic down into the domain, and try to make the model more domain-driven.
I hope you'll be able to see the value and benefits this will bring.

```csharp
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    if (gathering.Creator.Id == member.Id)
    {
        throw new Exception("Can't send invitation to the creator.");
    }

    if (gathering.ScheduledAtUtc < DateTime.UtcNow)
    {
        throw new Exception("Can't send invitation for the past.");
    }

    var invitation = new Invitation
    {
        Id = Guid.NewGuid(),
        Member = member,
        Gathering = gathering,
        Status = InvitationStatus.Pending,
        CreatedOnUtc = DateTime.UtcNow
    };

    gathering.Invitations.Add(invitation);

    _invitationRepository.Add(invitation);

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
```

## Moving Business Logic Into The Domain

The goal is to move as much of the business logic as possible into the domain.

Let's start with the `Invitation` entity and defining a constructor for it.
I can simplify the design by setting the `Status` and `CreatedOnUtc` properties inside the constructor.
I'm also going to make it `internal` so that an `Invitation` instance can only be created within the domain.

```csharp {3-10}
public sealed class Invitation
{
    internal Invitation(Guid id, Gathering gathering, Member member)
    {
        Id = id;
        Member = member;
        Gathering = gathering;
        Status = InvitationStatus.Pending;
        CreatedOnUtc = DateTime.Now;
    }

    // Data properties omitted for brevity.
}
```

The reason I made the `Invitation` constructor `internal` is so that I can introduce a new method on the `Gathering` entity.
Let's call it `SendInvitation` and it will be responsible for instantiating a new `Invitation` instance and adding it to the internal collection.

Currently, the `Gathering.Invitations` collection is `public`, which means anyone can obtain a reference and modify the collection.

We don't want to allow this, so what we can do is encapsulate this collection behind a `private` field.
This moves the responsibility for managing the `_invitations` collection to the `Gathering` class.

Here's how the `Gathering` class looks like now:

```csharp {3,7-12}
public sealed class Gathering
{
    private readonly List<Invitation> _invitations;

    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
```

## Moving Validation Rules Into The Domain

The next thing we can do is move the validation rules into the `SendInvitation` method, further enriching the domain model.

Unfortunately, this is still a bad practice because of throwing "expected" exceptions when a validation fails.
If you want to use exceptions to enforce your validation rules you should at least do it right, and use specific exceptions instead of generic ones.

But it would be even better to use a **result object** to express validation errors.

```csharp {7-10,12-15}
public sealed class Gathering
{
    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            throw new Exception("Can't send invitation to the creator.");
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            throw new Exception("Can't send invitation for the past.");
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
```

Here's how using **result objects** would look like:

```csharp {5,9,14,21}
public sealed class Gathering
{
    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        return Result.Success();
    }
}
```

The benefit of this approach is we can introduce constants for possible domain errors.
The catalog of domain errors will act as **documentation** for your domain, and make it more expressive.

Finally, here's how the `Handle` method looks like with all the changes so far:

```csharp {12-17}
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
```

If you take a closer look at the `Handle` method you'll notice it's doing two things:

- Persisting changes to the database
- Sending an email

This means it's **not atomic**.

There's a potential for the database transaction to complete, and the email sending to fail.
Also, sending the email will slow down the method which could affect performance.

How can make this method atomic?

By sending the email in the background. It's not important for our business logic, so this is safe to do.

## Expressing Side Effects With Domain Events

You can use **domain events** to express that something occurred in your domain that might be interesting to other components in your system.

I often use **domain events** to trigger actions in the background, like sending a notification or email.

Let's introduce an `InvitationSentDomainEvent`:

```csharp
public record InvitationSentDomainEvent(Invitation Invitation) : IDomainEvent;
```

We're going to raise this **domain event** inside the `SendInvitation` method:

```csharp {23}
public sealed class Gathering
{
    private readonly List<Invitation> _invitations;

    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        Raise(new InvitationSentDomainEvent(invitation));

        return Result.Success();
    }
}
```

The goal is to remove the code responsible for sending the email from the `Handle` method:

```csharp
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    return Result.Success();
}
```

We only want to worry about executing the business logic and persisting any changes to the database.
Part of those changes will also be the **domain event**, which the system will publish in the background.

Of course, we need a respective **handler** for the **domain event**:

```csharp
public sealed class InvitationSentDomainEventHandler
    : IDomainEventHandler<InvitationSentDomainEvent>
{
    private readonly IEmailService _emailService;

    public InvitationSentDomainEventHandler(IEmailService emailService)
    {
        _emailService = emailService;
    }

    public async Task Handle(InvitationSentDomainEvent domainEvent)
    {
        await _emailService.SendInvitationSentEmailAsync(
            domainEvent.Invitation.Member,
            domainEvent.Invitation.Gathering);
    }
}
```

We achieved two things:

- Handling the `SendInvitationCommand` is now atomic
- Email is sent in the background, and can be safely retried in case of an error

## Takeaway

Designing a **rich domain model** is a gradual process, and you can slowly evolve the domain model over time.

The first step could be making your domain model more defensive:

- Hiding constructors with the `internal` keyword
- Encapsulating collection access

The benefit is your domain models will have a fine-grained public API (methods) which act as an entry point for executing the business logic.

It's easy to test behavior when it's encapsulated in a class without having to mock external dependencies.

You can raise **domain events** to notify the system that something of important occurred, and any interested components can subscribe to that domain event.
Domain events allow you to develop a **decoupled** system, where you focus on the core domain logic, and don't have to worry about the side effects.

However, this doesn't mean that every system needs a **rich domain model**.

You should be pragmatic and decide when the complexity is worth it.

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Fiverr.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)
Fiverr is your one-stop solution for finding skilled freelance developers.
Build custom APIs and integrations tailored to your needs.
Seamlessly connect with payment providers (Stripe, PayPal) and automation platforms (Zapier) for streamlined operations.
Check it out [**here!**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)

And by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

</Sponsorships>

If you're building a **.NET** application, chances are high that you'll need to call an **external API** over **HTTP**.

The easy way to make HTTP requests in .NET is to use the `HttpClient` to send those requests.
And it's a great abstraction to work with, especially with the methods supporting **JSON** payloads and responses.

Unfortunately, it's easy to misuse the `HttpClient`.

**Port exhaustion** and **DNS behavior** are some of the most common problems.

So here's what you need to know about working with `HttpClient`:

- How not to use `HttpClient`
- How to simplify configuration with `IHttpClientFactory`
- How to configure **typed clients**
- Why you should avoid **typed clients** in singleton services
- When to use which option

Let's dive in!

## The Naive Way To Use HttpClient

The simplest way to work with the `HttpClient` is to just create a new instance, set the required properties and use it to send requests.

What could possibly go wrong?

`HttpClient` instances are meant to be **long-lived**, and reused throughout the lifetime of the application.

Each instance uses its own **connection pool** for isolation purposes, but also to prevent **port exhaustion**.
If a server is under high load, and your application is constantly creating new connections, it could lead to exhausting the available ports.
This will cause an exception at runtime, when trying to send a request.

So how can you avoid this?

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;

    public GitHubService(IOptions<GitHubSettings> settings)
    {
        _settings = settings.Value;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = new HttpClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## The Smart Way To Create HttpClient Using IHttpClientFactory

Instead of managing the `HttpClient` lifetime yourself, you can use an `IHttpClientFactory` to create the `HttpClient` instance.

Simply call the `CreateClient` method and use the returned `HttpClient` instance to send your HTTP requests.

Why is this a better approach?

The expensive part of the `HttpClient` is the actual message handler - `HttpMessageHandler`.
Each `HttpMessageHandler` has an internal HTTP **connection pool** that can be reused.

The `IHttpClientFactory` will **cache** the `HttpMessageHandler` and reuse it when creating a new `HttpClient` instance.

An important note here is that `HttpClient` instances created by `IHttpClientFactory` are meant to be **short-lived**.

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;
    private readonly IHttpClientFactory _factory;

    public GitHubService(
        IOptions<GitHubSettings> settings,
        IHttpClientFactory factory)
    {
        _settings = settings.Value;
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = _factory.CreateClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Reducing Code Duplication With Named Clients

Using `IHttpClientFactory` will solve most of the issues of manually creating an `HttpClient`.
However, we still need to configure the default request parameters every time we obtain a new `HttpClient` from the `CreateClient` method.

You can configure a **named client** by calling the `AddHttpClient` method and passing in the desired name.
The `AddHttpClient` accepts a delegate that you can use to configure the default parameters on the `HttpClient` instance.

```csharp
services.AddHttpClient("github", (serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

The main difference is you now have to obtain the client by passing the name of the client to `CreateClient`.

But the use of the `HttpClient` looks a lot simpler:

```csharp
public class GitHubService
{
    private readonly IHttpClientFactory _factory;

    public GitHubService(IHttpClientFactory factory)
    {
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = _factory.CreateClient("github");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Replacing Named Clients With Typed Clients

The downside of using **named clients** is having to resolve an `HttpClient` by passing in a name every time.

There's a better way to achieve the same behavior by configuring a **typed client**.
You can do this by calling the `AddClient<TClient>` method and configuring the service that will consume the `HttpClient`.

Under the hood, this is still using a **named client**, where the name is the same as the type name.

And this will also register `GitHubService` with a **transient lifetime**.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

Inside of `GitHubService`, you inject and use the typed `HttpClient` instance which will have all of the configuration applied.

No more dealing with `IHttpClientFactory` and creating `HttpClient` instances manually.

```csharp
public class GitHubService
{
    private readonly HttpClient client;

    public GitHubService(HttpClient client)
    {
        _client = client;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Why You Should Avoid Typed Clients In Singleton Services

You could run into a **problem** if you inject a **typed client** into a **singleton service**.
Since the **typed client** is **transient**, injecting it in a **singleton service** will cause it to be cached for the lifetime of the **singleton service**.

This will prevent the **typed client** from reacting to DNS changes.

If you want to use a **typed client** in a **singleton service**, the recommened approach is using `SocketsHttpHandler` as the primary handler,
and configuring the `PooledConnectionLifetime`.

Since the `SocketsHttpHandler` will handle connection pooling, you can disable recycling at the `IHttpClientFactory` level by setting `HandlerLifetime` to `Timeout.InfiniteTimeSpan`.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    return new SocketsHttpHandler()
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(15)
    };
})
.SetHandlerLifetime(Timeout.InfiniteTimeSpan);
```

## When Should You Use Which Option?

I showed you a few possible options for working with `HttpClient`.

But which one should you use and when?

Microsoft was kind enough to provide us with a set of best practices and [recommended use](https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient-guidelines#recommended-use)
for `HttpClient`.

- Use a `static` or **singleton** `HttpClient` instance with a `PooledConnectionLifetime` configured, since this solves both port exhaustion and tracking DNS changes
- Use `IHttpClientFactory` if you want to move the configuration to one place, but remember that clients are meant to be **short-lived**
- Use a **typed client** if you want the `IHttpClientFactory` configurability

I prefer working with a **typed client**, and I'm mindful of the fact that it's configured as a **transient service**.

Thanks for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

If you're building a scalable application, it's a common requirement to offload some work in your application to a **background job**.

Here are a few examples of that:

- Publishing email notifications
- Generating reports
- Updating a cache
- Image processing

How can you create a recurring **background job** in .NET?

[**Quartz.NET**](https://www.quartz-scheduler.net/) is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

There are three concepts you need to understand in **Quartz.NET**:

- **Job** - the actual background task you want to run
- **Trigger** - the trigger controlling when a job runs
- **Scheduler** - responsible for coordinating jobs and triggers

Let's see how we can use **Quartz.NET** to create and schedule **background jobs**.

## Adding The Quartz.NET Hosted Service

The first thing we need to do is install the **Quartz.NET** NuGet package.
There are a few to pick from, but we're going to install the `Quartz.Extensions.Hosting` library:

```powershell
Install-Package Quartz.Extensions.Hosting
```

The reason we're using this library is because it integrates nicely with .NET using an `IHostedService` instance.

To get the **Quartz.NET** hosted service up and running, we need two things:

- Add the required services with the DI container
- Add the hosted service

```csharp
services.AddQuartz(configure =>
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});
```

**Quartz.NET** will create jobs by fetching them from the DI container.
This also means you can use scoped services in your jobs, not just singleton or transient services.

Setting the `WaitForJobsToComplete` option to `true` will ensure that **Quartz.NET** waits for the jobs to complete gracefully before exiting.

## Creating Background Jobs With `IJob`

To crate a background job with **Quartz.NET** you need to implement the `IJob` interface.

It only exposes a single method - `Execute` - where you can place the code for your background job.

A few things worth noting here:

- We're using DI to inject the `ApplicationDbContext` and `IPublisher` services
- The job is decorated with `DisallowConcurrentExecution` to prevent running the same job concurrently

```csharp
[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List<OutboxMessage> messages = await _dbContext
            .Set<OutboxMessage>()
            .Where(m => m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject<IDomainEvent>(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}
```

Now that the **background job** is ready, we need to register it with the **DI** container and add a trigger that will run the job.

## Configuring the Job

I mentioned at the start that there are three key concepts in **Quartz.NET**:

- Job
- Trigger
- Scheduler

We already implemented the `ProcessOutboxMessagesJob` background job in the previous section.

The **Quartz.NET** library will take care of the scheduler.

And this leaves us with configuring the **trigger** for our `ProcessOutboxMessagesJob`.

```csharp
services.AddQuartz(configure =>
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob<ProcessOutboxMessagesJob>(jobKey)
        .AddTrigger(
            trigger => trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule => schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});
```

We need to uniquely identify our **background job** with a `JobKey`.
I like to keep it simple and use the job name.

Calling `AddJob` will register the `ProcessOutboxMessagesJob` with DI and also with Quartz.

After that we configure a trigger for this job by calling `AddTrigger`.
You need to associate the job with the trigger by calling `ForJob`, and then you configure the schedule for the background job.
In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.

**Quartz** also has support for configuring triggers using [cron expressions.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html)

## Job Persistence

By default, Quartz configures all jobs using the `RAMJobStore` which is the most performant because it keeps all of its data in RAM.
However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.

It could be useful to have a persistent job store in some scenarios and there's a built in `AdoJobStore` which works with SQL databases.
You need to create a set of database tables for Quartz.NET to use.

You can learn more about this in the [job stores documentation.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html)

## Takeaway

**Quartz.NET** makes running **background jobs** in .NET easy, and you can use all the power of DI in your **background jobs**.
It's also flexible for various scheduling requirements with configuration via code or using cron expressions.

There's some room for improvement to make scheduling jobs easier and reduce boilerplate:

- Add an extension method to simplify configuring jobs with a simple schedule
- Add an extension method to simplify configuring jobs with a cron schedule from application settings

If you want to see a tutorial on using **Quartz.NET**, I made an in-depth video about [**using Quartz for processing Outbox messages**](https://youtu.be/XALvnX7MPeo).

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Tower.**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)
Try Tower, a [**powerful Git client for Windows**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly) focused on productivity.
Make the complex simple with helpful features like Undo, Drag and Drop and Remote Services Integration!

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

Do you want to streamline your software development process and accelerate your release cycles?

Imagine being able to automatically build, test, and deploy your .NET applications with every code change.

With **CI/CD**, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.

And it's never been easier to get started with **CI/CD**.

**GitHub Actions** are completely free and simple to use.

So here's what we'll cover:

- Introduction to **CI/CD** & **GitHub Actions**
- Creating a **build & test pipeline** for **.NET**
- Creating a **deployment pipeline** for **Azure App Service**

Let's dive in.

## What Is Continuous Integration And Delivery?

I'll try to briefly explain what **CI/CD** is, before we take a look at **GitHub Actions**.

**CI/CD** is a method to increase the frequency of delivering new features by adding automation to your software development workflow.

**Continuous Integration ("CI")** refers to the automation process of syncing new code to a repository.
Any new changes to the application code are immediately built, tested and merged.

**Continuous Delivery, or Deployment, ("CD")** refers to the process of automating the deployment part of the workflow.
When you make a change which gets merged to the repository, this step takes care of deploying those changes to the production environment (or any other environment).

## Continuous Integration With GitHub Actions

If you're using **GitHub**, getting started with **Continuous Integration** has never been easier.

You can use [**GitHub Actions**](https://github.com/features/actions) to automate your build, test, and deployment **pipeline**.
You can create workflows that build and test every commit to your repository, or deploy to production when a new tag is created.

To create a **GitHub Action**, you write a _workflow_ to be triggered when some _event_ occurs in your repository.
An example event is a commit to the main branch, creation of a tag, or you can manually run the workflow.

Here's a **GitHub Actions** workflow to build and test a .NET project:

```yaml
name: Build & Test 🧪

on:
  push:
    branches:
      - main

env:
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Install dependencies 📂
        run: dotnet restore WebApi

      - name: Build 🧱
        run: dotnet build WebApi --configuration Release --no-restore

      - name: Test 🧪
        run: dotnet test WebApi --configuration Release --no-build
```

Let's unwrap what is happening here:

- Defining an event to trigger the workflow
- Setting up the **.NET SDK** with the version from `env.DOTNET_VERSION`
- Restoring, building and testing the project using the `dotnet` CLI tool

You can add this to your **GitHub** repository today, and start getting instant feedback when you commit code to the repository.

When a workflow run fails due to a build error or a failed test, you'll get an email notification.

## Continuous Delivery To Azure With GitHub Actions

**Continuous Integration** is a great way to start out with **CI/CD**, but the real value lies in **automating** your **deployment** process.

Imagine this:

- You make a change to your codebase
- The commit triggers the **deployment pipeline**
- A few minutes later your changes are live in production

Usually it's a little more nuanced, because we need to think about configuration, database migrations, etc.
But try to see the big picture here.

If you're running your application in the cloud, for example on **Azure**, chances are there's an existing **GitHub Action** you can use.

Here's a **deployment pipeline** I use to publish my application to an **Azure App Service** instance:

```yaml
name: Publish 🚀

on:
  push:
    branches:
      - main

env:
  AZURE_WEBAPP_NAME: web-api
  AZURE_WEBAPP_PACKAGE_PATH: './publish'
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Build and Publish 📂
        run: |
          dotnet restore WebApi
          dotnet build WebApi -c Release --no-restore
          dotnet publish WebApi -c Release --no-build
            --output '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'

      - name: Deploy to Azure 🌌
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          publish-profile: ${{ secrets.AZURE_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
```

This workflow is pretty similar to the previous one, with the differences being:

- Adding a publish step and configuring the output path
- Using the `azure/webapps-deploy@v2` action to deploy to **Azure**

If you need to safely and securely expose secret values in your workflows, you can use **GitHub secrets**.
You can define the secrets on GitHub, and use them in actions without having to add them to source control.

In the deployment workflow I'm using `secrets.AZURE_PUBLISH_PROFILE` to access my publish profile for the App Service instance.

## In Summary

**Continuous Integration and Delivery** can transform your development process by increasing the speed at which you release changes.

Try adding up how much time you spend on deployments.
I'm pretty sure you'll be surprised by the time savings potential of automating them.

And the good part is you will typically set up your **build and deployment pipelines** once, and then continue benefiting from them for the lifetime of your project.

If you want a step by step guide, I made a video showing [**how to implement a CI/CD pipeline from scratch.**](https://youtu.be/QP0pi7xe24s)
And the **source code** is also public, so you can add the **GitHub Actions** workflow file to your project.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Refactoring.**](https://refactoring.club/?utm_source=milan)
Refactoring is a weekly newsletter [**about making great software and working well with humans.**](https://refactoring.club/?utm_source=milan)
It is read every week by 35,000+ developers, managers and leaders at companies like Google, Meta, and Amazon.
Check it out [**here.**](https://refactoring.club/?utm_source=milan)

And by [**Jam.**](https://jam.dev/)
Reduce time spent on bug reporting with [**Jam**](https://jam.dev/) by 60-80%.
Auto-capture dev logs & network requests in one link, access Instant Replays for hard-to-repro bugs and integrate with Jira, Slack & more.
Free Team Tier for the rest of 2023.
[**Get Jam Now.**](https://jam.dev/)

</Sponsorships>

Most software applications today are built around the concept of **multi-tenancy**.

One application serves multiple customers, while keeping their data **isolated**.

You can approach **multi-tenancy** in two ways:

- **Single database** and **logical isolation** of tenants
- **Multiple databases** and **physical isolation** of tenants

Which option you decide to use will depend mostly on your requirements.
Some industries like healthcare require a high degree of **data isolation**, and using a **database per tenant** is a must.

So how are we going to **implement multi-tenancy** support with **EF Core**?

We can use **Query Filters** to apply a **tenant filter** to all database queries.

Implement it once and you can almost forget about it.

Let's see what are some of the problems we need to solve.

## How To Use EF Core Query Filters

If you want an in-depth dive into **Query filters**, take a look at the newsletter issue where I talked about
[**using query filters with EF Core.**](how-to-use-global-query-filters-in-ef-core)

Here's a quick refresher on **Query filters**:

- Configure the **query filter** by calling `HasQueryFilter` for your entity
- **EF** will apply it to all queries for that entity
- You can turn it off with `IgnoreQueryFilters`
- Only **one** query filter **per entity** is allowed

And here's a simple example:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

All queries to the `Order` table will include an `IsDeleted = FALSE` condition.

## Single Database Multi-Tenancy With EF Core

You will need two things to **implement multi-tenancy** on a **single database**:

- A way to know **who** the **current tenant** is
- A way to **filter** the **data** for that **tenant** only

The typical approach for **multi-tenancy** on a **single database** is having a `TenantId` column in your tables.
And then filtering on that column when querying the database.

You can use the **Query filters** feature in **EF Core** to apply a **global filter** for some entity.

Inside of the `OnModelCreating` method we configure the query filter on the `Order` entity:

```csharp
public class OrdersDbContext : DbContext
{
    private readonly string _tenantId;

    public OrdersDbContext(
        DbContextOptions<OrdersDbContext> options,
        TenantProvider tenantProvider)
        : base(options)
    {
        _tenantId = tenantProvider.TenantId;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder
            .Entity<Order>
            .HasQueryFilter(o => o.TenantId == _tenantId);
    }
}
```

We're using the `TenantProvider` class to get the **current tenant** value.

Here's what the `TenantProvider` implementation looks like:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;

    public TenantProvider(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];
}
```

The `TenantId` is coming from the HTTP request header in this example.

A few other options to get the `TenantId` are:

- Query string - `api/orders?tenantId=example-tenant-id`
- JWT Claim
- API Key

If you want a more **secure implementation** you should go with **JWT Claims** or **API Keys** to provide the `TenantId` value.

## Separate Databases Multi-Tenancy With EF Core

What if we want to **isolate** each tenant to a **separate database**?

Here are the changes we need to make:

- Applying different **connection string per tenant**
- Resolving the connection string for each tenant _somehow_

You can't use **Query filters** here, since we are working with different databases.

So you will need to store the **tenant information** and **connection strings** somewhere.

A simple example would be store them in the **application settings**:

```json
"Tenants": {
    { "Id": "tenant-1", "ConnectionString": "Host=tenant1.db;Database=tenant1" },
    { "Id": "tenant-2", "ConnectionString": "Host=tenant2.db;Database=tenant2" }
}
```

You can then register an `IOptions` instance with a list of `Tenant` objects.

And we need to slightly modify the `TenantProvider` class to return a **connection string** for the current tenant:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;
    private readonly TenantSettings _tenantSettings;

    public TenantProvider(
        IHttpContextAccessor httpContextAccessor,
        IOptions<TenantSettings> tenantsOptions)
    {
        _httpContextAccessor = httpContextAccessor;
        _tenants = tenantsOptions.Value;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];

    public string GetConnectionString()
    {
        return _tenantSettings.Tenants.Single(t => t.Id == TenantId);
    }
}
```

And the last part is registering your `DbContext` to **dynamically resolve** the **connection string** for the current **tenant**.

```csharp
builder.Services.AddDbContext<OrdersDbContext>((sp, o) =>
{
    var tenantProvider = sp.GetRequiredService<TenantProvider>();

    var connectionString = tenantProvider.GetConnectionString();

    o.UseSqlServer(connectionString);
});
```

On every request, we create a new `OrdersDbContext` and connect to the appropriate database for that tenant.

You should definitely consider storing the tenant **connection strings** in a **secure** place like **Azure Key Vault**.

## Closing Thoughts

I hope you now have a better understanding of how to build a **multi-tenant system** with **EF Core**.

I showed you the bare bones implementation, which you can improve to make it more robust and secure.

Building **multi-tenant systems** isn't easy, but when you understand the basic principles it shouldn't be too difficult either.

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel**](https://u.icepanel.io/1eec0e6f).
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** diagrams are a great way to communicate how you are planning to build a software system or how an existing software system works.

However, the majority of software architecture **diagrams** I've seen are a total mess.

If only there was a standard way to visualize your software architecture...

Enter the **C4 model**, which stands for **context**, **containers**, **components**, and **code**.

The **C4 model** is a lightweight approach to describing your **software architecture**.

And it's essentially just two things, a predefined set of common **abstractions** and four diagram types:

- **Context diagram** - high-level view of the system
- **Container diagram** - shows the objects running inside a system
- **Component diagram** - shows the building blocks that make each container
- **Code diagram** - rarely used, typically UML or ER diagram

Let's see how we can use the **C4 model** to visualize our **software architecture**.

## First You Need Abstractions

Before we dive into the **C4 diagrams**, we first need to understand the ubiquitous language of the **C4 model**.

> A software system is made up of one or more containers (applications and data stores),
> each of which contains one or more components, which in turn are implemented by one or more code elements (classes, interfaces, objects, functions, etc).
> And people may use the software systems that we build.

The **C4 model** defines a set of **abstractions** that you can use to describe your software systems:

- Person - the end user using your system
- Software system - highest level of abstraction that delivers value to end users
- Container - applications and data stores that make up a system
- Component - building blocks/modules that make up the container

With these high-level concepts in place, let's take a look at how to use them in our **C4 diagrams**.

## System Context Diagram

The **System Context diagram** is a high-level view of your software system.

It shows your software system as the central part, and any external systems and users that your system interacts with.

It should be **technology agnostic**, and the focus on the people and software systems instead of low-level details.

<Image
  src="blogs/mnw_037/system_context_diagram.jpg"
  width={2116}
  height={1172}
/>

The intended audience for the **System Context Diagram** is _everybody_.

If you can show it to non-technical people and they are able to understand it, then you know you're on the right track.

## Container Diagram

When you zoom into one software system, you get to the **Container diagram**.

Your software system is comprised of multiple running parts - **containers**.

A **container** can be a:

- Web application
- Single-page application
- Database
- File system
- Object store
- Message broker

You can look at a **container** as a **deployment unit** that executes code or stores data.

The **Container diagram** shows the high-level view of the software architecture and the **major technology choices**.

<Image src="blogs/mnw_037/container_diagram.jpg" width={1972} height={1732} />

The **Container diagram** is intended for technical people inside and outside of the software development team:

- Software architects
- Developers
- Operations/support staff

## Component Diagram

Next you can zoom into an individual **container** to decompose it into its building blocks.

The **Component diagram** show the individual **components** that make up a **container**:

- What each of the components are
- The technology and implementation details

<Image src="blogs/mnw_037/component_diagram.jpg" width={2100} height={1524} />

The **Component diagram** is intended for software architects and developers.

## Code Diagram

Finally, you can zoom into each **component** to show how it is implemented with **code**, typically using a UML class diagram or an ER diagram.

This level is rarely used as it goes into too much technical detail for most use cases.

However, there are supplementary diagrams that can be useful to fill in missing information by showcasing:

- Sequence of events
- Deployment information
- How systems interact at a higher level

It's only recommended for the most **important or complex components**.

Of course, the target audience are software architects and developers.

## Takeaway

The **C4 model** is a tool for visual and verbal communication, enabling your team to talk in the same language and have a common understanding of the design.

It's valuable for many reasons, to name a few:

- Product and business people benefit from understanding user behavior at the Context level
- Context and Container diagrams provide a simple view of how software systems work
- New developers can quickly understand system flows

Now that you understand what the **C4 model** is, there's one question left to answer.

Should you use the **C4 model** to express your **software architecture**?

I can't give you a definitive answer.

I used it with a lot of success.

But I think you should at least give it a try.

You can learn more about the **C4 model** [here.](https://c4model.com/)

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** is a blueprint for how you should structure your system.
You can follow this blueprint strictly, or you can give yourself varying levels of freedom.

But when deadlines are tight, and you start cutting corners, that beautiful software architecture you built crumbles like a house of cards.

How can you **enforce** your **software architecture**?

By writing **architecture tests**.

**Architecture tests** are automated tests that verify the structure and design of your code.

You can use them to enforce your software architecture and the direction of dependencies of your projects.

In this week's issue I'll explain how to:

- Write architecture tests
- Enforce architecture
- Enforce design rules

Let's dive in!

## Writing Architecture Tests

You write **architecture tests** the same as any unit test in your application.
There's an excellent library for writing architecture tests that already implements the boilerplate code we need to start writing tests.

We're going to use the `NetArchTest.Rules` library for writing architecture tests.

First, you have to install the NuGet package:

```powershell
Install-Package NetArchTest.Rules
```

And now you can use it to write rules in your test project.

The starting point for writing architecture tests is the static `Types` class, which you can use to load a set of types.

Once you have loaded your types you can further filter them to find a more specific set of types.

Some of the available filtering methods:

- `ResideInNamespace`
- `AreClasses`
- `AreInterfaces`
- `HaveNameStartingWith`
- `HaveNameEndingWith`

Finally, when you are satisfied with your selection, you can write the rule you want to enforce by calling `Should` or `ShouldNot`
and applying the condition you want to check.

Here's an example checking that all classes in the domain assembly are sealed:

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .That()
  .AreClasses()
  .Should()
  .BeSealed()
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Architecture Rules

**Architecture tests** are particularly useful to **enforce software architecture rules** in a layered architecture or **Modular Monolith**.

Let's take the example of the Clean architecture:

- Domain should not have any dependencies
- Application should not depend on Infrastructure
- Infrastructure should depend on Application and Domain

Here's how you can write tests for enforcing architecture rules.

**Domain should not have any dependencies**

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .ShouldNot()
  .HaveDependencyOnAny("Application", "Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Application should not depend on Infrastructure**

```csharp
var result = Types
  .InAssembly(AplicationAssembly)
  .Should()
  .NotHaveDependencyOn("Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Infrastructure should depend on Application and Domain**

How the `NetArchTest.Rules` library works is by scanning the imported namespaces of your types.

Because of this, writing negative conditions like in the previous two examples is straightforward.

But writing positive conditions has to be scoped to a more specific set of types.

For example, we can validate this dependency by checking that all repositories must have a dependency on the `Domain` namespace.

```csharp
var result = Types
  .InAssembly(InfrastructureAssembly)
  .HaveNameEndingWith("Repository")
  .Should()
  .HaveDependencyOn("Domain")
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Design Rules

Another valuable **use case** for **architecture tests** is **enforcing design rules** in your application.

Design rules are more specific than project references, and focus on the implementation details of your classes.

Here are some **design rules** that you can enforce:

- Services must be internal
- Entities and Value objects must be sealed
- Controllers can't depend on repositories directly
- Command (or query) handlers must follow a naming convention

The possibilities are endless, and it's up to you how many design rules you want to enforce.

Here's how you can write tests for enforcing design rules.

**Command handlers must end with `CommandHandler`**

```csharp
var result = Types
    .InAssembly(ApplicationAssembly)
    .That()
    .ImplementInterface(typeof(ICommandHandler))
    .Should()
    .HaveNameEndingWith("CommandHandler")
    .GetResult();

Assert.True(result.IsSuccessful);
```

**Controllers can't directly reference repositories**

```csharp
var result = Types
    .InAssembly(ApiAssembly)
    .That()
    .HaveNameEndingWith("Controller")
    .ShouldNot()
    .HaveDependencyOn("Infrastructure.Repositories")
    .GetResult();

Assert.True(result.IsSuccessful);
```

## Takeaway

**Architecture tests** are an easy way to **enforce software architecture** and design rules with automated tests.

One of the best investments you can make as a software engineer is writing automated tests.
You write the tests once, and use them to verify your system forever.
Granted, you also have to maintain the tests over time as your system grows.

Manually enforcing software architecture with pair programming and constant PR reviews is:

- Error prone
- Time consuming
- Not cost effective

**Architecture tests** really shine here, since you can write them quickly and reduce the cost of enforcing your software architecture rules to zero.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

We all want to build **robust** and **reliable** applications that can scale indefinitely and handle any number of requests.

But with **distributed systems** and **microservices architectures** growing in complexity, it's becoming increasingly harder to **monitor** the **health** of our applications.

It's vital that you have a system in place to receive quick feedback of your application **health.**

That's where **health checks** come in.

**Health checks** provide a way to monitor and verify the health of various components of an application including:

- Databases
- APIs
- Caches
- External services

Here's what I'll show you in this week's newsletter:

- [What are health checks](#what-are-health-checks)
- [Adding a custom health check](#adding-custom-health-checks)
- [Using existing health check libraries](#using-existing-health-check-libraries)
- [Customizing the health checks response format](#formatting-health-checks-response)

Let's see how to implement **health checks** in **ASP.NET Core**.

## What Are Health Checks?

**Health checks** are a proactive mechanism for monitoring and verifying the **health** and **availability** of an application in **ASP.NET Core.**

ASP.NET Core has **built-in support** for implementing **health checks.**

Here's the basic configuration, which registers the health check services and adds the `HealthCheckMiddleware` to respond at the specified URL.

```csharp {3,7}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks("/health");

app.Run();
```

The health check returns a `HealthStatus` value indicating the health of the service.

There are three distinct `HealthStatus` values:

- `HealthStatus.Healthy`
- `HealthStatus.Degraded`
- `HealthStatus.Unhealthy`

You can use the `HealthStatus` to indicate the different states of your application.

For example, if the application is functioning slower than expected you can return `HealthStatus.Degraded`.

## Adding Custom Health Checks

You can create **custom health checks** by implementing the `IHealthCheck` interface.

For example, you can implement a check to see if your **SQL** database is available.

It's important to use a query that can complete quickly in the database, like `SELECT 1`.

Here's a **custom health check** implementation example in the `SqlHealthCheck` class:

```csharp
public class SqlHealthCheck : IHealthCheck
{
    private readonly string _connectionString;

    public SqlHealthCheck(IConfiguration configuration)
    {
        _connectionString = configuration.GetConnectionString("Database");
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            using var sqlConnection = new SqlConnection(_connectionString);

            await sqlConnection.OpenAsync(cancellationToken);

            using var command = sqlConnection.CreateCommand();
            command.CommandText = "SELECT 1";

            await command.ExecuteScalarAsync(cancellationToken);

            return HealthCheckResult.Healthy();
        }
        catch(Exception ex)
        {
            return HealthCheckResult.Unhealthy(
                context.Registration.FailureStatus,
                exception: ex);
        }
    }
}
```

After you implement the **custom health check**, you need to register it.

The previous call to `AddHealthChecks` now becomes:

```csharp {2}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
```

We're giving it a custom name and setting which status to use as the failure result in `HealthCheckContext.Registration.FailureStatus`.

But stop and think for a moment.

Do you want to implement a **custom health check** on your own for **every external service** that you have?

Of course not! There's a better solution.

## Using Existing Health Check Libraries

Before you start implementing a custom **health check** for everything, you should first see if there's already an **existing library.**

In the [`AspNetCore.Diagnostics.HealthChecks`](https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks) repository you can find
a wide collection **health check** packages for frequently used services and libraries.

Here are just a few examples:

- SQL Server - `AspNetCore.HealthChecks.SqlServer`
- Postgres - `AspNetCore.HealthChecks.Npgsql`
- Redis - `AspNetCore.HealthChecks.Redis`
- RabbitMQ - `AspNetCore.HealthChecks.RabbitMQ`
- AWS S3 - `AspNetCore.HealthChecks.Aws.S3`
- SignalR - `AspNetCore.HealthChecks.SignalR`

Here's how to add health checks for **PostgreSQL** and **RabbitMQ**:

```csharp {3,4}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
    .AddNpgSql(pgConnectionString)
    .AddRabbitMQ(rabbitConnectionString)
```

## Formatting Health Checks Response

By default, the endpoint returning you **health check** status will return a string value representing a `HealthStatus`.

This isn't practical if you have **multiple health checks** configured, as you'd want to view the health status individually per service.

To make matters worse, if one of the services is failing the entire response will return `Unhealthy` and you don't know what's causing the issue.

You can solve this by providing a `ResponsWriter`, and there's an existing one in the `AspNetCore.HealthChecks.UI.Client` library.

Let's install the **NuGet** package:

```powershell
Install-Package AspNetCore.HealthChecks.UI.Client
```

And you need to slightly update the call to `MapHealthChecks` to use the `ResponseWriter` coming from this library:

```csharp {3-6}
app.MapHealthChecks(
    "/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });
```

After making these changes, here's what the response from the health check endpoint looks like:

```json
{
  "status": "Unhealthy",
  "totalDuration": "00:00:00.3285211",
  "entries": {
    "npgsql": {
      "data": {},
      "duration": "00:00:00.1183517",
      "status": "Healthy",
      "tags": []
    },
    "rabbitmq": {
      "data": {},
      "duration": "00:00:00.1189561",
      "status": "Healthy",
      "tags": []
    },
    "custom-sql": {
      "data": {},
      "description": "Unable to connect to the database.",
      "duration": "00:00:00.2431813",
      "exception": "Unable to connect to the database.",
      "status": "Unhealthy",
      "tags": []
    }
  }
}
```

## Takeaway

Application monitoring is important to track availability, resource usage, and changes to performance in your application.

I've used **health checks** before to implement **failover scenarios** in a **cloud deployment**.
When one application instance stops responding with a healthy result, a new one is created to continue serving requests.

It's easy to monitor the health of your ASP.NET Core applications by **exposing health checks** for your services.

You can decide to implement **custom health checks**, but first consider if there are **existing solutions**.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

What happens when a **message is retried** in an event-driven system?

It happens more often than you think.

The **worst case scenario** is that the **message is processed twice**, and the **side effects** can also be applied more than once.

Do you want your bank account to be double charged?

I'll assume the answer is no, of course.

You can use the **Idempotent Consumer** pattern to solve this problem.

In this week's issue I will show you:

- How the Idempotent Consumer pattern works
- How to implement an Idempotent Consumer
- The tradeoffs you need to consider

Let's see why the **Idempotent Consumer** pattern is valuable.

## How The Idempotent Consumer Pattern Works

What's the idea behind the **Idempotent Consumer pattern**?

> An idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

We want to avoid handling the same message more than once.

This would require **Exactly-once** message delivery guarantees from our messaging system.
And this is a really hard problem to solve in distributed systems.

A looser delivery guarantee is **At-least-once**, where we are aware that retries can happen and we can receive the same message more than once.

The **Idempotent Consumer** pattern works well with **At-least-once** message delivery, and solves the **problem of duplicate messages.**

Here's what the algorithm looks like from the moment we receive a message:

1. Was the message already processed?
2. If yes, it's a duplicate and there's nothing to do
3. If not, we need to handle the message
4. We also need to store the message identifier

<Image
  src="blogs/mnw_034/idempotent_consumer_algorithm.png"
  width={400}
  height={634}
/>

We need a **unique identifier** for every **message** we receive, and a table in the database to store processed messages.

However, it's interesting how we choose the implement the message handling and storing of the processed message identifier.

You can implement the idempotent consumer as a decorator around a regular message handler.

I'll show you two implementations:

- Lazy idempotent consumer
- Eager idempotent consumer

## Lazy Idempotent Consumer

The **lazy idempotent consumer** matches the flow shown in the algorithm above.

Lazy refers to how we store the message identifer to mark it as processed.

In the happy path, we handle the message and store the message identifier.

If the message handler throws an exception, we never store the message identifier and the consumer can be executed again.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        if (_messageRepository.IsProcessed(message.Id))
        {
            return;
        }

        await _decorated.Handle(message);

        _messageRepository.Store(message.Id);
    }
}
```

## Eager Idempotent Consumer

The **eager idempotent consumer** is slightly different from the lazy implementation, but the end result is the same.

In this version, we eagerly store the message identifier in the database and then continue to handle the message.

If the handler throws an exception, we need to perform cleanup in the database and remove the eagerly stored message identifier.

Otherwise, we risk leaving the system in an inconsistent state since the message was never handled correctly.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        try
        {
            if (_messageRepository.IsProcessed(message.Id))
            {
                return;
            }

            _messageRepository.Store(message.Id);

            await _decorated.Handle(message);
        }
        catch (Exception e)
        {
            _messageRepository.Remove(message.Id);

            throw;
        }
    }
}
```

## In Summary

Idempotency is an interesting problem to solve in a software system.

Some operations are **naturally idempotent**, and we don't need the overhead of the **Idempotent Consumer** pattern.

However, for those operations that aren't naturally idempotent, the **Idempotent Consumer** is a great solution.

The high-level algorithm is simple, and you can take two approaches in the implementation:

- Lazy storing of message identifiers
- Eager storing of message identifiers

I prefer to use the **lazy approach**, and only **store the message identifier** in the database when the **handler completes successfully.**

It's easier to reason about and there is one less call to the database.

Thanks for reading.

Hope that was helpful.

---



<Sponsorships>

This week's issue is sponsored by the [**Why You Should Join Newsletter.**](https://whyyoushouldjoin.substack.com/)
[**Why You Should Join**](<(https://whyyoushouldjoin.substack.com/)>) is a free, monthly newsletter doing deep dives on the most promising startups to join.
They receive inside info from Sequoia, Index, and [**other top VCs**](https://whyyoushouldjoin.substack.com/about) about which companies are doing best and recommend just one each month.
Prior suggestions [**Warp**](https://whyyoushouldjoin.substack.com/p/warp) and [**Pinecone**](https://whyyoushouldjoin.substack.com/p/pinecone)
have since raised significant follow-on funding - [**subscribe for free**](https://whyyoushouldjoin.substack.com/) so you don't miss the next Uber or Airbnb.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

**EF Core** is getting many new and exciting features in the upcoming version.

**EF7** introduced support for returning **scalar types** using **SQL** queries.

And now we're getting support for **querying unmapped types** with **raw SQL queries** in **EF8.**

This is exactly what **Dapper** offers out of the box, and it's good to see **EF Core** catching up.

In this week's newsletter, I'm going to cover how to use **EF Core** for:

- [Raw SQL queries](#ef-core-and-sql-queries)
- [Composing SQL queries with LINQ](#composing-sql-queries-with-linq)
- [Executing data modifications with SQL](#sql-queries-for-data-modifications)

Let's dive in!

## EF Core And SQL Queries

**EF7** added support for **raw SQL queries** returning scalar types.
**EF8** is taking this a step further with raw SQL queries that can return any mappable type, without having to include it in the **EF model**.

You can query unmapped types with the `SqlQuery` and `SqlQueryRaw` methods.

The `SqlQuery` method uses string interpolation to parameterize the query, protecting against SQL injection attacks.

Here's an example query returning an `OrderSummary` list:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .ToListAsync();
```

This will be the **SQL** sent to the database:

```sql
SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
```

The type used for the query result can have a parameterized constructor.
The property names don't need to match the column names in the database, but they do have to match the names of the values in the result set.

You can also execute raw SQL queries and return results with:

- Views
- Functions
- Stored procedures

## Composing SQL Queries With LINQ

An interesting thing about `SqlQuery` is that it returns `IQueryable`, which can be further composed with **LINQ.**

You can add a `Where` statement after calling `SqlQuery`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>("SELECT * FROM OrderSummaries AS o")
    .Where(o => o.CreatedOn >= startDate)
    .ToListAsync();
```

However, the generated **SQL** isn't optimal:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o
) AS s
WHERE s.CreatedOn >= @p0
```

Another possibility is to combine an `OrderBy` statement with `Skip` and `Take`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .OrderBy(o => o.Id)
    .Skip(10)
    .Take(5)
    .ToListAsync();
```

This would be the generated **SQL** for the previous query:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
) AS s
ORDER BY s.Id
OFFSET @p1 ROWS FETCH NEXT @p2 ROWS ONLY
```

In case you're wondering, the performance is similar to **LINQ** queries using `Select` projections.

I ran some benchmarks, and didn't notice any significant performance improvement.

This feature will be very useful if you're more comfortable with writing **SQL** or you want to fetch data from views, functions, and stored procedures.

## SQL Queries For Data Modifications

If you want to modify data in the database with **SQL**, you will typically write a query that doesn't return a result.

The SQL query can be an `UPDATE` or `DELETE` statement, or even a stored procedure call.

You can use the `ExecuteSql` method to execute this type of query with **EF Core**:

```csharp
var startDate = new DateOnly(2023, 1, 1);

dbContext.Database.ExecuteSql(
    $"UPDATE Orders SET Status = 5 WHERE CreatedOn >= {startDate}");
```

`ExecuteSql` also protects from SQL injection by parameterizing arguments, just like `SqlQuery`.

With **EF7** you can write the above query with **LINQ** and the `ExecuteUpdate` method.
There's also the `ExecuteDelete` method for deleting records.

## In Summary

**EF7** introduced support for raw SQL queries returning **scalar** values.

**EF8** will add support for **raw SQL queries** returning **unmapped types** with `SqlQuery` and `SqlQueryRaw`.

I like the direction that **EF** is going, introducing more flexibility for querying the database.

The performance isn't as good as **Dapper**, unfortunately.
But it's close enough that network costs will play the bigger factor.

I will probably be using only **EF** moving forward since it covers more use cases.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

Rate limiting is a technique to limit the number of requests to a server or an API.

A limit is introduced within a given time period to prevent server overload and protect against abuse.

In ASP.NET Core 7, we have a built-in rate limiter middleware that's easy to integrate into your API.

We're going to cover four rate limiter algorithms:

- [Fixed window](#fixed-window-limiter)
- [Sliding window](#sliding-window-limiter)
- [Token bucket](#token-bucket-limiter)
- [Concurrency](#concurrency-limiter)

Let's see how we can work with rate limiting.

## What Is Rate Limiting?

Rate limiting is about restricting the number of requests to an API, usually within a specific time window or based on other criteria.

This is practical for a few reasons:

- Prevents overloading of servers or applications
- Improves security and guards against DDoS attacks
- Reduces costs by preventing unnecessary resource usage

In a multi-tenant application, each unique user can have a limitation on the number of API requests.

## Configuring Rate Limiting

ASP.NET Core 7 introduced built-in rate limiting middleware in the `Microsoft.AspNetCore.RateLimiting` namespace.

To add rate limiting to your application, you first need to register the rate limiting services:

```csharp
builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // We'll talk about adding specific rate limiting policies later.
});
```

I suggest updating the `RejectionStatusCode` to `429 (Too Many Requests)` because it's more correct.
The default value is `503 (Service Unavailable)`.

And you also have to apply the `RateLimitingMiddleware`:

```csharp
app.UseRateLimiter();
```

That's everything you'll need.

Let's see the rate limiting algorithms we can use.

## Fixed Window Limiter

The `AddFixedWindowLimiter` method configures a fixed window limiter.

The `Window` value determines the time window.

When a time window expires, a new one starts, and the request limit is reset.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Sliding Window Limiter

The sliding window algorithm is similar to the fixed window, but it introduces segments in a window.

Here's how it works:

- Each time window is divided into multiple segments
- The window slides one segment each segment interval
- The segment interval is (window_time)/(segments_per_window)
- When a segment expires, the requests taken in that segment are added to the current segment

The `AddSlidingWindowLimiter` method configures a sliding window limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddSlidingWindowLimiter("sliding", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.SegmentsPerWindow = 2;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Token Bucket Limiter

The token bucket algorithm is similar to the sliding window, but instead of adding back the requests from the expired segment,
a fixed number of tokens are added after each replenishment period.

The total number of tokens can never exceed the token limit.

The `AddTokenBucketLimiter` method configures a token bucket limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 100;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
        options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        options.TokensPerPeriod = 20;
        options.AutoReplenishment = true;
    });
});
```

When `AutoReplenishment` is `true`, an internal timer will execute every `ReplenishmentPeriod` and replenish the tokens.

## Concurrency Limiter

The concurrency limiter is the most straightforward algorithm, and it just limits the number of concurrent requests.

The `AddConcurrencyLimiter` method configures a concurrency limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddConcurrencyLimiter("concurrency", options =>
    {
        options.PermitLimit = 10;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

There's no time component involved in this case. The only parameter is the number of concurrent requests.

## Using Rate Limiting In Your API

Now that we have configured our rate limiting policies, let's see how we can use them in our API.

There are slight differences between controllers and minimal API endpoints, so I'll cover them in separate examples.

**Controllers**

To add rate limiting on a controller we use the `EnableRateLimiting` and `DisableRateLimiting` attributes.

`EnableRateLimiting` can be applied on the controller or on the individual endpoints.

```csharp
[EnableRateLimiting("fixed")]
public class TransactionsController
{
    private readonly ISender _sender;

    public TransactionsController(ISender sender)
    {
        _sender = sender;
    }

    [EnableRateLimiting("sliding")]
    public async Task<IActionResult> GetTransactions()
    {
        return Ok(await _sender.Send(new GetTransactionsQuery()));
    }

    [DisableRateLimiting]
    public async Task<IActionResult> GetTransactionById(int id)
    {
        return Ok(await _sender.Send(new GetTransactionByIdQuery(id)));
    }
}
```

In the previous example:

- All endpoints in the `TransactionsController` will use a **fixed window** policy
- The `GetTransactions` endpoint will use a **sliding window** policy
- The `GetTransactionById` endpoint won't have any rate limiting applied

**Minimal APIs**

In a Minimal API endpoint you can configure the rate limit policy by calling `RequireRateLimiting` and specifying the policy name.

We're using the **token bucket** policy in this example.

```csharp
app.MapGet("/transactions", async (ISender sender) =>
{
    return Results.Ok(await sender.Send(new GetTransactionsQuery()));
})
.RequireRateLimiting("token");
```

## Closing Thoughts

It's great that we can quickly introduce **rate limiting** in ASP.NET Core.

You can choose from one of the existing rate limiter algorithms:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here are some resources if you want to learn more about rate limiting:

- [Rate Limiting pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern)
- [Announcing Rate Limiting for .NET](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/)

I'm excited to try out rate limiting in my projects.

That's all for today.

Have an awesome Saturday!

---



<Sponsorships>

This week's issue is sponsored by [**QueueExplorer.**](https://www.cogin.com/mq/)
QueueExplorer is an explorer-like [**management console for RabbitMQ, Azure Service Bus, MSMQ, and ActiveMQ.**](https://www.cogin.com/mq/)
With QueueExplorer you can see what happens in your queues, edit and fix problematic messages, extract business data from JSON or XML payloads,
or send thousands of messages for stress testing.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

Designing long-lived processes in a distributed environment is an interesting engineering challenge.

And a well known pattern for solving this problem is a [**Saga**](https://microservices.io/patterns/data/saga.html).

A **Saga** is a sequence of local transactions, where each local transaction updates the **Saga** state and publishes a message triggering the next step in the **Saga**.

Sagas come in two forms:

- **Orchestrated**
- **Choreographed**

With an orchestrated Saga, there's a central component responsible for orchestrating the individual steps.

In a choreographed Saga, processes work independently but coordinate with each other using events.

In this week's issue, I'll show you how to create an **orchestrated Saga** using the **Rebus** library with **RabbitMQ** for message transport.

Let's dive in.

## Rebus Configuration

[Rebus](https://github.com/rebus-org/Rebus) is a free .NET “service bus”, and it's practical for implementing asynchronous messaging-based
communication between the components of an application.

Let's install the following libraries:

- `Rebus.ServiceProvider` for managing the **Rebus** instance
- `Rebus.RabbitMq` for **RabbitMQ** message transport
- `Rebus.SqlServer` for **SQL Server** state persistence

```powershell
Install-Package Rebus.ServiceProvider -Version 8.4.0
Install-Package Rebus.RabbitMq -Version 8.0.0
Install-Package Rebus.SqlServer -Version 7.3.1
```

Inside of your **ASP.NET Core** application you will need the following configuration.

```csharp
services.AddRebus(
    rebus => rebus
        .Routing(r =>
            r.TypeBased().MapAssemblyOf<Program>("newsletter-queue"))
        .Transport(t =>
            t.UseRabbitMq(
                configuration.GetConnectionString("RabbitMq"),
                inputQueueName: "newsletter-queue"))
        .Sagas(s =>
            s.StoreInSqlServer(
                configuration.GetConnectionString("SqlServer"),
                dataTableName: "Sagas",
                indexTableName: "SagaIndexes"))
        .Timeouts(t =>
            t.StoreInSqlServer(
                builder.Configuration.GetConnectionString("SqlServer"),
                tableName: "Timeouts"))
);

services.AutoRegisterHandlersFromAssemblyOf<Program>();
```

Unpacking the individual configuration steps:

- `Routing` - Configures messages to be routed by their type
- `Transport` - Configures the message transport mechanism
- `Sagas` - Configures the Saga persistence store
- `Timeouts` - Configures the timeouts persistence store

You also need to specify the queue name for sending and receiving messages.

`AutoRegisterHandlersFromAssemblyOf` will scan the specified assembly and automatically register the respective message handlers.

## Creating The Saga With Rebus

We're going to create a **Saga** for a newsletter onboarding process.

When a user subscribes to the newsletter we want to:

- Send a welcome email immediately
- Send a follow-up email after 7 days

The first step in creating the **Saga** is defining the data model by implementing `ISagaData`.
We'll keep it simple and store the `Email` for **correlation**, and add two flags for the distinct steps in our **Saga**.

```csharp
public class NewsletterOnboardingSagaData : ISagaData
{
    public Guid Id { get; set; }
    public int Revision { get; set; }

    public string Email { get; set; }

    public bool WelcomeEmailSent { get; set; }

    public bool FollowUpEmailSent { get; set; }
}
```

Now we can define the `NewsletterOnboardingSaga` class by inheriting from the `Saga` class and implementing the `CorrelateMessages` method.

It's a best practice to use a unique value for correlation.
In our case this will be the `Email`.

You also configure how the `Saga` starts with `IAmInitiatedBy`, and the individual messages the `Saga` handles with `IHandleMessages`.

```csharp
public class NewsletterOnboardingSaga :
    Saga<NewsletterOnboardingSagaData>,
    IAmInitiatedBy<SubscribeToNewsletter>,
    IHandleMessages<WelcomeEmailSent>,
    IHandleMessages<FollowUpEmailSent>
{
    private readonly IBus _bus;

    public NewsletterOnboardingSaga(IBus bus)
    {
        _bus = bus;
    }

    protected override void CorrelateMessages(
        ICorrelationConfig<NewsletterOnboardingSagaData> config)
    {
        config.Correlate<SubscribeToNewsletter>(m => m.Email, d => d.Email);

        config.Correlate<WelcomeEmailSent>(m => m.Email, d => d.Email);

        config.Correlate<FollowUpEmailSent>(m => m.Email, d => d.Email);
    }

    /* Handlers omitted for brevity */
}
```

## Message Types And Naming Conventions

There are two types of messages you send in a **Saga**:

- Commands
- Events

Commands instruct the receiving component what to do.\
Think: **verb, imperative.**

Events notify the Saga which process was just completed.\
Think: **what happened, past tense.**

## Saga Orchestration With Messages

The `NewsletterOnboardingSaga` starts by handling the `SubscribeToNewsletter` command, and publishes a `SendWelcomeEmail` command.

```csharp
public async Task Handle(SubscribeToNewsletter message)
{
    if (!IsNew)
    {
        return;
    }

    await _bus.Send(new SendWelcomeEmail(message.Email));
}
```

The `SendWelcomeEmail` command is handled by a different component, which publishes a `WelcomeEmailSent` event when it completes.

In the `WelcomeEmailSent` handler we update the `Saga` state and publish a **deferred message** by calling `Defer`.
Rebus will persist the `SendFollowUpEmail` command, and publish it when the **timeout expires**.

```csharp
public async Task Handle(WelcomeEmailSent message)
{
    Data.WelcomeEmailSent = true;

    await _bus.Defer(TimeSpan.FromDays(3), new SendFollowUpEmail(message.Email));
}
```

Finally, the `SendFollowUpEmail` command is handled and we publish the `FollowUpEmailSent` event.

We update the `Saga` state again, and also call `MarkAsComplete` to complete the `Saga`.

```csharp
public Task Handle(FollowUpEmailSent message)
{
    Data.FollowUpEmailSent = true;

    MarkAsComplete();

    return Task.CompletedTask;
}
```

Completing the `Saga` will delete it from the database.

## Handling Commands With Rebus

Here's how the `SendWelcomeEmail` command handler looks like.

```csharp
public class SendWelcomeEmailHandler : IHandleMessages<SendWelcomeEmail>
{
    private readonly IEmailService _emailService;
    private readonly IBus _bus;

    public SendWelcomeEmailHandler(IEmailService emailService, IBus bus)
    {
        _emailService = emailService;
        _bus = bus;
    }

    public async Task Handle(SendWelcomeEmail message)
    {
        await _emailService.SendWelcomeEmailAsync(message.Email);

        await _bus.Reply(new WelcomeEmailSent(message.Email));
    }
}
```

The important thing to highlight here is that we're using the `Reply` method to send a message back.
This will reply back to the endpoint specified as the return address on the current message.

## In Summary

**Sagas** are practical for implementing a long-lived process in a distributed system.
Each business process is represented by a local transaction, and publishes a message to trigger the next step in the **Saga**.

Although **Sagas** are very powerful, they are also _complicated to develop, maintain and debug._

We didn't cover a few important topics in this newsletter:

- Error handling
- Message retries
- Compensating transactions

I think you'll have some fun researching these on your own.

Take a look at the [**source code for the example used in this newsletter**](https://github.com/m-jovanovic/newsletter-orchestrated-saga)
if you want to learn more about implementing Sagas.

If you have **Docker** installed, you should be able to run it without a problem and try it out.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

This week's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus Pro is the perfect one-up for serious Rebus users - use Fleet Manager to get Slack alerts when something fails,
and [**retry dead-lettered messages with a click of the mouse.**](https://pro.rebus.fm)

And by [**Uno Platform**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted).
Uno Platfrom is a free and open-source development platform that allows you to build single codebase applications for Windows, iOS, Android, WebAssembly,
macOS, and Linux with C# and XAML.
[**Easily create applications that look and feel native**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted)
to each platform without having to rewrite your codebase from scratch.

</Sponsorships>

**MediatR** is a popular library with a simple **mediator pattern** implementation in .NET.

Here's a definiton taken from MediatR's GitHub: **"In-process messaging with no dependencies."**

With the rise in popularity of the **CQRS pattern**, MediatR became the go-to library to implement commands and queries.

However, MediatR also has support for the **publish-subscribe** pattern using notifications.
You can publish an `INotification` instance and have multiple subscribers handle the published message.

Until recently, the handlers subscribing to an `INotification` message could only execute serially, one by one.

In this week's newsletter, I'll show you how to configure MediatR to **execute the handlers in parallel**.

Let's dive in.

## How Publish-Subscribe Works With MediatR

Before I talk about notification publishing strategies, let's see how **publish-subscribe** works with **MediatR**.

You need a class implementing the `INotification` interface:

```csharp
public record OrderCreated(Guid OrderId) : INotification;
```

Then you need a respective `INotificationHandler` implementation:

```csharp
public class OrderCreatedHandler : INotificationHandler<OrderCreated>
{
    private readonly INotificationService _notificationService;

    public OrderCreatedHandler(INotificationService notificationService)
    {
        _notificationService = notificationService;
    }

    public async Task Handle(
        OrderCreated notification,
        CancellationToken cancellationToken)
    {
        await _notificationService.SendOrderCreatedEmail(
            notification.OrderId,
            cancellationToken);
    }
}
```

And then you simply publish a message using either `IMediator` or `IPublisher`.
I prefer using the `IPublisher` because it's more expressive:

```csharp
await publisher.Publish(new OrderCreated(order.Id), cancellationToken);
```

MediatR will invoke all the respective handlers.

## Introducing Notification Publisher Strategies

Before MediatR v12, the publishing strategy would invoke each handler individually.

However, there's a new interface `INotificationPublisher` controlling how the handlers are called.

The default implementation of this interface is `ForeachAwaitPublisher`:

```csharp
public class ForeachAwaitPublisher : INotificationPublisher
{
    public async Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        foreach (var handler in handlerExecutors)
        {
            await handler
                .HandlerCallback(notification, cancellationToken)
                .ConfigureAwait(false);
        }
    }
}
```

But now you can also use the `TaskWhenAllPublisher`:

```csharp
public class TaskWhenAllPublisher : INotificationPublisher
{
    public Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        var tasks = handlerExecutors
            .Select(handler => handler.HandlerCallback(
                notification,
                cancellationToken))
            .ToArray();

        return Task.WhenAll(tasks);
    }
}
```

Here's a comparison between these two strategies.

`ForeachAwaitPublisher`:

- Invokes each handler one by one
- Fails when an exception occurs in one of the handlers

`TaskWhenAllPublisher`:

- Invokes all the handlers at the same time
- Executes all the handlers regardless of one of them throwing an exception

If you store the task returned by `TaskWhenAllPublisher` you can access the `Task.Exception` property, which will contain an `AggregateException` instance.
You can then implement more robust exception handling.

## Configuring MediatR Notification Publishing Strategy

How do we configure which `INotificationPublisher` strategy MediatR will use?

There's a new way to apply configuration options when calling the `AddMediatR` method.

You supply an `Action<MediatRServiceConfiguration>` delegate and configure the `MediatRServiceConfiguration` instance.

If you want to use the `TaskWhenAllPublisher` strategy, you can either:

- Provide a value for the `NotificationPublisher` property
- Specify the strategy type on the `NotificationPublisherType` property

```csharp
services.AddMediatR(config => {
    config.RegisterServicesFromAssemblyContaining<Program>();

    // Setting the publisher directly will make the instance a Singleton.
    config.NotificationPublisher = new TaskWhenAllPublisher();

    // Seting the publisher type will:
    // 1. Override the value set on NotificationPublisher
    // 2. Use the service lifetime from the ServiceLifetime property below
    config.NotificationPublisherType = typeof(TaskWhenAllPublisher);

    config.ServiceLifetime = ServiceLifetime.Transient;
});
```

You can also implement a custom `INotificationPublisher` instance and use your own implementation instead.

## How Is This Useful?

Being able to **run notification handlers in parallel** provides a significant **performance improvement** over the default behavior.

However, note that all handlers will use the same service scope.

If you have service instances that don't support concurrent access you may run into problems.

Unfortunately, one such service instance is the **EF Core** `DbContext`.

In any case, I think this is a great addition to the already amazing **MediatR** library.

That's all for today.

See you next week.

---



<Sponsorships>

This week's issue is sponsored by [**Localizely.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)
Localizely is the translation management platform that helps you translate texts in your app for targeting multilingual markets.
With Localizely you can collaborate with your team on translations, and reduce translation errors with better context and app screenshots.
[**Learn more here.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

And by [**QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)
QA Wolf gets you to 80% automated end-to-end test coverage in 4 months - and keeps you there.
They will build, run and maintain your test suite so that you don't have to.
[**Chat with QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)

</Sponsorships>

**Data-driven testing** is a testing method where test data is provided through some external source.
Hence it's also known as **parameterized testing**.

A popular testing library in .NET that supports parameterized testing is **xUnit**.
It uses attributes to define test methods.
The `Fact` attribute defines a simple test, and the `Theory` attribute defines a parameterized test.

In this week's newsletter, I'm going to show you four ways to write **parameterized tests** with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

And I'll discuss which approach I think is the best.

Let's dive in.

## Writing Parameterized Tests With InlineData

The simplest way to write parameterized tests with xUnit is using the `InlineData` attribute.
You provide test data by passing in values to the `InlineData` constructor.

Here's how that would look like:

```csharp
[Theory]
[InlineData("test@test.com", "test.com")]
[InlineData("milan@milanjovanovic.tech", "milanjovanovic.tech")]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}
```

In this example, we provide two `string` values to the `InlineData` attribute, which represent the `email` and `expectedDomain` parameters in the test.
We can specify the `InlineData` attribute as many times as we want, to introduce more test cases.

The downside of this approach is that it becomes very verbose when we have many test cases.
And we are limited to only using constant data for the parameters.

## Writing Parameterized Tests With MemberData

With the `MemberData` attribute we have the ability to programmatically provide the test data.
You can load the test data from a static property or member of a type.

Here's an example of using the `MemberData` attribute to load test data from a property:

```csharp
[Theory]
[MemberData(nameof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public static IEnumerable<object[]> EmailTestData => new List<object>
{
    new object[] { "test@test.com", "test.com" },
    new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" }
};
```

You specify the name of the member in the `MemberData` attribute, and it's a best practice to use the `nameof` operator
so that you can rename the property (or method) in the future without breaking your test.

The one constraint is that the property (or method) has to return `IEnumerable<object[]>`, so there is no strong typing.

## Writing Parameterized Tests With ClassData

The `ClassData` attribute allows you to extract test data into its own class.
This is helpful for organizing your test data separately from your tests, and it allows for easier reuse.
You load the test from a class the inherits from `IEnumerable<object[]>` and implements the `GetEnumerator` method.

Here's an example of using the `ClassData` attribute to load test data from a class:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : IEnumerable<object[]>
{

    public IEnumerable<object[]> GetEnumerator()
    {
        yield return new object[] { "test@test.com", "test.com" };
        yield return new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" };
    }

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
};
```

Unfortunately, this approach is complicated because you have to implement the `IEnumerable` interface.

It almost defeats the purpose of separating test data from the actual tests.

And we still suffer from lack of type-safety.

Is there a better solution?

## Writing Parameterized Tests With TheoryData

Let me introduce you to `TheoryData`, which is my preferred way of providing test data for parameterized tests.
Using the `TheoryData` class you can implement a class to provide test data while having the benefit of type-safety.

Here's an example of using `TheoryData` in combination with `ClassData`:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : TheoryData<string, string>
{
    public EmailTestData()
    {
        Add("test@test.com", "test.com");
        Add("milan@milanjovanovic.tech", "milanjovanovic.tech");
    }
};
```

How does `TheoryData` work?

It's a generic class that allows us to specify the types for our parameterized test.

You just call the `Add` method in the `EmailTestData` constructor to provide test data for a single test case.
And introducing more test cases comes down to calling the `Add` method multiple times.

You can also use `TheoryData` in combination with `MemberData`, and return `TheoryData` from a property or method.

## Which Approach Should You Use?

I showed you four approaches to write parameterized tests with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

So which one should you use?

Here's my personal preference that you can follow if you want:

- `InlineData` for simple test cases
- `TheoryData` using `ClassData` for complex test cases

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2),
the translation management platform that helps you
[**translate texts in your app for targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

</Sponsorships>

**Entity Framework Core (EF Core)** is a popular ORM in .NET that allows you to work with SQL databases.
EF Core uses a `DbContext`, which represents a session with the database and is responsible for tracking changes,
performing database operations, and managing database connections.

It's common to have only one `DbContext` for the entire application.

But what if you need to have **multiple DbContexts**?

In this week's newsletter we're going to explore:

- When you may want to use multiple DbContexts
- How to create multiple DbContexts
- What are the benefits of using multiple DbContexts

Let's dive in!

## Why Use Multiple DbContexts?

There are a few cases where using **multiple DbContexts can be useful.**

**Multiple Databases**\
Does your application need to work with multiple SQL databases?
Then you're forced to use multiple DbContexts, each one dedicated to a specific SQL database.

**Separating Concerns**\
If the application you're building has a complex domain model, you may see an improvement by separating concerns
between a few DbContexts, where each one is responsible for a specific area of the domain model.

**Modular Monolith**\
When you're building a **Modular Monolith**, using multiple DbContexts can be practical because you can configure
a different **database schema** per `DbContext`, giving you logical separation at the database level.

**Read Replicas**\
You can configure a separate `DbContext` instance to access the read replica of your database, and use that
`DbContext` for read-only queries.
You can also configure `QueryTrackingBehavior.NoTracking` on the `DbContext` level to improve query performance.

## Creating Multiple DbContexts In a Single Application

Here's how you can easily configure multiple DbContexts.
Let's say we have a `CatalogDbContext` and an `OrderDbContext` in our application.
We want to configure them using the following constrainsts:

- Both DbContexts use the **same database**
- Each DbContext has a **separate database schema**

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }
}
```

First we need to configure the `CatalogDbContext` and `OrderDbContext` with the DI container.
You can do this by calling the `AddDbContext` method and specifying which `DbContext` is being configured,
and then using the SQL provider specific method to pass the connection string.
In this case I'm connecting to SQL Server with the `UseSqlServer` method.

```csharp
using Microsoft.EntityFrameworkCore;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));
```

If you just want to use both DbContexts in the same schema, then this is all the configuration you need.
You can now inject the `DbContext` instances in your application and use them.

However, if you want to configure a different schema for each `DbContext` then you also need to override
the `OnModelCreating` method and specify the custom schema with `HasDefaultSchema`.

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("catalog");
    }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("order");
    }
}
```

Limitations with multiple DbContexts:

1. It's not possible to do a join between different `DbContext` instances, because **EF Core** doesn't know if they
   are using the same database
2. Transactions will only work if the DbContexts are using the same database. You have to create a new transaction
   and share it between the DbContexts by [calling the `UseTransaction` method](/blog/working-with-transactions-in-ef-core#using-existing-transactions-with-ef-core)

**Migrations History Table**

If you decide to use different schemas per `DbContext`, you will be unpleasantly surprised to learn
that the default schema doesn't apply to the migrations history table.

You need to configure this by calling the `MigrationsHistoryTable` method and specifying the table name
and schema where the migrations history for that context will live.
I used the `HistoryRepository.DefaultTableName` constant in this example, but you can specify a custom
table name if you want to.

```csharp
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Migrations;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "catalog")));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "order")));
```

## Benefits Of Using Multiple DbContexts

Using multiple DbContexts can offer several benefits to your application:

- Separation of concerns
- Better performance
- More control & security

Each DbContext can be responsible for a specific subset of the application's data, which can help organize the code
and make it more modular.

When you separate data access into multiple DbContexts, the application can reduce the risk of contention and improve
concurrency, and this can improve performance.

And if you're using multiple DbContexts, you can configure more granular access control to improve application security.
You can also optimize performance and resource usage.

## In Summary

Using **multiple EF Core DbContexts** in a single application is straightforward and has many benefits.

For ready-heavy applications you can configure a separate `DbContext` to turn off query tracking by default
and get improved performance.

Also, using multiple DbContexts is practical if you're building a **Modular monolith**.
You can configure the DbContexts to be in **separate database schemas**, giving you logical separation at the database level.

That's all for today.

See you next week.

---



<Sponsorships>

Today's issue is sponsored by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website),
the translation management platform that helps you translate texts in your app for
[**targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website)

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly)
on new or existing SQL Server, enabling teams to ship apps faster.

</Sponsorships>

Although **C#** is an **object-oriented programming** language, it received many new functional features in recent versions.

To mention just a few of these features:

- Pattern matching
- Switch expressions
- Records

You're probably already doing **functional programming** without even knowing it.

Do you use LINQ?
If you do, then you're doing functional programming.
Because LINQ is a functional .NET library.

In today's issue, I will show you how to **refactor** some **imperative code** with **functional programming**.

Let's dive in.

## Benefits Of Functional Programming

Before we take a look at some code, let's see what are the **benefits** of using **functional programming**:

- Emphasis on immutability
- Emphasis on function purity
- Code is easier to reason about
- Less prone to bugs and errors
- Ability to compose functions and create higher-order functions
- Easier to test and debug

From my experience, I think functional programming is more enjoyable once you get used to it.
Starting out, it feels strange because your old object-oriented programming habits will kick in.
But after a while, **functional programming** feels easier to work with than imperative code.

## Starting With Imperative Code

**Imperative programming** is the most basic programming approach.
We describe a step-by-step process to execute a program.
It's easier for beginners to reason with imperative code by following along with the steps in the process.

Here's an example of an `EmailValidator` class written with imperative code:

```csharp
public class EmailValidator
{
    private const int MaxLength = 255;

    public (bool IsValid, string? Error) Validate(string email)
    {
        if (string.IsNullOrEmpty(email))
        {
            return (false, "Email is empty");
        }

        if (email.Length > MaxLength)
        {
            return (false, "Email is too long");
        }

        if (email.Split('@').Length != 2)
        {
            return (false, "Email format is invalid");
        }

        if (Uri.CheckHostName(email.Split('@')[1]) == UriHostNameType.Unknown)
        {
            return (false, "Email domain is invalid");
        }

        return (true, null);
    }
}
```

You can clearly see the distinct steps:

- Check if email is null or empty
- Check if email is not too long
- Check if email format is valid
- Check if email domain is valid

Let's see how we can refactor this using **functional programming**.

## Applying Functional Programming

The basic building block in **functional programming** is - **a function**.
And programs are written by composing function calls.
There are a few other things you need to keep in mind, like keeping your functions pure.
A function is pure if it always returns the same output for the same input.

We can capture each step from the imperative version of `EmailValidator` with a `Func` delegate.
To also capture the respective error message together with the validation check, we can use a tuple.
And since we know all of our validation steps, we can create an array of `Func` delegates to store all of the individual **functions**.

```csharp
public class EmailValidator
{
    const int MaxLength = 255;

    static readonly Func<string, (bool IsValid, string? Error)>[] _validations =
    {
        email => (!string.IsNullOrEmpty(email), "Email is empty"),
        email => (email.Length <= MaxLength, "Email is too long"),
        email => (email.Split('@').Length == 2, "Email format is invalid"),
        email => (
            Uri.CheckHostName(email.Split('@')[1]) != UriHostNameType.Unknown,
            "Email domain is invalid")
    };

    static readonly (bool IsValid, string? Error) _successResult = (true, null);

    public (bool IsValid, string? Error) Validate(string email)
    {
        var validationResult = _validations
            .Select(func => func(email))
            .FirstOrDefault(func => !func.IsValid);

        return validationResult is { IsValid: false, Error: { Length: >0 } } ?
            validationResult : _successResult;
    }
}
```

Notice that this allows us to do all sorts of interesting things with the `_validations` array.
How hard would it be to modify this function to return _all of the errors_ instead of just the first one?

If you're thinking we can use LINQ's `Select` method somehow, you're thinking in the right direction.

## Further Reading

We only scratched the surface of what functional programming is, and what you can do with it.
If you want to learn more, here are some learning materials:

- [Functional Programming in C#, by Enrico Buonanno](https://www.manning.com/books/functional-programming-in-c-sharp)
- [Functional Programming With C# Using Railway-Oriented Programming](https://youtu.be/dDasAmowFts)
- [How Function Composition Can Make Your Code Better](https://youtu.be/zuy2j8vxgYc)
- [Make Your ASP.NET Core API Controllers Incredibly Simple With Functional Programming](https://youtu.be/AVA2mKG4WOc)

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

[**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website) is a translation management platform that
helps you translate texts in your app for targeting multilingual markets.

</Sponsorships>

Working with **Microservices**, or any distributed system for that matter, is difficult.
In a distributed system many things can go wrong, and there are even research papers about this.
If you want to explore this topic further, I suggest that you read about the [fallacies of distributed computing](https://www.se.rit.edu/~se442/doc/fallacies.pdf).

Reducing the surface area for things to go wrong should be one of your goals, as an engineer.
In this week's newsletter, we'll try to achieve exactly that using the **Outbox pattern**.

How can you implement reliable communication between components in a distributed system?

The **Outbox pattern** is an elegant solution to this problem, allowing you to achieve transactional
guarantees in a single service and at-least-once message delivery to external systems.

Let's see how the **Outbox pattern** solves this and how can we implement it.

## What Problem Does The Outbox Pattern Solve?

To understand what problem the **Outbox pattern** solves, first we need a problem, of course.

Here's an example of a user registration flow.
There are a few things going on here:

- Saving the `User` to the database
- Sending a welcome email to the `User`
- Publishing a `UserRegisteredEvent` to a message bus

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    await _unitOfWork.SaveChangesAsync(token);

    await _emailService.SendWelcomeEmailAsync(user, token);

    await _eventBus.PublishAsync(new UserRegisteredEvent(user.Id), token);
}
```

In the happy path, all of the operations complete without any issues and all is well.

But what happens if any one of these operations fail?

- The database is unavailable, and saving the `User` fails
- The email service is down and sending an email crashes
- Publishing an event to the service bus doesn't succeed

Also, imagine a situation where you manage to save a `User` to the database,
send him a welcome email, but fail to publish the `UserRegisteredEvent` to notify other services.
How are you going to recover from this scenario?

The **Outbox pattern** allows you to **atommically** update the database and send messages to the message bus.

## Implementing The Outbox Pattern

The first step is to introduce a table in your database to represent the **Outbox**.
We can call this table `OutboxMessages`, and it's intended to store all messages that need to be delivered.
Now instead of directly making requests to external services, we simply store a message as a new row in the **Outbox** table.
The messages are usually stored as JSON in the database.

The second step is to introduce a **background process** that will periodically poll the `OutboxMessages` table.
If the worker process finds a row with an unprocessed message, it's going to publish that message and mark it as sent.
If publishing the message fails for some reason, the work process can **retry** in the next execution.

Notice that with retries, you now have **at-least-once message delivery** implemented.
The message will be published exactly once for the happy path, and more than one time in case or retries.

We can rewrite the `RegisterUserAsync` method from the previous example, now using an **Outbox**:

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    _outbox.Insert(new UserRegisteredEvent(user.Id));

    await _unitOfWork.SaveChangesAsync(token);
}
```

The **Outbox** is part of the same transaction as our unit of work, so we can atomically save the `User` to the database
and also persist the `OutboxMessage`.
If saving to the database fails, the entire transaction is rolled back and no messages are sent to the message bus.

And since we now moved the publishing of the `UserRegisteredEvent` to the worker process, we need to add a handler
so that we can send the welcome email to the user.
Here's an example of that in the `SendWelcomeEmailHandler` class:

```csharp
public class SendWelcomeEmailHandler : IHandle<UserRegisteredEvent>
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailService _emailService;

    public SendWelcomeEmailHandler(
        IUserRepository userRepository,
        IEmailService emailService)
    {
        _userRepository = userRepository;
        _emailService = emailService;
    }

    public async Task Handle(UserRegisteredEvent message)
    {
        var user = await _userRepository.GetByIdAsync(message.UserId);

        await _emailService.SendWelcomeEmailAsync(user);
    }
}
```

## Architecture Diagram With Outbox

Here's a high level overview of the system architecture with the **Outbox** introduced to the system.
You can see the `Outbox` table in the database.
What changes now is that you store messages to the `Outbox` table in the same transaction along with your entities.

<Image src="blogs/mnw_026/outbox.png" width={1801} height={989} />

## Further Reading

After reading this newsletter you should have a pretty good understanding of what the **Outbox pattern** is
and what problems it solves.
If you need to implement reliable messaging in a distributed system, it's a great solution for your problem.

What's missing is more details about how to implement the **Outbox pattern**, so here are a few videos you can watch:

- [How to use the Domain Events pattern](https://youtu.be/BimfDeDV4yU)
- [How to implement the Outbox pattern](https://youtu.be/XALvnX7MPeo)
- [How to add retries to the Outbox with Polly](https://youtu.be/xajVttkZntU)

Thanks for reading, and have an amazing Saturday.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

**Structured logging** is a practice where you apply the same message format to all of your
application logs.
The end result is that all your logs will have a similar structure, allowing them to be
easily searched and analyzed.

[Serilog](https://serilog.net/) is a popular logging library in .NET, packed with many features.
It provides logging to files, logging to the console, and elsewhere.

However, **Serilog** is unique because it comes with support for **structured logging** out of the box.

Let's see how we can install **Serilog** and configure it an **ASP.NET Core** application.

## Installing Serilog

To install **Serilog** in **ASP.NET Core** you can add the following NuGet package:

```powershell
Install-Package Serilog.AspNetCore
```

This NuGet packages comes with a simple API to integrate **Serilog** into your application.
You can call the `UseSerilog` method on the `HostBuilder` instance to provide a lambda
method to configure **Serilog**.

I think the most flexible way to configure Serilog is through application settings,
which is achieved by calling `ReadFrom.Configuration()`.

You can also call the `UseSerilogRequestLogging()` method to introduce automatic HTTP request logging
in your API.

```csharp
var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((context, configuration) =>
    configuration.ReadFrom.Configuration(context.Configuration));

var app = builder.Build();

app.UseSerilogRequestLogging();

app.Run();
```

The next question is how do you provide the actual configuration values to **Serilog**?

## Configuring Serilog With `appsettings.json`

You need to add a `Serilog` section in your `appsettings.json` file.

Here you can configure, among other things:

- Which **sinks** to use with **Serilog**
- Override default and minimum log levels
- Configure file logging arguments

In this example, we're adding the `Console` and `File` sinks to **Serilog**.
And we're adding some additional configuration for the `File` sink in the `Serilog.WriteTo` configuration section.
We can configure the output path for the log files, the naming format, which formatter to use for the logs and so on.

```json
"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning"
    }
  },
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "/logs/log-.txt",
        "rollingInterval": "Day",
        "rollOnFileSizeLimit": true,
        "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
      }
    }
  ],
  "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ]
}
```

You can get a more detailed overview of what's supported with the `Serilog.Configuration` library in the
[documentation](https://github.com/serilog/serilog-settings-configuration).

## Using Serilog In ASP.NET Core

We managed to successfully install and configure **Serilog**.
But how do we actually use it?

**Serilog** integrates with the `ILogger` interaface coming from the `Microsoft.Extensions.Logging` namespace.
If you're already using `ILogger` for logging, everything will continue working correctly.

Here's a simple example of logging inside of a Minimal API endpoint:

```csharp
app.MapGet("/serilog-is-cool", (ILogger logger) =>
{
    logger.LogInformation("This is a log inside of the Minimal API endpoint.");

    return Results.Ok(new { Message = "success" });
});
```

You just inject an `ILogger` or `ILogger<T>` instance and Serilog will provide its own implementation at runtime.

## Structured Logging Syntax

The idea behind **structured logging** is that you can introduce additional contextual information inside
of your logs.
**Serilog** does this using a message template syntax, where you can specify named parameters and then
pass in their values separately.

Here's an example of what this message template would look like.
You specify parameters inside of curly bracers and provide a name, for example `{NamedParameter}`.
The value provided for the parameter will be serialized as a property inside of the corresponding
structured log.

```csharp
var book = new { Author = "Domain-Driven Design", Title = "Eric Evans" };
var orderNumber = 1;

log.LogInformation(
    "Processing book {@Book}, order number = {@OrderNumber}",
    book,
    orderNumber);
```

There are a few things to unpack here:

- `{@Book}` parameter which accepts an object
- `{OrderNumber}` parameter which accepts a scalar value

The `@` operator in front of `Book` tells Serilog to serialize the object passed in, instead of converting
it using `ToString()`.

## Benefits Of Structured Logging

Lastly, I want to highlight what are some of the benefits of **structured logging** and why you should be using it.

As I said at the beginning, the main idea with **structured logging** is that all log message follow the same
structure.
This structure can be a JSON document for example, or a row in a relational table.
Since structured logs are in a machine-readable format, it's much easier to search through them for
specific information.

When an error occurs, structured logs can provide more context and details about the error, making it easier to
identify the root cause and fix the problem.

It's very easy to start doing structured logging with **Serilog**, and I hope you'll give it a try.

See you next week, and have an excellent Saturday.

---



If you're working in a **distributed system**, you need to be able to communicate between
multiple services.
There are a few ways that you can implement this.
Depending on your chosen approach, you can either introduce tight coupling between your
services or stay **loosely coupled**.

**Loose coupling** is an important quality in distributed systems.
It allows you to evolve your services independently.
So how do you implement loosely coupled **communication between services**?

You need a **messaging system**.

And [Azure Service Bus](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview)
is an excellent choice.

In this week's newsletter, I'll show you how to create an **Azure Service Bus** instance,
and how to implement messaging over a **queue**.

Let's dive in.

## Creating An Azure Service Bus Instance

You can create a new **Azure Service Bus** instance from the **Azure** portal.

I won't go into detail on that, since I find the **Azure** UI pretty intuitive.

<Image src="blogs/mnw_024/service_bus.png" width={898} height={695} />

After creating your **Azure Service Bus** instance, you'll need to do two more things:

- Create a new **queue**
- Find the **connection string**

After that you can proceed with implementing the pub-sub pattern over a queue.
And you'll use the connection string from the Azure portal for connecting to the Azure Service Bus instance.

## Publishing Messages To The Azure Service Bus Queue

The first thing we need to do is to create the publishing side of our system, and then
we'll see how we can process messages.
We're going to use the `Azure.Messaging.ServiceBus` library to connect to the queue
running in Azure Service Bus.

You can install the NuGet package by running the following command:

```powershell
Install-Package Azure.Messaging.ServiceBus
```

And now, let's write the code for publishing a message to an Azure Service Bus queue.

To work with the Azure Service Bus instance, you will use the [`ServiceBusClient`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusclient?view=azure-dotnet)
class.
It requires a connection string to be able to connect to the Azure Service Bus instance.

The `ServiceBusClient` is safe to cache and reuse in the application, so it can be registered
as a service with the singleton lifetime.

With the `ServiceBusClient` you can create a [`ServiceBusSender`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebussender?view=azure-dotnet)
instance, which is responsible
for sending the actual messages.
You also need to specify which queue it will be sending messages to.
This can also be the name of a topic, if you are publishing to a topic instead.

```csharp
using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusSender sender = client.CreateSender(QueueName);

// This will be the payload for the message ✉️
var productCreated = new ProductCreatedEvent(
    eventId: Guid.NewGuid(),
    product.Id,
    product.Name);

string json = JsonSerializer.Serialize(productCreated);

var message = new ServiceBusMessage(json);

await sender.SendMessageAsync(message);
```

As you can see, publishing a message to the queue is relatively simple.

We're creating a new instance of `ProductCreatedEvent`, which represents our message payload.
The payload itself is serialized into a `JSON` string, wrapped inside a `ServiceBusMessage`.

For publishing messages to Azure Service Bus, you simply call the `SendMessageAsync` method.

## Receiving Messages From The Azure Service Bus Queue

Publishing messages to a queue is only half of the job.
You also need to be able to receive messages from the queue.
However, you'll see that this is very similar to the publishing side.

On the receiving side, you'll also need to create a `ServiceBusClient`.
And then use it to create an instance of [`ServiceBusProcessor`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor?view=azure-dotnet),
which is used for consuming messages.
You need to tell the `ServiceBusProcessor` which queue it will subscribe to.

The `ServiceBusProcessor` exposes two events, which represent callbacks for when a message is received.
These events are [`ProcessMessageAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processmessageasync?view=azure-dotnet)
and [`ProcessErrorAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processerrorasync?view=azure-dotnet).

You need to provide a handler for these two events, to properly consume messages from the queue.
In the example below, we're using the `HandleMessageAsync` and `HandleErrorAsync` local functions.

```csharp
using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusProcessor processor  = client.CreateProcessor(QueueName);

processor.ProcessMessageAsync += HandleMessageAsync;

processor.ProcessErrorAsync += HandleErrorAsync;

await processor.StartProcessingAsync();

async Task HandleMessageAsync(ProcessMessageEventArgs args)
{
    string json = args.Message.Body.ToString();

    var productCreated = JsonSerializer.Deserialize<ProductCreatedEvent>(json);

    Console.WriteLine(productCreated);

    await args.CompleteMessageAsync(args.Message);
}

Task HandleErrorAsync(ProcessErrorEventArgs args)
{
    var exception = args.Exception;

    Console.WriteLine(exception.ToString());

    return Task.CompletedTask;
}
```

The `ServiceBusProcessor` begins listening to messages coming from the queue after calling the `StartProcessingAsync` method.

Notice that the success and error event handlers need to accept the
[`ProcessMessageEventArgs`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processmessageeventargs?view=azure-dotnet)
and [`ProcessErrorEventArgs`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processerroreventargs?view=azure-dotnet),
respectively.

From the `ProcessMessageEventArgs` you can access the `Message.Body` which contains the mesage payload.

## Further Reading

**Azure Service Bus** is a very feature rich service cloud.
I showed you how to work with **queues**, which are great if you only have one publisher and one subscriber.
However, if you need the ability to have multiple subscribers to a single message, you can't achieve
this with queues.

You will have to use [topics](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-queues-topics-subscriptions#topics-and-subscriptions),
and I invite you to research this _topic_ further (pun intended).

**Azure Functions** have excellent support for integrating with **Azure Service Bus**.
You can define a `QueueTrigger` that will run your **Azure Function** when you receive a message to an Azure Service Bus **queue**.

I also released a video showing how to [publish and consume messages using RabbitMQ](https://youtu.be/CTKWFMZVIWA),
and I think you might enjoy it after reading this newsletter.

Have an excellent weekend, and stay awesome!

---



<Sponsorships>

[**Ultimate ASP.NET Core Web API - Second Edition**](https://code-maze.lemonsqueezy.com/checkout/buy/b0e9ff04-cb9f-4bc2-a69b-d434fad56790)
is a step-by-step and easy-to-digest guide to creating scalable, robust, and high-performance Web APIs.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

Every software engineer working with **SQL databases** needs to know about **transactions**.
And since most of the time the **SQL database** will be abstracted by an ORM like **EF Core**,
it's important to understand how you can work with **transactions** using the available
abstractions.

So today, I'll show you how to work with **transactions** in **EF Core**.

Here's what we will cover:

- Default transaction behavior
- Creating transactions
- Using existing transactions

Let's dive in.

## Default Transaction Behavior

What is the **default** EF Core **transaction behavior**?

By default, all changes made in a single call to `SaveChanges` are applied in a
transaction. If any of the changes fail, the entire transaction is rolled back
and no changes are applied to the database. Only if all changes are successfully
persisted to the database, the call to `SaveChanges` can complete.

This is a wonderful feature of **SQL databases** and it saves us many headaches.
We don't have to think about the databases remaining in an inconsistent state,
because database **transactions** can do the work for us.

Let's take a look at an example.

```csharp
using var context = new ShoppingContext();

context.LineItems.Add(new LineItem
{
    ProductId = productId,
    Quantity = quantity
});

var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

stock.Quantity -= quantity;

context.SaveChanges();
```

Because we are adding a `LineItem`, and in the same scope reducing the `Stock`
quantity, the call to `SaveChanges` will apply both changes inside of a transaction.
We can guarantee that the database will remain in a **consistent state**.

## Creating Transactions With EF Core

What if you want to have more control over **transactions** when working with **EF Core**?

You can manually create a transaction by accessing the `Database` facade available
on a `DbContext` instance and calling `BeginTransaction`.

Here's an example where we have multiple calls to `SaveChanges`. In the default
scenario, both calls would run in their own transaction. This leaves the possibility
of the second call to `SaveChanges` failing, and leaving the database in an
inconsistent state.

```csharp
using var context = new ShoppingContext();
using var transaction  = context.Database.BeginTransaction();

try
{
    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    // When we commit the changes, they will be applied to the databases.
    // The transaction will auto-rollback when it is disposed,
    // if any command fails.
    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
```

We call `BeginTransaction` to manually start a new **database transaction**.
This will create a new transaction and return it, so that we can `Commit` the
transaction when we want to complete the operation. You also want to add a
`try-catch` block around your code, so that you can `Rollback` the transaction
if there are any exceptions.

## Using Existing Transactions With EF Core

Creating a transaction using the EF Core `DbContext` isn't the only option.
You can create a `SqlTransaction` instance and pass it to **EF Core**, so that
the changes applied with EF Core can be committed inside the same **transaction**.

Here's what I mean:

```csharp
using var sqlConnection = new SqlTransaction(connectionString);
sqlConnection.Open();

using var transaction = sqlConnection.BeginTransaction();

try
{
    using var context = new ShoppingContext();

    // Tell EF Core to use an existing transaction.
    context.UseTransaction(transaction);

    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
```

## In Summary

**EF Core** has excellent support for **transactions** and it's very easy to work with.

You have three options available:

- Rely on the default transaction behavior
- Create a new transaction
- Use an existing transaction

Most of the time, you want to rely on the default behavior and not have to
think about it.

As soon as you need to perform multiple `SaveChanges` calls, you should manually
create a transaction, and manage the transaction yourself.

See you next week, and have an excellent Saturday.

---



In this week's newsletter I want to show you how to implement **API Key authentication**
in **ASP.NET Core**. This authentication approach uses an **API Key** to authenticate the
client of an API. You can pass the **API Key** to the API in a few ways, such as through
the query string or a request header.

I will show you how to implement **API Key authentication** where the **API key** is passed
in a request header. But the implementation would be similar if we were to use any
other approach.

When would you want to use **API Key authentication**? This kind of authentication
mechanism is common in **Server-to-Server (S2S)** communication. When your API serves
request for other server-side applications to consume and integrate with. It's
less common in client-server communication scenarios.

Let's see how we can implement **API Key authentication** in ASP.NET Core!

## Implementing API Key Authentication

We will start off by creating an attribute that we can place on endpoints
where we want to apply **API Key authentication**. It won't be any kind of
attribute, because we will use a `ServiceFilterAttribute`.

What a `ServiceFilterAttribute` allows us to do is specify a type for the
filter that will be created for that attribute.
This means we can implement our authentication logic in an `IAuthorizationFilter`.
With a `ServiceFilterAttribute` we also have support for dependency injection
in our `IAuthorizationFilter` implementation.

Let's first define the `ApiKeyAttribute` class:

```csharp
public class ApiKeyAttribute : ServiceFilterAttribute
{
    public ApiKeyAttribute()
        : base(typeof(ApiKeyAuthorizationFilter))
    {
    }
}
```

In the `ApiKeyAttribute` we specify `ApiKeyAuthorizationFilter` class as the
filter that will be resolved from the DI container. Here's what it looks like:

```csharp
public class ApiKeyAuthorizationFilter : IAuthorizationFilter
{
    private const string ApiKeyHeaderName = "X-API-Key";

    private readonly IApiKeyValidator _apiKeyValidator;

    public ApiKeyAuthorizationFilter(IApiKeyValidator apiKeyValidator)
    {
        _apiKeyValidator = apiKeyValidator;
    }

    public void OnAuthorization(AuthorizationFilterContext context)
    {
        string apiKey = context.HttpContext.Request.Headers[ApiKeyHeaderName];

        if (!_apiKeyValidator.IsValid(apiKey))
        {
            context.Result = new UnauthorizedResult();
        }
    }
}
```

The implementation comes down to validating the **API Key** obtained from
the header of the current request. If we determine that the **API Key**
is not valid, we set the value of `AuthorizationFilterContext.Result`
to a new instance of an `UnauthorizedResult`.

And lastly, all that's left for us to do is implement our custom
validation logic for the **API Key** inside of `ApiKeyValidator`:

```csharp
public class ApiKeyValidator : IApiKeyValidator
{
    public bool IsValid(string apiKey)
    {
        // Implement logic for validating the API key.
    }
}

public interface IApiKeyValidator
{
    bool IsValid(string apiKey);
}
```

The actual implementation for validating the **API Key** will vary based
on your use case, and where you are storing the API keys.
For example, if you store the API keys in the database you would check
if the provided **API Key** exists in the database.
If it exists, then validation passes.
If it doesn't exist, then validation fails and we return an
`UnauthorizedResult`.

## Registering Services With Dependency Injection

We have to make sure to register our `ApiKeyAuthorizationFilter` and
`ApiKeyValidator` services with the dependency injection container.

```csharp
builder.Services.AddSingleton<ApiKeyAuthorizationFilter>();

builder.Services.AddSingleton<IApiKeyValidator, ApiKeyValidator>();
```

This will register them as singleton services in our application.
You can use a different service scope such as `Transient` or `Scoped`
if you need to.

## Applying API Key Authentication To Endpoints

Finally, with our **API Key authentication** in place, we can apply the
`ApiKeyAttribute` attribute to our endpoints:

```csharp
public class NewslettersController : ControllerBase
{
    [ApiKey]
    [HttpGet]
    public IActionResult Get()
    {
        // ...
    }
}
```

In this case I'm applying the `ApiKeyAttribute` to an endpoint, but
you can also apply it on the `NewslettersController` and it will add
authentication to all the endpoints for that controller.

## Next Steps

Now that you know how to implement **API Key authentication**, I think you
should also learn how to implement **JWT authentication**. And while you're
at it, why not throw **authorization** into the mix.

I made a few videos about **JWT authentication** and **permission authorization**
that you should take a look at next:

- [Token Authentication In ASP.NET Core 7 With JWT](https://youtu.be/4cFhYUK8wnc)
- [Introduction To Permission Authorization In ASP.NET Core 7](https://youtu.be/PlbAuNvR16s)
- [Managing Permissions With EF Core Migrations](https://youtu.be/v4vXDRJ9_sg)
- [Implementing A Custom Authorization Handler In ASP.NET Core](https://youtu.be/SZtZuvcMBA0)
- [Using Custom JWT Claims For Authorization In ASP.NET Core](https://youtu.be/SUyFPp6BPV0)

---



<Sponsorships>

[**Dotnetjobs**](https://dotnet-jobs.com/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnet-jobs.com/).

</Sponsorships>

In this week's newsletter I want to talk about the `yield` keyword in **C#**.
I think it's a powerful **C#** feature and I wanted to highlight the benefits.

The `yield` keyword tells the compiler that the method in which it appears
is an **iterator block**. An iterator block, or method, returns an `IEnumerable`
as the result. And the `yield` keyword is used to return the values for the
`IEnumerable`.

An interesting thing aboug `IEnumerable` is that it is lazily evaluted.
Calling a method with an iterator block doesn't run any code. It's only
when the `IEnumerable` is iterated over, or enumerated, that we get
the actual values. I'll talk about this more later.

Let's see how we can start using the `yield` keyword!

## How To Use The Yield Keyword

The `yield` keyword on it's own doesn't do anything, you have to combine
it with the `return` or `break` statement:

- `yield return` - provides the next value of the iterator
- `yield-break` - signals the end of iteration

In every project I worked on, there's a piece of code similar to the
following. You create a list to hold the results, add elements to the
list, and return the list in the end.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    var result = new List<SoftwareEngineer>();

    for(var i = 0; i < 10; i++)
    {
        result.Add(new SoftwareEngineer
        {
            Id = i
        });
    }

    return result;
}
```

You can simplify the method using the `yield return` statement, and
completely remove the intermediate list required to hold the results.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    for(var i = 0; i < 10; i++)
    {
        yield return new SoftwareEngineer
        {
            Id = i
        };
    }
}
```

However, it's important to note these two implementation are fundamentally
different from each other. In the first example, the entire list is
populated and materialized. In the second example, the `IEnumerable`
returned will not be materialized and you have to either iterate over
it inside a `foreach` loop or call `ToList()`.

## Stopping Iteration With Yield Break

You can use the `yield break` statement to stop iteration and exit
the iterator block. Typically you would do this when a certain
condition is met, or you only want to return a specific set of values
from the iterator block.

Here's an example where this would be useful:

```csharp
Console.WriteLine(string.Join(", ", TakeWhilePositive(new[] { 1, 2, -3, 4 })));
// Output: 1, 2

public IEnumerable<int> TakeWhilePositive(IEnumerable<int> numbers)
{
    foreach(int num in numbers)
    {
        if (num > 0)
        {
            yield return num;
        }
        else
        {
            yield break;
        }
    }
}
```

## Working With IAsyncEnumerable

In **C# 8** we got the `IAsyncEnumerable` type which allows us to
iterate over a collection asynchronously with the `yield` statement.

For example, this can be useful when you want to call a thid-party
API multiple times to fetch some data. A common situation is when
you get a list of users from the database, and then have to call
an external storage service to get profile picture information.

Without `IAsyncEnumerable` you would have to do something like this:

```csharp
public async Task<IEnumerable<User>> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);
    }

    return users;
}

// And you would call the method like this.
var users = await GetUsersAsync();

foreach(var user in users)
{
    Console.WriteLine(user);
}
```

Now, consider this same example with the use of `IAsyncEnumerable`:

```csharp
public async IAsyncEnumerable<User> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);

        yield return user;
    }
}

// And you would call the method like this.
await foreach(var user in GetUsersAsync())
{
    Console.WriteLine(user);
}
```

The second implementation will iterate over the users returned from
the database when they are yielded by the `IAsyncEnumerable`.

## When Should I Use Yield?

I've found a few interesting practical applications for the `yield` keyword.
One example is when implementing Domain-Driven Design value objects.

Value objects need to support structural equality. They need to implement
a method that returns all of the equality components. Here's an example of
that using the `yield return` statement:

```csharp
public class Address
{
    public string City { get; init; }

    public string Street { get; init; }

    public string Zip { get; init; }

    public string Country { get; init; }

    public IEnumerable<object> GetEqualityComponents()
    {
        yield return City;
        yield return Street;
        yield return Zip;
        yield return Country;
    }
}
```

---



<Sponsorships>

Have some [**C# and .NET experience**](https://packt.link/PJqza), and want to get started building apps?
Get introduced to the most common technologies like gRPC, Blazor, GraphQL, Cosmos DB, SignalR, .NET MAUI, and more in
[**Apps and Services with .NET 7**](https://packt.link/PJqza).

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to introduce you to an interesting feature
in **EF Core** called **Compiled Queries**.

If you have queries that you execute frequently in your application with a
different set of parameters, it can be helpful to explicitly compile the
query and reuse it throughout the lifetime of your application.

**Compiled Queries** are more performant than standard EF queries, because they
can take advantage of some additional optimizations.

Let me show you how to use **Compiled Queries**, and how much performance
improvement to expect!

## How To Create Compiled Queries

Lets define a simple class that will represent out data model that we will
use for writing **EF** queries:

```csharp
public class Newsletter
{
    public long Id { get; init; }

    public string Title { get; init; }

    public int ReadTimeInMinutes { get; init; }
}
```

How would we write a simple query to fetch a `Newsletter` by the `Id`?
I think you can do this in your sleep.

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id);
```

Now, how do we convert this query into a **Complied Query**?

There are a few steps involved:

- Create a **Compiled Query** by calling `EF.CompileQuery`
- Store the **Compiled Query** in a static field, so that it can be reused
- Execute the database query using the **Compiled Query**

You can define the **Compiled Query** in a static field inside of the `AppDbContext`.
And then expose a method that will accept an argument, and pass it to the
**Compiled Query** to invoke it.

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, long, Newsletter?> GetNewsletter =
        EF.CompileQuery(
            (dbContext, id) =>
                dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id));

    public Newsletter? GetNewsletter(long id)
    {
        return GetNewsletter(this, id);
    }
}
```

This is how we would call the method which invokes the **Compiled Query**:

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.GetNewsletter(id);
```

I ran some benchmarks, with the following setup:

- **EF Core 7**
- **SQL Server 2022**
- Table with 10,000 records

The **Compiled Query** was consistently around **10% faster**.

I also tried running a no-tracking query by calling `AsNoTracking()` and
observed similar results.

## Why Are Compiled Queries Faster?

So we can conclude that **Compiled Queries** are faster. But why is that?

Let's examine what happens when we execute an **EF** LINQ query. Before **EF**
can convert the query into valid SQL that can be executed in the database,
it needs to compile the query. The compiled query is cached and **EF** will be
able to reuse that cached query. In some situations the query needs to be
recompiled, introducing additional performance costs.

When we explicitly compile the query by calling `EF.CompileQuery`, we
can utilize some optimization techniques that aren't available at runtime.

Note that **Compiled Queries** only improve the performance of the in-memory
portion of executing an EF query. The round trip time and materializing
results from the database remain unaffected.

## Can We Make Compiled Queries Asynchronous?

I showed you how to write a synchronous **Compiled Query**. But due to
performance considerations we almost always want to execute database
queries asynchronously.

Here's how we can create an asynchronous **Compiled Query**:

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, string, Task<Newsletter?>> GetByTitle =
        EF.CompileAsyncQuery(
            (AppDbContext context, string title) =>
                context.Set<Newsletter>().FirstOrDefault(c => c.Title == title));

    public async Task<Newsletter?> GetNewsletterByTitleAsync(string title)
    {
        return await GetByTitle(this, title);
    }
}
```

It's interesting that we aren't writing an asynchronous query in the expression
passed to `EF.CompileAsyncQuery`. It will be converted to an asynchronous query
during compilation.

## Compiled Queries Aren't a Silver Bullet

You might be tempted to go and convert all of your **EF** queries into
**Compiled Queries**, to squeeze out that last little bit of performance.
I urge you not do it. **Compiled Queries** are a useful tool, but they
aren't the solution to all your problems.

Instead, I think we should use **Compiled Queries** sparingly, only in
situations where we really need to do these kinds of micro-optimizations.

Although **Compiled Queries** seem great, we can't deny they increase
the complexity of our code considerably. If you think the slight
performance improvement gained from using **Compiled Queries** justifies
the increase in complexity, then by all means, you should use them.
Otherwise, I would look for other ways to improve performance.

---



<Sponsorships>

[**Dotnetjobs**](https://dotnet-jobs.com/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnet-jobs.com/).

</Sponsorships>

In this week's newsletter I will show you how to easily add **validation**
to the strongly typed configuration objects injected with `IOptions`.

The **Options pattern** allows us to use classes to provide strongly typed
configuration values in our application at runtime.

But you have no guarantee that the configuration values injected with
`IOptions` will be correctly read from the application settings.

Let's see how we can introduce validation for `IOptions`, and make sure the application settings are correct.

## Strongly Typed Configuration

I first want to define a simple class that will represent our strongly
typed configuration. Let's say we want to integrate with the **GitHub API**,
so we create a `GitHubSettings` class to hold our configuration:

```csharp
public class GitHubSettings
{
    public string AccessToken { get; init; }

    public string RepositoryName { get; init; }
}
```

Inside of our `appsettings.json` file we need to create a section to
hold our configuration values:

```json
"GitHubSettings": {
    "AccessToken": "access-token-value",
    "RepositoryName": "youtube-projects"
}
```

And with this in place, we can configure our `GitHubSettings`:

```csharp
builder.Services.Configure<GitHubSettings>(
    builder.Configuration.GetSection("GitHubSettings"));
```

Finally, our `GitHubSettings` is properly configured and we can inject
it with `IOptions<GitHubSettings>`.

## What Could Go Wrong?

If we leave the implementation like this, we're moving the responsibility
for providing the correct configuration values to the developer. I'm not
saying we are the problem, but I've forgotten to add application settings
a few times. I'm sure this happened to you also.

Here are just a few things that can go wrong:

- Passing an incorrect section name to `IConfiguration.GetSection`
- Forgetting to add the settings values in `appsettings.json`
- Typo in a property name in the class or in the configuration
- Unbindale properties without a setter
- Data type mismatch resulting in incompatible values

Depending on which one of these mistakes is made, the application will
behave differently at runtime.

The best case scenario is that the incorrect application settings cause
a runtime exception, and you realize you have a problem and fix it.

The worst case scenario, and this happens more often than you may think,
is that the application silently fails. The application settings aren't
correctly set on the value provided by `IOptions`, but you don't get a
runtime exception. The problem may go undetected for some time.

How do we solve this?

## Validation For The Options Pattern

There is a simple way to introduce **validation** to the settings class
using **data annotations**. We just add the validation attributes that
we need to the properties of the settings class.

For example, we can add the `Required` attribute to the `GitHubSettings`
properties:

```csharp
public class GitHubSettings
{
    [Required]
    public string AccessToken { get; init; }

    [Required]
    public string RepositoryName { get; init; }
}
```

We have to slightly change how we configure the `GitHubSettings`:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations();
```

A few things to note here:

- `AddOptions` - returns an `OptionsBuilder<TOptions>` that binds to
  the `GitHubSettings` class
- `BindConfiguration` - binds the values from the configuration section
- `ValidateDataAnnotations` - enables **validation** using **data annotations**

With this in place, if we try to inject `GitHubSettings` with any of
the properties missing a value, we will get a runtime exception.

You can also define a **custom delegate** for the **validation** logic, instead
of using data annotations:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .Validate(gitHubSettings =>
    {
        if (string.IsNullOrEmpty(gitHubSettings.AccessToken))
        {
            return false;
        }

        return true;
    });
```

## Running Validation At Application Start

It would be great if we could run **validation** on the configuration values
when our application is starting, instead of at runtime.

We can do that by calling `ValidateOnStart` method when configuring
our settings class:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations()
    .ValidateOnStart(); // 👈 the magic happens here
```

When we start the application, the validation will run on `GitHubSettings`
and an exception is thrown if validation fails. The validation exception
will look something like this:

```yaml
Unhandled exception. Microsoft.Extensions.Options.OptionsValidationException:
  DataAnnotation validation failed for 'GitHubSettings' members:
```

This shortens the feedback loop, and you will know right away that you have
a problem. This is much better than finding out that you have a problem at
runtime, like in the previous examples.

## Closing Thoughts

The **Options pattern** is very flexible and allows us to use strongly typed
settings in ASP.NET Core.

If you want to see how to implement the [**Options pattern**](https://youtu.be/wxYt0motww0),
I made a [**video about it where I go into the details.**](https://youtu.be/wxYt0motww0)
I covered the differences between `IOptions`, `IOptionsSnapshot` and `IOptionsMonitor`.

And now you know how to use the `ValidateOnStart` method, which was introduced
in **.NET 6**, to validate your application settings on app start up. This allows
you to learn about configuration issues as soon as possible, instead of at runtime.

I also made a video showing how to add [validation to the Option pattern](https://youtu.be/qRruEdjNVNE).

---



In this week's newsletter I will share 5 simple tips on how
you can be a better software engineer in 2023.

I find it a little amusing that the last newsletter of the year
and is also coming out on the last day of the year.

Here are 5 tips for being a better software engineer in 2023:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

I'm confident that you will become a better software engineer if you apply one of these, but I challenge you to work on all of them in the coming year.

Let's dive in.

## 1. Keep Learning And Acquiring New Skills

The field of software engineering is constantly evolving, so it's
important to stay up-to-date with [**new technologies**](https://youtu.be/dDasAmowFts)
and best practices.

We have a new .NET release every year, and it's easy to get lost
with the latest news. How I stay up to date is through online courses,
attending conferences and meetups, or working on [**personal projects**](https://youtu.be/Ru6_b50wdfo).

Personal projects transition best to my day job, and I don't shy away
from exploring new topics. I shared many of my personal projects on [**GitHub**](https://github.com/m-jovanovic),
maybe you can find some inspiration over there.

## 2. Invest In Code Quality

In addition to writing clean and readable code, it's important to also
consider the overall quality of your code.

This includes things like performance, security, and maintainability.
By writing high-quality code, you'll be able to build systems that are
more reliable, scalable, and easier to maintain over time.

One way to write clean and quality code is [**with the help of static code analysis**](https://youtu.be/0nVT1gM4vPg).

Investing in code quality will pay dividends in the later stages of any project.

## 3. Work On Complex Systems

If you aspire to be senior engineer or software architect, you have to
work on complex systems. You need to be in a position to solve the toughest problems.

What do I consider to be a [**complex system**](https://youtu.be/Ru6_b50wdfo)?

That's difficult to say, but here are some rough guidelines:

- Microservices
- Event-driven systems
- High performance systems

Of course, you don't need a [**fancy architecture**](https://youtu.be/Ru6_b50wdfo) to work in a complex system.

If you are working in a business domain with many domain rules,
I consider that a complex system also.

You should strive to always be in a position to work on a challenging project,
this will help you grow.

## 4. Be Comfortable In The Cloud

The cloud is here to stay, and you should be familiar with at least one
of the major cloud providers:

- Microsoft Azure
- Amazon Web Services
- Google Cloud Platform

Most of them give you free credits to get started and explore the services they offer.

I stayed away from [**cloud development**](https://youtu.be/QP0pi7xe24s) for too long in my career, and now I wish I started sooner.

You want to be [**comfortable in the cloud**](https://youtu.be/QP0pi7xe24s) to be a better software engineer.

## 5. Take Care Of Yourself

Being a software engineer is mentally and physically demanding, and it's
important to take care of yourself in order to maintain a healthy work-life balance.

This includes getting enough sleep, eating well, and taking breaks when needed.

Taking care of yourself can also help you stay focused and productive in your work.

I like to take short breaks every hour or so, and not think about work for a few minutes.
This helps me replenish my energy, and allows me to continue to operate on a high level.

## What I'm Doing To Make 2023 Amazing

What I'm Doing To Make 2023 Amazing
Making New Year's resolutions is popular at the start of the year. The problem is most people get too excited about it, but then proceed to not accomplish anything in a few months.

I've found the simplest rule to always make progress is taking action. I try to make a small step forward every day. And when I look back on the year, I realize I made a lot of progress.

Again, my 5 tips so you can be a better software engineer in 2023 are:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

Where I failed most in 2022 was taking care of myself, and this will be one of my main improvement points.

I also want to learn many new things, so I can share them with you in this newsletter and on my social media channels.

I wish you a very happy and prosperous New Year.
Write lots of code, squash many bugs,
and may you have green unit tests year round.

Stay awesome! 🎁\
Milan

---



<Sponsorships>

Coordinating deployments across 37,000 stores and 500,000 database
instances for 6+ years with zero incidents? Impossible! But it's
the reality for one Fortune 100 fast-food empire who chose a NoSQL
point-of-sale database designed to smartly scale.
[**Learn how RavenDB excels as a point-of-sale DB**](https://ravendb.net/whitepapers/ravendb-point-of-sale-database/?utm_campaign=mnw_newsletter_17&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_whitepaper&utm_content=ad_link).

</Sponsorships>

In the world of software development, there are countless approaches
and methodologies to choose from. It's easy to get swayed with the
latest trends, and loose sight of architectural principles that really matter.

One of the more popular ones is **Clean Architecture**, a design approach
that prioritizes maintainability, scalability, flexibility, and productivity.

In this week's newsletter, we will explore the key benefits of using
**Clean Architecture** and how it can help your team build better software.

Let's dive in.

## What Is Clean Architecture?

**Clean Architecture**, also known as "The Onion Architecture," was first
introduced by Robert C. Martin (aka "Uncle Bob") in his book
"Clean Architecture: A Craftsman's Guide to Software Structure and Design".

At its core, **Clean Architecture** is a way of organizing a software system
in a way that separates the concerns of the various components,
making it easier to understand and maintain.

In **Clean Architecture**, the core of the system is the **"inner circle"**,
which contains the business rules and logic.

Surrounding this **inner circle** are layers of abstraction,
each one representing a different concern.

<Image src="blogs/mnw_017/clean_architecture.png" width={1920} height={1080} />

The typical outer layers are **Infrastructure** and **Presentation** layers.
The **Infrastructure** layer handles external concerns such as APIs and databases.
While the **Presentation** layer exposes an interface for clients to interact with
the application.

The key principle of **Clean Architecture** is that the **inner circle**
should not depend on the outer layers. Instead, the outer layers should
depend on the **inner circle**. This helps to ensure that the **core** of the
system is flexible and easy to modify, without worrying about the impact
on other parts of the system.

## Benefits Of Using Clean Architecture

I want to highlight some of the key benefits of using **Clean Architecture**.

### Improved Maintainability

One of the primary benefits of using **Clean Architecture** is improved **maintainability**.
By separating the concerns of the various components and enforcing the dependency rule,
it becomes much easier to understand and modify the code.
Depending on abstractions allows you to design your business logic in a flexible way,
without having to know the implementation details.

### Modularity and Separation of Concerns

**Clean Architecture** helps to create a clear **separation of concerns**
within the codebase. Each layer has a specific purpose and is decoupled
from the others, making it easier to understand and modify individual
components without affecting the rest of the system. This modularity
also makes it easier to reuse components in other projects.

### Testability

**Clean Architecture** also makes it **easier to test** and debug the code.
Because the inner circle is independent of the outer layers,
it's easier to write unit tests that focus specifically on the business
rules. This can help to catch errors early on in the development
process and reduce the overall testing effort.

### Loose Coupling of Components

**Clean Architecture** also promotes **loose coupling** between the various
components of the system. This means that it's easier to swap out
external dependencies or make other modifications without affecting
the core business logic. This can be especially useful when it comes
to upgrading or replacing technology.

### Increased Flexibility

Another key benefit of **Clean Architecture** is increased **flexibility**.
By separating the concerns of the various components, it's easier
to modify and adapt the code to changing requirements. This can be
especially useful in fast-paced environments where requirements
are constantly evolving.

### Improved Team Productivity

**Clean Architecture** can help to improve team **productivity**.
By establishing clear separation of responsibilities and well-defined
boundaries, it's easier for team members to understand their roles
and responsibilities. This can improve communication and collaboration,
leading to more efficient and effective work.

## Clean Architecture In The Real World

This all sounds nice in theory, but how does **Clean Architecture**
perform in the real world?

I have used **Clean Architecture** on roughly 10 projects in the
last 5 years, and I've had a lot of success with it. It was
easy to add new features, and scale the applications when
necessary. **Clean Architecture** can easily be broken down
into multiple modules or services, if performance is suffering
and there is a need to scale out.

One problem with the **Clean Architecture** is that it is **easy to overengineer**.

Dogmatism is a real issue, as I see many people with strong
opinions of what **Clean Architecture** should be.
I've been guilty of this myself in the past.

Recently, I try to be more **pragmatic** when using **Clean Architecture**.
I apply what I like, and give myself the flexibility of **"breaking"**
**Clean Architecture** if I think it will simplify things in the long run.

## Closing Thoughts

By following the principles of **Clean Architecture**, you can create a flexible
and maintainable codebase that is well-suited to evolving requirements and technology.

However, it's important to be **pragmatic** with **Clean Architecture** and
allow yourself to be flexible in the design, in order to simplify things in the long run.

If you want to see how to apply **Clean Architecture** in practice,
I have a
[**playlist with more than 20 videos on Clean Architecture**](https://youtu.be/tLk4pZZtiDY?list=PLYpjLpq5ZDGstQ5afRz-34o_0dexr1RGa).

---



<Sponsorships>

New to C# and .NET, or want a refresher on modern .NET?\
[**C# 11 and .NET 7 - Modern Cross-Platform Development Fundamentals**](https://packt.link/402oY)
is a concise, beginner-friendly introduction to the C# language,
.NET libraries, and ASP.NET Core.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

Did you know you can turn **PostgreSQL** into a fully-fledged **Document database**?

**Marten** is a **.NET** library that allows developers to use the **PostgreSQL**
database as both a **document database** and a fully-featured **event store**.

You don't need to install anything else to be able to use **PostgreSQL**
as a **document database**, outside of the Nuget package. **Marten** relies
on the **JSONB** support available since **PostgreSQL** 9.4.

In this week's newsletter, I want to introduce you to the basics of working
with **Marten** and show you how easy it is to get started.

Let's dive in.

## Installing And Configuring Marten

What are you going to need to start using **PostgreSQL** as a **Document datbase**?

Other than a running instance of **PostgreSQL**, of course, you will need
to install the **Marten** Nuget package:

```csharp
dotnet add package Marten
```

**Marten** can build the required database schema and necessary tables on the fly,
and I suggest using this approach in development.
For a production environment, you definitely want to apply schema
changes on your own with migration scripts.

To register **Marten** with dependency injection, you need to call the `AddMarten`
method.

Here's an example **Marten** configuration inside of a **.NET 7** application:

```csharp
builder.Services.AddMarten(options =>
{
    options.Connection(builder.Configuration.GetConnectionString("Marten"));
});
```

This will register a few services with dependency injection:

- `IDocumentStore` - used to create sessions, generate schema migrations, and do bulk inserts
- `IDocumentSession` - used for read and write operations
- `IQuerySession` - used for read operations

Let's see how we can work with documents using **Marten**.

## Storing Documents With Marten

Storing documents in the database is very straightforward. You need to create
a new `DocumentStore` instance, and open an `IDocumentSession` which exposes
methods for storing and persisting documents.

Let's see how we can store a `Product` document:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.OpenSession();

var product = new Product
{
    Name = "C# 11 and .NET 7 - Modern Cross-Platform Fundamentals",
    Price = 46.87
};

session.Store(product);

await session.SaveChangesAsync();
```

We're creating a new `DocumentStore` instance which we use to open
a session to **PostgreSQL**. And then we just call `Store` and pass in
the `Product` instance. Note that **Marten** will populate the
`Product.Id` at this point. **Marten** can populate keys of `Guid`,
`int`, `long`, and other data types. It uses the HiLo algorithm
for numeric keys. Finally, when we call `SaveChangesAsync` the
`Product` is serialized into **JSON** and persisted as a document.

An important thing to be aware of is that the `IDocumentSession`
created by `OpenSession` doesn't track changes on the entities automatically.
You need to create a session with dirty checking enabled by
calling `DirtyTrackedSession` on the `DocumentStore` to enable
automatic change detection.

## Querying Documents With Marten

**Marten** has rich support for querying documents in the database.
You can write and execute queries using **LINQ**, which you are
familiar with if you worked with **EF Core**. And you can also
write and execute **SQL** queries, because it's still a **PostgreSQL**
database underneath.

Here's an example query to return products that have a higher price
than the one which is specified:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.QuerySession();

var products = session.Query<Product>().Where(p => p.Price > 9.99).ToList();
```

**Marten** also has support for:

- Including related documents
- Batched queries
- Paging
- Full text search

## Advanced Options With Marten

**Marten** can utilize the full capabilities **PostgreSQL** has to offer,
notably transactions and indexing. **Marten** sessions are transactional
by default, either all of the documents are persisted together or
none of them are. And you can configure indexes on your documents
for faster queries.

**Marten** isn't just a **document database** on top of **PostgreSQL**!

You have fully-fledged support for **event sourcing** with **Marten**,
as well as projections. This makes it a perfect choice for
implementing **CQRS**. But this is a topic for a separate newsletter.

## Closing Thoughts

I'm absolutely amazed with [Marten](https://martendb.io/) and what it has to offer.
And **PostgreSQL** is also my favorite database, so it's like a match
made in heaven. I don't get too excited about learning new
technologies, but **Marten** has been an endless source
of joy this past week.

I still need to explore a few more topics before I can consider it
for production use:

- Schema migrations
- Relationships and foreign keys
- Advanced configuration options

Considering that **PostgreSQL** is cheaper than most **document databases**,
I think using **Marten** is an interesting alternative. And if you are
familiar with **SQL** databases, you can still use all of that knowledge.

---



<Sponsorships>

Keeping your business logic and data in-sync doesn't have to be a drag.
Database configuration-as-code and auto-indexes mean your logic is always
querying against what you expect after every release.
[**Take off with RavenDB today**](https://ravendb.net/learn/bootcamp/?utm_campaign=mnw_newsletter_15&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_bootcamp&utm_content=ad_link)
and fly faster tomorrow.

</Sponsorships>

In this week's newsletter we are going to explore **Minimal APIs**,
which were introduced in **.NET 6**.

**Minimal APIs** were introduced to remove some of the ceremony
of creating traditional APIs with controllers.
To define an endpoint, you can use the new extension
methods, such as `MapGet` to define a **GET** endpoint.

I see one big issue with **Minimal APIs**, and that is the lack
of clear guidance around how to structure applications
built with **Minimal APIs**.

In this newsletter, I want to offer a few solutions for that problem.

Let's dive in.

## How To Create Minimal APIs?

Let's define a simple **Minimal API** application with two endpoints.
We're going to create one `GET` endpoint for getting a list of products.
And one `POST` endpoint for saving a product to the database.

We're using the powerful **DI** feature that allows us to inject services
as expression arguments, which you can see in the two expressions below
where we are injecting the `AppDbContext`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

var app = builder.Build();

app.MapGet("/products", async (AppDbContext dbContext) =>
{
    return Results.Ok(await dbContext.Products.ToListAsync());
});

app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
{
    dbContext.Products.Add(product);

    await dbContext.SaveChangesAsync();

    return Results.Ok(product);
});

app.Run();
```

And with this we have a functioning **Minimal API** that we can develop further
as we continue to add more endpoints.

## The Problem With Maintaining Minimal APIs

There is one potential problem with structuring our **Minimal APIs** like
in the previous example. If we keep adding the **Minimal API** endpoints
in the same file, our API will become hard to maintain as it grows
in complexity. How can we solve the maintance problem with **Minimal APIs**?

One solution can be to use extension methods to encapsulate the
definiton of the **Minimal API** endpoints.

Here's an example of that:

```csharp
public static class ProductsModule
{
    public static void RegisterProductsEndpoints(this IEndpointRouteBuilder  endpoints)
    {
        endpoints.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        endpoints.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

And then inside of `Program` we need to register the endpoints:

```csharp
app.RegisterProductsEndpoints();
```

You can see that this simplifies our **Minimal API** definition,
and we also have our endpoints grouped by feature in one place.
I think this improve the maintainability of **Minimal APIs**,
but it comes at a cost. And that cost is having to define
extension methods for each group of endpoints you want to encapsulate,
and then you have to remember to call that extensions method in `Program`.

Can we do better?

## Structuring Minimal API Projects With Modules

I want to introduce you to an interesting open source library
called [Carter](https://github.com/CarterCommunity/Carter),
which has a concept of modules that we can use to group endpoints.

Here's how we can define our `ProductsModule` with **Carter**:

```csharp
public class ProductsModule : ICarterModule
{
    public void AddRoutes(IEndpointRouteBuilder app)
    {
        app.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

This takes care of configuring our **Minimal API** endpoints, but we still
need to tell the framework to use these endpoints. We have to slightly
modify the `Program` to register the required **Carter** services by
calling `AddCarter`, and also map our endpoints by calling `MapCarter`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

builder.Services.AddCarter();

var app = builder.Build();

app.MapCarter();

app.Run();
```

When we want to define additional **Minimal API** endpoints we just need to
implement a new `ICarterModule`, and register our endpoints. **Carter** will
automatically take care of registering the new endpoints after that.

## Would I Use Minimal APIs In a Real Project?

I think **Minimal APIs** have evolved nicely since they were first introduced
in **.NET 6**. I would be careful with using them in very large applications,
but I'm definitely going to explore options for using them on smaller projects.

A good use case can be for building a microservice that has a limited
number of endpoints.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_14&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter we will talk about running **background tasks** in **ASP.NET Core**.
After reading this newsletter, you will be able to set up a **background task**
and have it up and running within minutes.

**Background tasks** are used to offload some work in your application to the background,
outside of the normal application flow. A typical example can be asynchronously
processing messages from a queue.

I will show you how to create a simple **background task** that runs once and completes.

And you will also see how to configure a continuous **background task**, that repeats after a specific period.

Let's dive in.

## Background Tasks With IHostedService

You can define a **background task** by implementing the `IHostedService` interface.
It has only two methods.

Here's what the `IHostedService` interface looks like:

```csharp
public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);

    Task StopAsync(CancellationToken cancellationToken);
}
```

All you have to do is implement the `StartAsync` and `StopAsync` methods.

Inside of `StartAsync` you would usually perform the background processing.
And inside of `StopAsync` you would perform any cleanup that is necessary,
such as disposing of resources.

To configure the **background task** you have to call the `AddHostedService` method:

```csharp
builder.Services.AddHostedService<MyBackgroundTask>();
```

Calling `AddHostedService` will configure the **background task**
as a **singleton** service.

So does dependency injection still work in `IHostedService` implementations?\
Yes, but you can only inject **transient** or **singleton** services.

However, I don't like to implement the `IHostedService` interface myself.
I prefer using the `BackgroundService` class instead.

## Background Tasks With BackgroundService

The `BackgroundService` class already implements the `IHostedService` interface,
and it has an `abstract` method that you need to override - `ExecuteAsync`.
When you are using the `BackgroundService` class, you only have to think about
the operation you want to implement.

Here's an example **background task** that runs **EF** migrations:

```csharp
public class RunEfMigrationsBackgroundTask : BackgroundService
{
    private readonly IServiceProvider _serviceProvider;

    public RunEfMigrationsBackgroundTask(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = _serviceProvider.CreateScope();

        await using AppDbContext dbContext =
            scope.ServiceProvider.GetRequiredService<AppDbContext>();

        await dbContext.Database.MigrateAsync(stoppingToken);
    }
}
```

The **EF** `DbContext` is a **scoped** service, which we can't inject directly
inside of `RunEfMigrationsBackgroundTask`. We have to inject an instance of
`IServiceProvider` which we can use to create a custom service scope,
so that we can resolve the scoped `AppDbContext`.

I would _not recommend_ running the `RunEfMigrationsBackgroundTask` in production.
**EF** migrations can easily fail and you'll run into problems.
However, I think it's perfectly fine for local development.

## Periodic Background Tasks

Sometimes we want run a **background task** continuously, and have it
perform some operation on repeat. For example, we want consume messages
from a queue every ten seconds. How do we build this?

Here's an example `PeriodicBackgroundTask` to get you started:

```csharp
public class PeriodicBackgroundTask : BackgroundService
{
    private readonly TimeSpan _period = TimeSpan.FromSeconds(5);
    private readonly ILogger<PeriodicBackgroundTask> _logger;

    public PeriodicBackgroundTask(ILogger<PeriodicBackgroundTask> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using PeriodicTimer timer = new PeriodicTimer(_period);

        while (!stoppingToken.IsCancellationRequested &&
               await timer.WaitForNextTickAsync(stoppingToken))
        {
            _logger.LogInformation("Executing PeriodicBackgroundTask");
        }
    }
}
```

We're using a [PeriodicTimer](https://learn.microsoft.com/en-us/dotnet/api/system.threading.periodictimer?view=net-6.0)
to asynchronously wait for a given period, before executing our **background task**.

## What If You Need A More Robust Solution?

It should be obvious by now that `IHostedService` is useful when you need
simple **background tasks** that are running while your application is running.

What if you want to have a scheduled **background task** that runs at 2AM every day?

You can probably build something like this yourself, but there are existing solutions
that you should consider first.

Here are two popular solutions for running **background tasks** that I worked with before:

- [Quartz](https://www.quartz-scheduler.net/)
- [Hangfire](https://www.hangfire.io/)

I also have an example of [using Quartz for processing Outbox messages](https://youtu.be/XALvnX7MPeo)
on my YouTube channel that you can take a look at.

---



<Sponsorships>

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_13&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter, we're going to explore the new
`ExecuteUpdate` and `ExecuteDelete` methods that were released with **EF7**.

`ExecuteUpdate` allows us to write a query and run a bulk update
operation on the entities matching that query.

Similarly, `ExecuteDelete` allows us to write a query and delete
the entities matching that query.

We can significantly improve performance using the new methods in some scenarios, and I'm going to show you what those scenarios are.

Let's dive in.

## Updating And Deleting Entities Before EF Core 7

If you want to update a collection of entities before **EF7**,
you need to load the entities into memory using the `DatabaseContext`.

The **EF ChangeTracker** will then track any changes made to these entities.
When you are ready to commit the changes to the database,
you simply call the `SaveChanges` method.

Here's an example where we load a few notifications,
and we want to snooze them so they aren't sent:

```csharp
var notifications = dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ToList();

foreach(var notification in notifications)
{
    notification.Snoozed = true;
}

dbContext.SaveChanges();
```

**EF7** will generate the following **SQL** statement to update the records in the database:

```sql
UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_1;

...

UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_N;
```

Notice that for every notification we end up with one **SQL UPDATE** statement.
This won't scale well as the number of notifications increases.

## Updating Entities With ExecuteUpdate

With **EF7**, we now have access to the new `ExecuteUdpate` method.
It also has an asynchronous version - `ExecuteUpdateAsync`.

How do you use it?

You need to write a query that will select the records you want to update,
and then call the `ExecuteUpdate` method on the resulting `IQueryable`.

Let's rewrite the previous example using the new approach:

```csharp
dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ExecuteUpdate(s => s.SetProperty(
        n => n.Snoozed,
        n => true));
```

In the call to `ExecuteUpdate` we call the `SetProperty` method to specify
which properties we want to update, and what values we want to set.
The `SetProperty` method can be called multiple times, if you need to update more than one property.

In this case, **EF7** will generate the following **SQL** query:

```sql
UPDATE n
SET n.[Snoozed] = TRUE
FROM [Notifications] AS n
WHERE n.[Snoozed] = FALSE;
```

Notice that this time we only have one **SQL** query being sent to the database.
This is a major performance improvement. It can be as much as 10x faster
than the old version, from my testing.

## Deleting Entities With ExecuteDelete

Let's also see how we can do bulk deletes using the `ExecuteDelete` and `ExecuteDeleteAsync` methods.

Again, you have to write a query that will select the records you want to delete,
and then call the `ExecuteDelete` method on the resulting `IQueryable`.

If you want to delete all snoozed notifications:

```csharp
dbContext
    .Notifications
    .Where(n => n.Snoozed)
    .ExecuteDelete();
```

And **EF7** will generate the following **SQL** query:

```sql
DELETE FROM n
FROM [Notifications] AS n
WHERE n.[Snoozed] = TRUE;
```

I think this will be incredibly useful when you want to
delete records in the database based on a specific condition.

## Transactions, Change Tracking And Query Filters With Bulk Methods

You need to be aware how transactions and change tracking
work with the new bulk methods. `ExecuteUpdate` and `ExecuteDelete` will
immediately go to database, and run the **SQL** query.

**What does this mean for transactions?**

If you want to run a bulk method together with other updates
applied with `SaveChanges`, by default they won't run in the same transaction.
You need to open an explicit **transaction** using the `DatabaseContext` to keep everything consistent.

**What does this mean for change tracking?**

`ExecuteUpdate` and `ExecuteDelete` run directly on the database, without loading any entities into memory.
**EF7** will not track these entities in the `ChangeTracker`.

If you have any database interceptors defined, they won't execute
after calling one of the bulk update methods.
This also means that if you override `SaveChanges`
to add custom behavior, it won't be called.

**Do Query Filters still work?**

Yes, **query filters** will be **correctly applied** when calling `ExecuteUpdate` or `ExecuteDelete`.

## When Should You Use The New Bulk Methods?

I think this is an excellent new addition to **EF7**,
and it solves a real problem when you need to run
a typical **UPDATE** or **DELETE** query with a
**WHERE** statement applied.

Previously, you had to write raw **SQL** and execute it using something like **Dapper**.

I will likely use this approach when it applies to my projects.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to show you how you can use
the powerful **options pattern** in **ASP.NET Core 7**.

The **options pattern** uses classes to provide **strongly typed settings**
in your application at runtime.

The values for the **options** instance can come from multiple sources.
The typical use case is to provide the settings from application configuration.

You can configure the **options pattern** in a few different ways in **ASP.NET Core**.
I want to discuss some of the approaches and their potential benefits.

Let's dive in.

## Creating The Options Class

I want to set the stage first, by creating the **options** class and
explaining what settings we want to bind to it.

We want to configure JWT Authentication for our application,
so we decided to create the `JwtOptions` class to hold that configuration:

```csharp
public class JwtOptions
{
    public string Issuer { get; init; }
    public string Audience { get; init; }
    public string SecretKey { get; init; }
}
```

And let's imagine that inside of our `appsettings.json` file
we have the following configuration values:

```json
"Jwt": {
    "Issuer": "Gatherly",
    "Audience": "Gatherly",
    "SecretKey": "dont-tell-anyone!"
}
```

Alright, that's looking good. Now I want to show you a few ways to bind
the values from JSON to our `JwtOptions` class.

## Setting Up Options Pattern Using IConfiguration

The most straightforward approach is to use the `IConfiguration` instance
that we can access while registering services.

We need to call the `IServiceCollection.Configure<TOptions>` method,
and specify the `JwtOptions` as the generic argument:

```csharp
builder.Services.Configure<JwtOptions>(
    builder.Configuration.GetSection("Jwt"));
```

It doesn't get simpler than this, does it?

The only downside is that we are limited to the configuration
values provided through application configuration.

This can be extended to include environment variables and user secrets also.

## Setting Up Options Pattern Using IConfigureOptions

If you want a more robust approach, I have you covered.
We're going to use the `IConfigureOptions` interface to define a class
to configure our **strongly typed options**.

There are two steps that we need to follow in this case:

- Create the `IConfigureOptions` implementation
- Call `IServiceCollection.ConfigureOptions<TOptions>` with our `IConfigureOptions`
  implementation as the generic argument

To start off, we will create the `JwtOptionsSetup` class:

```csharp
public class JwtOptionsSetup : IConfigureOptions<JwtOptions>
{
    private const string SectionName = "Jwt";
    private readonly IConfiguration _configuration;

    public JwtOptionsSetup(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    public void Configure(JwtOptions options)
    {
        _configuration
            .GetSection(SectionName)
            .Bind(options);
    }
}
```

We wrote more code, to achieve the same thing. Was it worth it?

Perhaps, if you consider that we now have access to **dependency injection** in the `JwtOptionsSetup` class.
This means that we can resolve other services that we can use to get the configuration values.

We also need to tell the application to use the `JwtOptionsSetup` class:

```csharp
builder.Services.ConfigureOptions<JwtOptionsSetup>();
```

When we try to inject our `JwtOptions` somewhere, the `JwtOptionsSetup.Configure`
method will be called first the calculate the correct values.

## Injecting Options With IOptions

We've seen a few examples for how to configure the **options pattern** with the `JwtOptions` class.

But how do we actually use the **options pattern**?

Easy, you just need to inject `IOptions<JwtOptions>` from the constructor.

I'll just show the `JwtProvider` constructor here, for brevity.

```csharp
public JwtProvider(IOptions<JwtOptions> options)
{
    _options = options.Value;
}
```

The actual `JwtOptions` instance is available on the `IOptions<JwtOptions>.Value` property.

The `IOptions` instance that we injected here is configured
as a **Singleton** in dependency injection. This is very important to be aware of.

## What About IOptionsSnapshot and IOptionsMonitor?

If you want to use the latest configuration values every time
you inject an **options** class, then injecting `IOptions` won't work.

However, you can use the `IOptionsSnapshot` interface instead:

- It provides the latest configuration snapshot (cached per request)
- It is registered as a **Scoped** service
- It detects configuration changes after application start

You can also use the `IOptionsMonitor` which retrieves the current
option values at any time, and it's a **Singleton** service.

## Wrapping up

The **options pattern** gives us a way to use strongly typed configuration classes in our application.

We can configure the options class in a simple way with [`IConfiguration`](#setting-up-options-pattern-using-iconfiguration),
or we can create an [`IConfigureOptions`](#setting-up-options-pattern-using-iconfigureoptions)
implementation if we need something more powerful.

When it comes to using the **options pattern**, we have three approaches:

- [`IOptions`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptions-1?view=dotnet-plat-ext-7.0)
- [`IOptionsSnapshot`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionssnapshot-1?view=dotnet-plat-ext-7.0)
- [`IOptionsMonitor`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionsmonitor-1?view=dotnet-plat-ext-7.0)

Deciding which of them to use in your application depends on what kind of behavior you want.
If you don't need to support refreshing configuration values
after application start, [`IOptions`](#injecting-options-with-ioptions) is a perfect solution.

---



<Sponsorships>

Wish you could calculate prices of your 2.68MM online products 3,000X faster without a batch job?
By ditching their 12-JOIN SQL incantations, the “Amazon of medical supplies”
went from pricing items in 3,000 milliseconds to 1 millisecond - all from within the database.
[**Learn how RavenDB revolutionized the world's largest online medical supply store.**](https://ravendb.net/articles/3000x-faster-database-transactions-for-the-worlds-largest-medical-supplier?utm_campaign=mnw_newsletter_11&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_use_case&utm_content=ad_link)

</Sponsorships>

In this week's newsletter I want to highlight a few interesting things
that are now available with the release of **C# 11** and **.NET 7**.

In case you missed it, **.NET 7** was released November 8th.

There are many new features, and you can be sure I had a hard time choosing which ones to highlight.

Here's what we are going to cover:

- [Required members](#required-members)
- [Generic attributes](#generic-attributes)
- [Static abstract members in interfaces](#static-abstract-members-in-interfaces)
- [`file` keyword](#file-keyword)
- [LINQ Order and OrderDescending](#linq-order-and-orderdescending)

Let's see what the new features look like!

## Required Members

We can now define a class member as required by using the `required` keyword.
It can be applied to a _field_ or _property_ and it tells the compiler
these members must be initialized by all constructors or by the object initializer.

Why is this useful?

Before **C# 11**, the only way to enforce a property being set was through a constructor.
If you used an object initializer you could bypass the constructor and not initialize some properties.

Here's how you can say that a property is required:

```csharp
public class ContentCreator
{
    public required string Firstname { get; init; }
    public string? MiddleName { get; init; }
    public required string LastName { get; init; }
}
```

If you try to create a new `ContentCreator` instance without initializing
the `required` properties you get a compile error:

```csharp
var creator = new ContentCreator
{
    FirstName = "Milan" // Error: No LastName
}
```

## Generic Attributes

You can now declare a _generic_ class whose base class is `Attribute`.

Before **C# 11**, if you wanted to pass in a type as a parameter
to an `Attribute` you would need to pass it through the constructor:

```csharp
public class TypedAttribute : Attribute
{
    public TypedAttribute(Type t) => Param = t;

    public Type Param { get; }
}
```

And here's how you would use it with the `typeof` operator:

```csharp
[TypedAttribute(typeof(int))]
public int Method() => default;
```

Using the generic attributes feature, you can now define it like this:

```csharp
public class TypedAttribute<T> : Attribute { ... }
```

Now, we can specify the type parameter as a generic argument:

```csharp
[TypedAttribute<int>()]
public int Method() => default;
```

## Static Abstract Members in Interfaces

This is a very interesting feature that allows abstracting of static operations.
An example of this would be operators.

```csharp
public interface IMonoid<TSelf> where TSelf : IMonoid<TSelf>
{
    public static abstract TSelf operator +(TSelf a, TSelf b);

    public static abstract TSelf Zero { get; }
}
```

How can we use the `IMonoid` interface?

It may be confusing at first, since the members are virtual
and there is no instance to call the virtual members on.
The solution is to use generics and let the compiler infer the rest.

Here's a simple example:

```csharp
T AddAll<T>(params T[] elements) where T : IMonoid<T>
{
    T result = T.Zero;

    foreach (var element in elements)
    {
         result += element;
    }

    return result;
}
```

If you want to learn more, check out the docs on
[static abstract interface methods](https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#static-abstract-interface-methods)
and [generic math](https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#generic-math).

## File Keyword

With the new `file` keyword you can define a type whose scope and visibility
is restricted to the file in which it is declared.

```csharp
file class HiddenClass
{
}
```

This feature is practical when used inside of source generators, to avoid collisions when naming generated types.

But you may be able to find a use for it in your application.

## LINQ Order and OrderDescending

The new `Order` and `OrderDescending` methods allow us to sort an
`IEnumerable`, which simplifies the code for sorting.

Here's an example of ordering an array:

```csharp
var array = new[] { 19, 91, 21 };

var arrayAsc = array.Order();

var arrayDesc = array.OrderDescending();
```

I want to highlight that `IQueryable` also supports the new methods.

## Will You Upgrade to .NET 7?

**.NET 7** is not an LTS (Long Term Support) release,
and will be in support until May 2024,
with **.NET 8** releasing in November 2023.

Here are a few reasons why you should consider upgrading:

- Major performance improvements
- New features in **.NET 7**
- New features in **EF Core 7**
- Easier migration to **.NET 8**

I will be moving some of my new projects from **.NET 6** to **.NET 7**.

And I will also upgrade all of my YouTube content to **.NET 7**.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
[**Visual Studio extension**](https://github.com/QuadSpinner/Highlighter) that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter, we will take a look at five different ways to check if a collection **contains duplicates**.

I'm going to explain the idea behind each **algorithm**, discuss the **algorithm complexity** (Big O Notation), and at the end, we'll look at some **benchmark results**.

The five approaches for finding a duplicate will use the:

- [`foreach`](#check-for-duplicates-with-foreach-loop) loop
- LINQ [`Any`](#check-for-duplicates-with-linq-any) method
- LINQ [`All`](#check-for-duplicates-with-linq-all) method
- LINQ [`Distinct`](#check-for-duplicates-with-linq-distinct) method
- LINQ [`ToHashSet`](#check-for-duplicates-with-linq-tohashset) method

Let's see how we can implement each approach!

## Check For Duplicates With ForEach Loop

The first implementation will use the `foreach` loop and the `HashSet` data structure.

Here's the code for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   foreach(var element in enumerable)
   {
      if (!set.Add(element))
      {
         return true;
      }
   }

   return false;
}
```

The idea is simple:

- Loop through the collection
- Add each element to the `HashSet`
- When `HashSet.Add` returns false we found a duplicate
- If we loop through the entire collection there are no duplicates

In terms of **algorithm complexity**, this would be **O(n)** or linear complexity.
This is because there's only one iteration through the collection.

Adding an element to a `HashSet` is a constant operation - **O(1)**.
So it doesn't affect the overall complexity.

## Check For Duplicates With LINQ Any

We'll combine the idea from the previous implementation of
using the `HashSet` and pair it with the LINQ `Any`
method to iterate over the collection.

Here's the implementation for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   return enumerable.Any(element => !set.Add(element));
}
```

You can see this implementation is significantly shorter.
But it works the same as the one with the `foreach` loop.

If any element in the collection satisfies the specified expression,
`Any` will _short-circuit_ and return `true`.
Otherwise, it will iterate over the entire collection and return `false`.

We're still looking at linear complexity here, **O(n)**.

## Check For Duplicates With LINQ All

For our third implementation, we will use the opposite
of the LINQ `Any` method - the LINQ `All` method.

Here's the implementation with LINQ `All`:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   return !enumerable.All(set.Add);
}
```

The idea here is a little different than in the previous implementation.

`All` will return `true` if all elements in a collection
satisfy the specified expression.

If at least one element doesn't satisfy the condition -
in our case when a **duplicate** is found - it will _short-circuit_ and return `false`.

This is still linear complexity, **O(n)**.

## Check For Duplicates With LINQ Distinct

So far, we've seen a few implementations using the `HashSet` data structure.
Now let's consider a different approach.

We can use the LINQ `Distinct` method to check for duplicates.

Here's the code for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   return enumerable.Distinct().Count() != enumerable.Count();
}
```

The idea is first find the `Distinct` elements and `Count` them,
and then compare that to the number of all elements.

If the number of distinct elements is not equal to
the number of all elements, we have a **duplicate** value.

In terms of **algorithm complexity**, this is still linear complexity.

But we have at least two iterations through the collection
or three in the worst-case scenario.

We have one iteration for `Distinct` and one more
iteration for the call to `Count` right after that.
The last call to `Count` can return in constant time,
if the collection is an `array` or `List`.

## Check For Duplicates With LINQ ToHashSet

For the last implementation we will use the LINQ `ToHashSet` method.

It takes a collection and creates a `HashSet` instance from it.

Here's what the `ContainsDuplicates` implementation looks like:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   return enumerable.ToHashSet().Count != enumerable.Count();
}
```

We compare the number of elements in the `HashSet` to the number of elements in the collection.

If they are different, we have a **duplicate** value.

This is also linear complexity, **O(n)**.

## Benchmark Results

Now that we've seen our implementations let's put them to the test.

I ran the benchmark for collections of varying sizes:

- 100
- 1,000
- 10,000

Each collection contains exactly one duplicate value located somewhere around the middle of the collection.

Here are the results:

<Image src="blogs/mnw_010/benchmark.png" width={1339} height={627} />

The approach using the `foreach` loop comes out as the clear winner in terms of performance.

However, I would lean towards using the implementations with LINQ `Any` or `All` because of their simplicity.

You can find the [source code for the benchmark](https://github.com/m-jovanovic/find-duplicates-benchmark)
on my GitHub. Feel free to submit a PR with a faster implementation if you can think of one.

---



<Sponsorships>

Delivered straight to your inbox every week,
[**Microns.io**](https://www.microns.io/) is a newsletter that helps you discover the best micro-startups for sale.
Join 3,500+ entrepreneurs and investors looking for their next micro-startup
acquisition opportunity and, along the way, learn about how great deals are done.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets
you highlight important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter, I'll show you how you can remove repetitive conditions in **EF Core** database queries.

Which kinds of queries fit this description?

An example would be when you implement **soft-delete**, and have to check if a record was **soft-deleted** or not in every query.

Also, it's practical if you're working in a multi-tenant system and need to specify a `tenantId` on every query.

**EF Core** has a powerful feature that can help you remove repetitive conditions from your code.

I'm talking about [Query Filters](https://learn.microsoft.com/en-us/ef/core/querying/filters).

Let's see how we can implement it.

## How To Apply Query Filters

Before introducing **Query Filters**, we will see how the standard approach looks.
We have an `Orders` table that supports **soft-deleting**.
And we never want to return **soft-deleted** orders.

We'll start with an `Order` entity that has an `IsDeleted` property.

```csharp
public class Order
{
   public int Id { get; set; }
   public bool IsDeleted { get; set; }
}
```

And we have a business requirement that we can only query orders that are not deleted.

Here's what an **EF** query to get a single `Order` might look like:

```csharp
dbContext
   .Orders
   .Where(order => !order.IsDeleted)
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

This works perfectly for what we need to do.

However, we need to remember to apply this condition every time we want to query the `Order` entity.

Now, let's see how we can define a **Query Filter** on the `Order` entity to
apply this check when querying the database.

Inside of the `OnModelCreating` method on the database context, we need to
call the `HasQueryFilter` method and specify the expression we want:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

Now we can omit the **soft-delete** check from the previous **LINQ** expression:

```csharp
dbContext
   .Orders
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

And this is the **SQL** that **EF** will generate with the **Query Filter**:

```sql
SELECT o.*
FROM Orders o
WHERE o.IsDeleted = FALSE AND o.Id = @orderId
```

## Disabling Query Filters

You may run into a situation where you need to disable **Query Filters** for a specific query.
Luckily, there is an easy way to do this.

In your **LINQ** expression, you need to call the `IgnoreQueryFilters` method,
and all the **Query Filters** configured for this entity will be disabled:

```csharp
dbContext
   .Orders
   .IgnoreQueryFilters()
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

Be careful when doing this, as you can easily introduce unwanted behavior in your application.

## Good Things To Know Before Using Query Filters

Here are a few more details that you should know about **Query Filters** before using them.
Hopefully, this will save you some trouble if you decide to use them in your application.

**Configuring multiple Query Filters**

Configuring multiple **Query Filters** on the same entity will only apply the last one.
If you need more than one condition, you can do that with the logical `AND` operator (&&).

**Ignoring specific Query Filters**

If you need to ignore a specific expression in a **Query Filter** and leave the rest in place,
unfortunately, you can't do that. Only one **Query Filter** is allowed per entity type.

One solution is calling `IgnoreQueryFilters`, which will remove the configured **Query Filter**
for that entity type. And then manually apply the condition that you need for that specific query.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

In this week's newsletter, we'll see how we can work with **locking** in **.NET 6**.

We won't talk about how the lock is actually implemented at the operating system level.
Instead, I will focus on application-level **locking** mechanisms.

**Locking** allows us to control how many **threads** can access some piece of code.
Why would you want to do this?

Usually because you want to protect access to **expensive resources**,
and you need the **concurrency control** that locking enables.

We will use a simple `BankAccount` class with a `Deposit` method
to illustrate how to implement locking.

## The C# Lock Statement

The C# language supports locking with the [`lock` statement](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/lock).
You can use the `lock` statement to define a code block that only one thread can access.

The `lock` statement acquires a mutual-exclusion lock (mutex) for a given object,
executes the statement block, and releases the lock.

```csharp
lock(_lock)
{
   // Your code...
}
```

Here `_lock` is a reference type, usually an `object` instance.

Let's see how we can implement the `BankAccount` class using the `lock` statement:

```csharp
public class BankAccount
{
   private static readonly object _lock = new();
   private decimal _balance;

   public void Deposit(decimal amount)
   {
      lock(_lock)
      {
         _balance += amount;
      }
   }
}
```

The first thread to reach and execute the `lock` statement will be allowed to update
the `_balance`. Any other threads will block until the lock is released.

## Locking With Semaphore

The [`Semaphore`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphore?view=net-6.0)
class is another option we can use to achieve the same effect.

We'll use the `Semaphore` constructor to set the `initialCount` to 0,
which means that the `Semaphore` is open at the start.
And we will also set the `maximumCount` to 1,
which means that only one thread is allowed to enter the `Semaphore`.

Let's see how we can implement the `BankAccount` class using the `Semaphore`:

```csharp
public class BankAccount
{
   private static readonly Semaphore _semaphore = new(
      initialCount: 0,
      maximumCount: 1);

   private decimal _balance;

   public void Deposit(decimal amount)
   {
      _semaphore.WaitOne();

      _balance += amount;

      _semaphore.Release();
   }
}
```

To enter the `Semaphore`, we have to call the `WaitOne` method.

If no thread was previously inside, our thread is allowed
to enter the `Semaphore` and update the balance.

After updating the balance, we call the `Release` method to
release the `Semaphore` for other threads that might be waiting.

## Asynchronous Locking With SemaphoreSlim

What if we wanted to call an asynchronous method in a locked context?

We can't use the `lock` statement as it doesn't support asynchronous calls.
Awaiting an asynchronous call inside a `lock` statement will cause a compilation error.

The `Semaphore` class can solve this problem.

But I want to show you another option that we have, [`SemaphoreSlim`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphoreslim?view=net-6.0).
It's a lightweight alternative to the `Semaphore` class and has `async` methods.

Let's see how we can implement the `BankAccount` class using `SemaphoreSlim`:

```csharp
public class BankAccount
{
   private static readonly SemaphoreSlim _semaphore = new(
      initialCount: 0,
      maximumCount: 1);

   private decimal _balance;

   public async Task Deposit(decimal amount)
   {
      await _semaphore.WaitAsync();

      _balance += amount;

      _semaphore.Release();
   }
}
```

Notice that I updated the `Deposit` method to return a `Task`.

This time, we're calling `WaitAsync` to block the current
thread until it can enter the semaphore.

After updating the balance, we call the `Release` method
to release the `SemaphoreSlim` like in the previous example.

## Are There Other Options For Locking in .NET?

So far I mentioned three options to implement locking:

- [`lock` statement](#the-c-lock-statement)
- [`Semaphore`](#locking-with-semaphore)
- [`SemaphoreSlim`](#asynchronous-locking-with-semaphoreslim)

However, **.NET** has other classes for **concurrency control** that you can
explore like
[`Monitor`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.monitor?view=net-6.0),
[`Mutex`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.mutex?view=net-6.0),
[`ReaderWriterLock`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.readerwriterlock?view=net-6.0)
and many more.

I hope you enjoyed this brief introduction to a very complex topic.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

Performance optimization is my favorite thing about software engineering.
Over the last 5 years, I've encountered various performance problems
that taught me different ways to overcome them.

About a month ago, I ran into an issue with an API endpoint that wasn't scaling well.

This endpoint is used to calculate a report for an e-commerce web application.
It needed to talk to multiple modules (services) to gather all the necessary data,
combine it and perform the calculations.

I made a [post about it on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:6966700329111310336/)
that resonated with many people.

<Image src="blogs/mnw_007/linkedin_post.png" width={762} height={742} />

In this newsletter, I want to break down what I did to achieve a **15x performance improvement**.

## Focus On Bottlenecks First

The first thing I do when I'm solving a performance problem
is determine where the slowest piece of the code is.
Fixing this part of the code will usually give the most significant improvement.

Solving one bottleneck can also reveal where the next bottleneck is.\
This is a continual process.

In my situation, there were a few bottlenecks:

- Calling the database from a loop
- Calling an external service multiple times
- Executing a complex calculation multiple times with identical parameters

How can you measure performance?

A simple approach can be using `System.Timers.Timer` where you
manually log execution times between method calls.
Or you can use a performance profiler.

## Reduce The Number of Round Trips

A round trip between your application and a database
(or some other service) can last 5-10ms, or more.
If you have many round trips in your flow, it's going to add up quickly.

Here are a few things you can do reduce the number of round trips:

1. Don't call the database from a loop. This can usually be solved with a simple query like this:

```sql
   SELECT * FROM [TableName] WHERE Id IN (list_of_ids)
```

2. Use a query that returns multiple result sets from the database.
   One library that supports this is [Dapper](https://github.com/DapperLib/Dapper), with the `QueryMultiple` method.

3. If you need to make multiple calls to another service, try to convert that into one call.
   And in the service, aggregate the required data and return everything at once.

## Parallelize External Calls

I had a situation where I was awaiting multiple asynchronous calls from a few services.
These calls had no dependencies on each other, so I used a simple technique
to gain a significant performance improvement.

Let's say you're awaiting two tasks:

```csharp
var task1Result = await CallService1Async();

var task2Result = await CallService2Async();

// Use the results.
```

A simple way to parallelize these calls is using the `Task.WhenAll` method:

```csharp
var task1 = CallService1Async();

var task2 = CallService2Async();

await Task.WhenAll(task1, task2);

// Use the results.
task1.Result;
task2.Result;
```

Notice that I'm directly accessing the `Result` property on the tasks.
This can be **detrimental** if you're using it to block on an asynchronous call,
and can even lead to deadlocks.

However, in this situation it is perfectly safe to do,
because the two tasks will have completed after the call to `Task.WhenAll` completes.

Of course, whether or not these tasks will be executed in parallel
when calling `Task.WhenAll` depends on a few factors, which I won't cover here.

## Caching As a Last Resort

I try to leave caching for the end, after I have exhausted
all other possibilities to improve performance.
While I love to use caching in general, I'm aware
it can introduce some unwanted behavior when data is stale.

You have to consider how long you can safely cache the data,
and how you are going to clear the cache if the underlying data changes.

In simple applications, I use `IMemoryCache`
that is available in **ASP.NET Core** out of the box.
But you can also use an external cache like [Redis](https://redis.io/).

A good candidate for caching is data that is frequently accessed, but rarely modified.

## Closing Thoughts

I think that for most Web applications,
performance optimization can be boiled down to the following approaches:

- [Focus on bottlenecks first](#focus-on-bottlenecks-first)
- [Reduce the number of round trips](#reduce-the-number-of-round-trips)
- [Parallelize external calls](#parallelize-external-calls)
- [Caching](#caching-as-a-last-resort)

I didn't talk about database optimization and indexes here,
but this should also be on your mind if the database is your bottleneck.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

Let's imagine we have an existing `Repository` implementation, and we want to introduce caching to reduce the load on the database.

How can we achieve this without changing the original `Repository` implementation?

**Decorator pattern** is a structural design pattern that allows you
to introduce new behavior to an existing class, without modifying the original class in any way.

I'll show you how you can implement this with the **ASP.NET Core DI** container.

## How To Implement The Decorator Pattern

We'll start with an existing `MemberRepository` implementation that implements the `IMemberRepository` interface.

It has only one method, which loads the `Member` from the database.

Here's what the implementation looks like:

```csharp
public interface IMemberRepository
{
    Member GetById(int id);
}

public class MemberRepository : IMemberRepository
{
    private readonly DatabaseContext _dbContext;

    public MemberRepository(DatabaseContext dbContext)
    {
        _dbContext = dbContext;
    }

    public Member GetById(int id)
    {
        return _dbContext
            .Set<Member>()
            .First(member => member.Id == id);
    }
}
```

We want to introduce caching to the `MemberRepository` implementation without modifying the existing class.

To achieve this, we can use the **Decorator pattern** and create a wrapper around our `MemberRepository` implementation.

We can create a `CachingMemberRepository` that will have a dependency on `IMemberRepository`.

```csharp
public class CachingMemberRepository : IMemberRepository
{
    private readonly IMemberRepository _repository;
    private readonly IMemoryCache _cache;

    public CachingMemberRepository(
        IMemberRepository repository,
        IMemoryCache cache)
    {
        _repository = repository;
        _cache = cache;
    }

    public Member GetById(int id)
    {
        string key = $"members-{id}";

        return _cache.GetOrCreate(
            key,
            entry => {
                entry.SetAbsouluteExpiration(
                    TimeSpan.FromMinutes(5));

                return _repository.GetById(id);
            });
    }
}
```

Now I'm going to show you the power of **ASP.NET Core DI**.

We will configure the `IMemberRepository` to resolve an instance of `CachingMemberRepository`,
while it will receive the `MemberRepository` instance as its dependency.

## Configuring The Decorator In ASP .NET Core DI

For the DI container to be able to resolve `IMemberRepository` as `CachingMemberRepository`,
we need to manually configure the service.

We can use the overload that exposes a service provider,
that we will use to resolve the services required to construct a `MemberRepository`.

Here's what the configuration would look like:

```csharp
services.AddScoped<IMemberRepository>(provider => {
    var context = provider.GetService<DatabaseContext>();
    var cache = provider.GetService<IMemoryCache>();

    return new CachingRepository(
         new MemberRepository(context),
         cache);
});
```

Now you can inject the `IMemberRepository`, and the DI will be able to resolve an instance of `CachingMemberRepository`.

## Configuring The Decorator With Scrutor

If the previous approach seems _cumbersome_ to you and like a lot of manual work - that's because it is.

However, there is a simpler way to achieve the same behavior.

We can use the **[Scrutor](https://github.com/khellang/Scrutor)** library to register the decorator:

```csharp
services.AddScoped<IMemberRepository, MemberRepository>();

services.Decorate<IMemberRepository, CachingMemberRepository>();
```

**[Scrutor](https://github.com/khellang/Scrutor)** exposes the `Decorate` method.
The call to `Decorate` will register the `CachingMemberRepository` while ensuring
that it receives the expected `MemberRepository` instance as its dependency.

I think this approach is much simpler, and it's what I use in my projects.

---



In this newsletter, we'll be covering three ways to create middleware in **ASP.NET Core** applications.

**Middleware** allows us to introduce additional logic before or after executing an HTTP request.

You are already using many of the built-in middleware available in the framework.

I'm going to show you three approaches to how you can define custom middleware:

- [With Request Delegates](#adding-middleware-with-request-delegates)
- [By Convention](#adding-middleware-by-convention)
- [Factory-Based](#adding-factory-based-middleware)

Let's go over each of them and see how we can implement them in code.

## Adding Middleware With Request Delegates

The first approach to defining a middleware is by writing a **Request Delegate**.

You can do that by calling the `Use` method on the `WebApplication` instance
and providing a lambda method with two arguments.
The first argument is the `HttpContext` and the second argument is
the actual next request delegate in the pipeline `RequestDelegate`.

Here's what this would look like:

```csharp
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.Use(async (context, next) =>
{
    // Add code before request.

    await next(context);

    // Add code after request.
});
```

By awaiting the `next` delegate, you are continuing the request pipeline execution.
You can _short-circuit_ the pipeline by not invoking the `next` delegate.

This overload of the `Use` method is the one suggested by **Microsoft**.

## Adding Middleware By Convention

The second approach requires us to create a class that will represent our middleware.
We have to follow the convention when creating this class so that we can use it as middleware in our application.

I'm first going to show you what this class looks like, and then explain what is the convention we are following here.

Here's how this class would look like:

```csharp
public class ConventionMiddleware(
    RequestDelegate next,
    ILogger<ConventionMiddleware> logger)
{
    public async Task InvokeAsync(HttpContext context)
    {
        logger.LogInformation("Before request");

        await next(context);

        logger.LogInformation("After request");
    }
}
```

The convention we are following has a few rules:

- We need to inject a `RequestDelegate` in the constructor
- We need to define an `InvokeAsync` method with an `HttpContext` argument
- We need to invoke the `RequestDelegate` and pass it the `HttpContext` instance

There's one more thing that's required, and that is to tell our application to use this middleware.

We can do that by calling the `UseMiddleware` method:

```csharp
app.UseMiddleware<ConventionMiddleware>();
```

And with this, we have a functioning middleware.

## Adding Factory-Based Middleware

The third and last approach requires us to also create a class that will represent our middleware.

However, this time we're going to implement the `IMiddleware` interface.
This interface has only one method - `InvokeAsync`.

Here's what this class would like:

```csharp
public class FactoryMiddleware(ILogger<FactoryMiddleware> logger) : IMiddleware
{
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        logger.LogInformation("Before request");

        await next(context);

        logger.LogInformation("After request");
    }
}
```

The `FactoryMiddleware` class will be resolved at runtime from dependency injection.

Because of this, we need to register it as a service:

```csharp
builder.Services.AddTransient<FactoryMiddleware>();
```

And like the previous example, we need to tell our application to use our factory-based middleware:

```csharp
app.UseMiddleware<FactoryMiddleware>();
```

With this, we have a functioning middleware.

## A Word On Strong Typing

I'm a big fan of **strong typing** whenever possible.
Out of the three approaches I just showed you, the one using the
`IMiddleware` interface satisfies this constraint the most.
This is also my **preferred** way to implement **middleware**.

Since we're implementing an interface, it's very easy to create
a generic solution to never forget to register your middleware.

You can use reflection to scan for classes implementing
the `IMiddleware` interface and add them to dependency injection,
and also add them to the application by calling `UseMiddleware`.

---



**Clean Architecture** is a popular approach to structuring your application.

It's a layered architecture that splits the project into four layers:

- [Domain](#domain-layer)
- [Application](#application-layer)
- [Infrastructure](#infrastructure-layer)
- [Presentation](#presentation-layer)

Each of the layers is typically one project in your solution.

Here's a visual representation of the **Clean Architecture**:

<Image src="blogs/mnw_004/clean_architecture.png" width={1920} height={1080} />

How do we create this in our .NET solutions?

## Domain Layer

The **Domain layer** sits at the core of the **Clean Architecture**.
Here we define things like: entities, value objects, aggregates, domain events, exceptions, repository interfaces, etc.

Here is the folder structure I like to use:

```
📁 Domain
|__ 📁 DomainEvents
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Shared
|__ 📁 ValueObjects
```

You can introduce more things here if you think it's required.

One thing to note is that the **Domain layer** is not allowed to reference other projects in your solution.

## Application Layer

The **Application layer** sits right above the **Domain layer**.
It acts as an orchestrator for the **Domain layer**, containing the most important use cases in your application.

You can structure your use cases using services or using commands and queries.

I'm a big fan of the **CQRS** pattern, so I like to use the command and query approach.

Here is the folder structure I like to use:

```
📁 Application
|__ 📁 Abstractions
    |__ 📁 Data
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Behaviors
|__ 📁 Contracts
|__ 📁 Entity1
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
|__ 📁 Entity2
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
```

In the `Abstractions` folder, I define the interfaces required for the **Application layer**.
The implementations for these interfaces are in one of the upper layers.

For every entity in the **Domain layer**, I create one folder with the commands, queries, and events definitions.

## Infrastructure Layer

The **Infrastructure layer** contains implementations for external-facing services.

What would fall into this category?

- Databases - PostgreSQL, MongoDB
- Identity providers - Auth0, Keycloak
- Emails providers
- Storage services - AWS S3, Azure Blob Storage
- Message queues - Rabbit MQ

Here is the folder structure I like to use:

```
📁 Infrastructure
|__ 📁 BackgroundJobs
|__ 📁 Services
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Persistence
    |__ 📁 EntityConfigurations
    |__ 📁 Migrations
    |__ 📁 Repositories
    |__ #️⃣ ApplicationDbContext.cs
|__ 📁 ...
```

I place my `DbContext` implementation here if I'm using **EF Core**.

It's not uncommon to make the Persistence folder its project.
I frequently do this to have all database facing-code inside of one project.

## Presentation Layer

The **Presentation layer** is the entry point to our system.
Typically, you would implement this as a Web API project.

The most important part of the **Presentation layer** is the `Controllers`, which define the API endpoints in our system.

Here is the folder structure I like to use:

```
📁 Presentation
|__ 📁 Controllers
|__ 📁 Middlewares
|__ 📁 ViewModels
|__ 📁 ...
|__ #️⃣ Program.cs
```

Sometimes, I will move the **Presentation layer** away from the actual Web API project.
I do this to isolate the `Controllers` and enforce stricter constraints.
You don't have to do this if it is too complicated for you.

## Is This The Only Way?

You don't have to follow the folder structure I proposed to the T.
**Clean Architecture** is very flexible, and you can experiment with it and structure it the way you like.

Do you like more granularity? Create more specific projects.

Do you dislike a lot of projects? Separate concerns using folders.

I'm here to give you options to explore. But it's up to you to decide what's best.

---



I recently ran into an issue with **Entity Framework Core**.

The query I was running was constantly timing out.

I tried to scale up the application server, and it didn't help.

I tried to scale up the database server, and it didn't help.

So how did I solve the problem?

## What Was The Problem With This Query?

I'm working on an application in the e-commerce domain.
To be specific, it's an order management system for a kitchen cabinet manufacturer.

The table that I frequently query on is the `Orders` table.
The `Order` can have one or more `LineItems`.
A typical `Order` will contain 50 `LineItems`.
Also, `LineItems` have a table that contains the valid dimensions - `LineItemDimensions`.

This is the query I was trying to run:

```csharp
dbContext
    .Orders
    .Include(order => order.LineItems)
    .ThenInclude(lineItem => lineItem.Dimensions)
    .First(order => order.Id == orderId);
```

When EF Core converts this into SQL, this is what it will send to the database:

```sql
SELECT o.*, li.*, d.*
FROM Orders o
LEFT JOIN LineItems li ON li.OrderId = o.Id
LEFT JOIN LineItemDimensions d ON d.LineItemId = li.Id
WHERE o.Id = @orderId
ORDER BY o.Id, li.Id, d.Id;
```

In most cases, this query will execute just fine.

However, in my situation I was running into the problem of _Cartesian Explosion_.
This is mainly because of the join to the `LineItemDimensions` table.
And this is what's causing my query to fail, and time out.

So how did I solve this problem?

## Query Splitting To The Rescue

With the release of **EF Core 5.0** we got a new feature called **Query Splitting**.
This allows us to specify that a given LINQ query should be split into multiple `SQL` queries.

To use **Query Splitting**, all you need to do is call the `AsSplitQuery` method:

```csharp
dbContext
    .Orders
    .Include(order => order.LineItems)
    .ThenInclude(lineItem => lineItem.Dimensions)
    .AsSplitQuery()
    .First(order => order.Id == orderId);
```

In this case, EF Core will generate the following SQL queries:

```sql
SELECT o.*
FROM Orders o
WHERE o.Id = @orderId;

SELECT li.*
FROM LineItems li
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;

SELECT d.*
FROM LineItemDimensions d
JOIN LineItems li ON d.LineItemId = li.Id
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;
```

Notice that for each `Include` statement we have a separate `SQL` query.
The benefit here is that we are not duplicating data when fetching from the database,
as we were in the previous case.

## Turning On Query Splitting For All Queries

You can enable **Query Splitting** at the database context level.
When configuring your database context you need to call the `UseQuerySplittingBehavior` method:

```csharp
services.AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.UseQuerySplittingBehavior(
            QuerySplittingBehavior.SplitQuery)));
```

This will cause all queries that EF Core generates to be split queries.
To revert back to a single query, you need to call the `AsSingleQuery` method:

```csharp
dbContext
    .Orders
    .Include(o => o.LineItems)
    .ThenInclude(li => li.Dimensions)
    .AsSingleQuery()
    .First(o => o.Id == orderId);
```

## What You Should Know About Query Splitting

Although query splitting is an excellent addition to EF Core, there are a few things you need to be aware of.

There is no consistency guarantee for multiple SQL queries.
You may run into a problem if you have a concurrent update going through
at the same time when you query your data.
To mitigate this, you can wrap the queries inside of a transaction, but this will only introduce performance issues elsewhere.

Each query will require one network round trip. This can degrade performance if your latency to the database is high.

Now that you are armed with this knowledge, go and make your EF queries faster!

---



Today, I'm going to share some fascinating things you can do with records and anonymous types.
I will introduce you to the concept of non-destructive mutation.
And I will talk about when and why we might want to use this C# language feature.

## What Is a Record?

With **C# 9** we can use **records** that are a new reference type.
**C# 10** introduced record structs so that you can define records as value types.
Records are distinct from classes in that record types use value-based equality.

Let's see how we would define a `record`:

```csharp
public record Food(string Name, double Price);
```

This way of declaring a `record` is called a positional record.
The constructor we have defined here is called the **primary constructor**.

The `Name` and `Price` properties are init only properties.
This means they can only be set in the constructor or using a property initializer.

Since our properties are init only, is there any way to change their value?

## Non-Destructive Mutation Using The With Expression

We said we can't modify the properties of our `record`, because the properties are init only.
However, we can use the `with` expression (introduced in **C# 9**) to create a new instance
of our record with modified values.

Let's see how we would use the `with` expression:

```csharp
var banana = new Food("🍌", 1.95);

var bananaOnSale = banana with
{
    Price = 0.99
};
```

It's important to highlight two things here:

- The original banana instance remains unchanged
- The `with` expression creates a new record instance with only the `Price` property modified

I mentioned Anonymous Types in the title, so let me show you something interesting you can do with them.

## Anonymous Types And Non-Destructive Mutation

Did you know that you can use the `with` expression with anonymous types?

Just a reminder that the `with` expression is available from **C# 9** and later.

Let's create an anonymous type:

```csharp
var apple = new
{
    Name = "🍎",
    Price = 1.21
};
```

This is how we can modify it using the with expression:

```csharp
var orange = apple with
{
    Name = "🍊"
};
```

And again the same rules apply:

- The original apple instance remains unchanged
- The with expression creates a new anonymous type instance with only the Name property modified

I found this feature useful in LINQ method chains.

For example, loading an anonymous type from the database where some properties have a default value.
You can then use this feature to calculate the values for these properties in memory.

---



## Wishing You a Warm Welcome

First, I want to welcome you to the first edition of **Milan's .NET Weekly** newsletter.

I hope that this newsletter can become a positive force in the .NET community.
To bring many of us together so that we can all continue learning and improving.

With that out of the way, let's get into .NET!

## The Problem With Wide LINQ

Let's consider the following LINQ expression from a code style perspective.

I call this a wide LINQ expression because it stretches horizontally across the entire screen.

```csharp
dbContext.Animals.Where(animal => animal.HasBigEars)
    .OrderBy(animal => animal.IsDangerous).Select(
        animal => (animal.Id, animal.Name)).ToList();
```

- It is difficult to read.
- It is difficult to reason about.
- It is difficult to extend or maintain.

To improve this, I created a simple rule that you can follow:

> When writing LINQ, try to go tall, not wide.

## How to Write Tall LINQ

So how do we write tall LINQ expressions?

I'm going to rewrite the previous expression, to improve it.

Try to follow the _one dot per line rule_:

```csharp
dbContext
    .Animals
    .Where(animal => animal.HasBigEars)
    .OrderBy(animal => animal.IsDangerous)
    .Select(animal => (animal.Id, animal.Name))
    .ToList();
```

Is the new version easier to read? **Yes**, very much so.

It is easier to understand what each expression does,
and how it feeds into the next one in the chain.

If you are working in a team, try to propose this as a coding standard (if it isn't one already).\
You will see that over time this will make a noticeable difference.

---


Searched: "log/search"