
<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

What happens when a **message is retried** in an event-driven system?

It happens more often than you think.

The **worst case scenario** is that the **message is processed twice**, and the **side effects** can also be applied more than once.

Do you want your bank account to be double charged?

I'll assume the answer is no, of course.

You can use the **Idempotent Consumer** pattern to solve this problem.

In this week's issue I will show you:

- How the Idempotent Consumer pattern works
- How to implement an Idempotent Consumer
- The tradeoffs you need to consider

Let's see why the **Idempotent Consumer** pattern is valuable.

## How The Idempotent Consumer Pattern Works

What's the idea behind the **Idempotent Consumer pattern**?

> An idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

We want to avoid handling the same message more than once.

This would require **Exactly-once** message delivery guarantees from our messaging system.
And this is a really hard problem to solve in distributed systems.

A looser delivery guarantee is **At-least-once**, where we are aware that retries can happen and we can receive the same message more than once.

The **Idempotent Consumer** pattern works well with **At-least-once** message delivery, and solves the **problem of duplicate messages.**

Here's what the algorithm looks like from the moment we receive a message:

1. Was the message already processed?
2. If yes, it's a duplicate and there's nothing to do
3. If not, we need to handle the message
4. We also need to store the message identifier

<Image
  src="blogs/mnw_034/idempotent_consumer_algorithm.png"
  width={400}
  height={634}
/>

We need a **unique identifier** for every **message** we receive, and a table in the database to store processed messages.

However, it's interesting how we choose the implement the message handling and storing of the processed message identifier.

You can implement the idempotent consumer as a decorator around a regular message handler.

I'll show you two implementations:

- Lazy idempotent consumer
- Eager idempotent consumer

## Lazy Idempotent Consumer

The **lazy idempotent consumer** matches the flow shown in the algorithm above.

Lazy refers to how we store the message identifer to mark it as processed.

In the happy path, we handle the message and store the message identifier.

If the message handler throws an exception, we never store the message identifier and the consumer can be executed again.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        if (_messageRepository.IsProcessed(message.Id))
        {
            return;
        }

        await _decorated.Handle(message);

        _messageRepository.Store(message.Id);
    }
}
```

## Eager Idempotent Consumer

The **eager idempotent consumer** is slightly different from the lazy implementation, but the end result is the same.

In this version, we eagerly store the message identifier in the database and then continue to handle the message.

If the handler throws an exception, we need to perform cleanup in the database and remove the eagerly stored message identifier.

Otherwise, we risk leaving the system in an inconsistent state since the message was never handled correctly.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        try
        {
            if (_messageRepository.IsProcessed(message.Id))
            {
                return;
            }

            _messageRepository.Store(message.Id);

            await _decorated.Handle(message);
        }
        catch (Exception e)
        {
            _messageRepository.Remove(message.Id);

            throw;
        }
    }
}
```

## In Summary

Idempotency is an interesting problem to solve in a software system.

Some operations are **naturally idempotent**, and we don't need the overhead of the **Idempotent Consumer** pattern.

However, for those operations that aren't naturally idempotent, the **Idempotent Consumer** is a great solution.

The high-level algorithm is simple, and you can take two approaches in the implementation:

- Lazy storing of message identifiers
- Eager storing of message identifiers

I prefer to use the **lazy approach**, and only **store the message identifier** in the database when the **handler completes successfully.**

It's easier to reason about and there is one less call to the database.

Thanks for reading.

Hope that was helpful.

---



<Sponsorships>

[**EF Core is too slow?**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter)
Discover how you can easily insert 14x faster (reducing saving time by 94%).
Boost your performance with our method integrated within EF Core: Bulk Insert, update, delete, and merge.
Join 5,000+ satisfied customers who have trusted our library since 2014.
[**Click here to learn more**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter).

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

**EF Core** is getting many new and exciting features in the upcoming version.

**EF7** introduced support for returning **scalar types** using **SQL** queries.

And now we're getting support for **querying unmapped types** with **raw SQL queries** in **EF8.**

This is exactly what **Dapper** offers out of the box, and it's good to see **EF Core** catching up.

In this week's newsletter, I'm going to cover how to use **EF Core** for:

- [Raw SQL queries](#ef-core-and-sql-queries)
- [Composing SQL queries with LINQ](#composing-sql-queries-with-linq)
- [Executing data modifications with SQL](#sql-queries-for-data-modifications)

Let's dive in!

## EF Core And SQL Queries

**EF7** added support for **raw SQL queries** returning scalar types.
**EF8** is taking this a step further with raw SQL queries that can return any mappable type, without having to include it in the **EF model**.

You can query unmapped types with the `SqlQuery` and `SqlQueryRaw` methods.

The `SqlQuery` method uses string interpolation to parameterize the query, protecting against SQL injection attacks.

Here's an example query returning an `OrderSummary` list:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .ToListAsync();
```

This will be the **SQL** sent to the database:

```sql
SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
```

The type used for the query result can have a parameterized constructor.
The property names don't need to match the column names in the database, but they do have to match the names of the values in the result set.

You can also execute raw SQL queries and return results with:

- Views
- Functions
- Stored procedures

## Composing SQL Queries With LINQ

An interesting thing about `SqlQuery` is that it returns `IQueryable`, which can be further composed with **LINQ.**

You can add a `Where` statement after calling `SqlQuery`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>("SELECT * FROM OrderSummaries AS o")
    .Where(o => o.CreatedOn >= startDate)
    .ToListAsync();
```

However, the generated **SQL** isn't optimal:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o
) AS s
WHERE s.CreatedOn >= @p0
```

Another possibility is to combine an `OrderBy` statement with `Skip` and `Take`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .OrderBy(o => o.Id)
    .Skip(10)
    .Take(5)
    .ToListAsync();
```

This would be the generated **SQL** for the previous query:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
) AS s
ORDER BY s.Id
OFFSET @p1 ROWS FETCH NEXT @p2 ROWS ONLY
```

In case you're wondering, the performance is similar to **LINQ** queries using `Select` projections.

I ran some benchmarks, and didn't notice any significant performance improvement.

This feature will be very useful if you're more comfortable with writing **SQL** or you want to fetch data from views, functions, and stored procedures.

## SQL Queries For Data Modifications

If you want to modify data in the database with **SQL**, you will typically write a query that doesn't return a result.

The SQL query can be an `UPDATE` or `DELETE` statement, or even a stored procedure call.

You can use the `ExecuteSql` method to execute this type of query with **EF Core**:

```csharp
var startDate = new DateOnly(2023, 1, 1);

dbContext.Database.ExecuteSql(
    $"UPDATE Orders SET Status = 5 WHERE CreatedOn >= {startDate}");
```

`ExecuteSql` also protects from SQL injection by parameterizing arguments, just like `SqlQuery`.

With **EF7** you can write the above query with **LINQ** and the `ExecuteUpdate` method.
There's also the `ExecuteDelete` method for deleting records.

## In Summary

**EF7** introduced support for raw SQL queries returning **scalar** values.

**EF8** will add support for **raw SQL queries** returning **unmapped types** with `SqlQuery` and `SqlQueryRaw`.

I like the direction that **EF** is going, introducing more flexibility for querying the database.

The performance isn't as good as **Dapper**, unfortunately.
But it's close enough that network costs will play the bigger factor.

I will probably be using only **EF** moving forward since it covers more use cases.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

Rate limiting is a technique to limit the number of requests to a server or an API.

A limit is introduced within a given time period to prevent server overload and protect against abuse.

In ASP.NET Core 7, we have a built-in rate limiter middleware that's easy to integrate into your API.

We're going to cover four rate limiter algorithms:

- [Fixed window](#fixed-window-limiter)
- [Sliding window](#sliding-window-limiter)
- [Token bucket](#token-bucket-limiter)
- [Concurrency](#concurrency-limiter)

Let's see how we can work with rate limiting.

## What Is Rate Limiting?

Rate limiting is about restricting the number of requests to an API, usually within a specific time window or based on other criteria.

This is practical for a few reasons:

- Prevents overloading of servers or applications
- Improves security and guards against DDoS attacks
- Reduces costs by preventing unnecessary resource usage

In a multi-tenant application, each unique user can have a limitation on the number of API requests.

## Configuring Rate Limiting

ASP.NET Core 7 introduced built-in rate limiting middleware in the `Microsoft.AspNetCore.RateLimiting` namespace.

To add rate limiting to your application, you first need to register the rate limiting services:

```csharp
builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // We'll talk about adding specific rate limiting policies later.
});
```

I suggest updating the `RejectionStatusCode` to `429 (Too Many Requests)` because it's more correct.
The default value is `503 (Service Unavailable)`.

And you also have to apply the `RateLimitingMiddleware`:

```csharp
app.UseRateLimiter();
```

That's everything you'll need.

Let's see the rate limiting algorithms we can use.

## Fixed Window Limiter

The `AddFixedWindowLimiter` method configures a fixed window limiter.

The `Window` value determines the time window.

When a time window expires, a new one starts, and the request limit is reset.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Sliding Window Limiter

The sliding window algorithm is similar to the fixed window, but it introduces segments in a window.

Here's how it works:

- Each time window is divided into multiple segments
- The window slides one segment each segment interval
- The segment interval is (window_time)/(segments_per_window)
- When a segment expires, the requests taken in that segment are added to the current segment

The `AddSlidingWindowLimiter` method configures a sliding window limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddSlidingWindowLimiter("sliding", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.SegmentsPerWindow = 2;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Token Bucket Limiter

The token bucket algorithm is similar to the sliding window, but instead of adding back the requests from the expired segment,
a fixed number of tokens are added after each replenishment period.

The total number of tokens can never exceed the token limit.

The `AddTokenBucketLimiter` method configures a token bucket limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 100;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
        options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        options.TokensPerPeriod = 20;
        options.AutoReplenishment = true;
    });
});
```

When `AutoReplenishment` is `true`, an internal timer will execute every `ReplenishmentPeriod` and replenish the tokens.

## Concurrency Limiter

The concurrency limiter is the most straightforward algorithm, and it just limits the number of concurrent requests.

The `AddConcurrencyLimiter` method configures a concurrency limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddConcurrencyLimiter("concurrency", options =>
    {
        options.PermitLimit = 10;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

There's no time component involved in this case. The only parameter is the number of concurrent requests.

## Using Rate Limiting In Your API

Now that we have configured our rate limiting policies, let's see how we can use them in our API.

There are slight differences between controllers and minimal API endpoints, so I'll cover them in separate examples.

**Controllers**

To add rate limiting on a controller we use the `EnableRateLimiting` and `DisableRateLimiting` attributes.

`EnableRateLimiting` can be applied on the controller or on the individual endpoints.

```csharp
[EnableRateLimiting("fixed")]
public class TransactionsController
{
    private readonly ISender _sender;

    public TransactionsController(ISender sender)
    {
        _sender = sender;
    }

    [EnableRateLimiting("sliding")]
    public async Task<IActionResult> GetTransactions()
    {
        return Ok(await _sender.Send(new GetTransactionsQuery()));
    }

    [DisableRateLimiting]
    public async Task<IActionResult> GetTransactionById(int id)
    {
        return Ok(await _sender.Send(new GetTransactionByIdQuery(id)));
    }
}
```

In the previous example:

- All endpoints in the `TransactionsController` will use a **fixed window** policy
- The `GetTransactions` endpoint will use a **sliding window** policy
- The `GetTransactionById` endpoint won't have any rate limiting applied

**Minimal APIs**

In a Minimal API endpoint you can configure the rate limit policy by calling `RequireRateLimiting` and specifying the policy name.

We're using the **token bucket** policy in this example.

```csharp
app.MapGet("/transactions", async (ISender sender) =>
{
    return Results.Ok(await sender.Send(new GetTransactionsQuery()));
})
.RequireRateLimiting("token");
```

## Closing Thoughts

It's great that we can quickly introduce **rate limiting** in ASP.NET Core.

You can choose from one of the existing rate limiter algorithms:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here are some resources if you want to learn more about rate limiting:

- [Rate Limiting pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern)
- [Announcing Rate Limiting for .NET](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/)

I'm excited to try out rate limiting in my projects.

That's all for today.

Have an awesome Saturday!

---



<Sponsorships>

This week's issue is sponsored by [**QueueExplorer.**](https://www.cogin.com/mq/)
QueueExplorer is an explorer-like [**management console for RabbitMQ, Azure Service Bus, MSMQ, and ActiveMQ.**](https://www.cogin.com/mq/)
With QueueExplorer you can see what happens in your queues, edit and fix problematic messages, extract business data from JSON or XML payloads,
or send thousands of messages for stress testing.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

Designing long-lived processes in a distributed environment is an interesting engineering challenge.

And a well known pattern for solving this problem is a [**Saga**](https://microservices.io/patterns/data/saga.html).

A **Saga** is a sequence of local transactions, where each local transaction updates the **Saga** state and publishes a message triggering the next step in the **Saga**.

Sagas come in two forms:

- **Orchestrated**
- **Choreographed**

With an orchestrated Saga, there's a central component responsible for orchestrating the individual steps.

In a choreographed Saga, processes work independently but coordinate with each other using events.

In this week's issue, I'll show you how to create an **orchestrated Saga** using the **Rebus** library with **RabbitMQ** for message transport.

Let's dive in.

## Rebus Configuration

[Rebus](https://github.com/rebus-org/Rebus) is a free .NET “service bus”, and it's practical for implementing asynchronous messaging-based
communication between the components of an application.

Let's install the following libraries:

- `Rebus.ServiceProvider` for managing the **Rebus** instance
- `Rebus.RabbitMq` for **RabbitMQ** message transport
- `Rebus.SqlServer` for **SQL Server** state persistence

```powershell
Install-Package Rebus.ServiceProvider -Version 8.4.0
Install-Package Rebus.RabbitMq -Version 8.0.0
Install-Package Rebus.SqlServer -Version 7.3.1
```

Inside of your **ASP.NET Core** application you will need the following configuration.

```csharp
services.AddRebus(
    rebus => rebus
        .Routing(r =>
            r.TypeBased().MapAssemblyOf<Program>("newsletter-queue"))
        .Transport(t =>
            t.UseRabbitMq(
                configuration.GetConnectionString("RabbitMq"),
                inputQueueName: "newsletter-queue"))
        .Sagas(s =>
            s.StoreInSqlServer(
                configuration.GetConnectionString("SqlServer"),
                dataTableName: "Sagas",
                indexTableName: "SagaIndexes"))
        .Timeouts(t =>
            t.StoreInSqlServer(
                builder.Configuration.GetConnectionString("SqlServer"),
                tableName: "Timeouts"))
);

services.AutoRegisterHandlersFromAssemblyOf<Program>();
```

Unpacking the individual configuration steps:

- `Routing` - Configures messages to be routed by their type
- `Transport` - Configures the message transport mechanism
- `Sagas` - Configures the Saga persistence store
- `Timeouts` - Configures the timeouts persistence store

You also need to specify the queue name for sending and receiving messages.

`AutoRegisterHandlersFromAssemblyOf` will scan the specified assembly and automatically register the respective message handlers.

## Creating The Saga With Rebus

We're going to create a **Saga** for a newsletter onboarding process.

When a user subscribes to the newsletter we want to:

- Send a welcome email immediately
- Send a follow-up email after 7 days

The first step in creating the **Saga** is defining the data model by implementing `ISagaData`.
We'll keep it simple and store the `Email` for **correlation**, and add two flags for the distinct steps in our **Saga**.

```csharp
public class NewsletterOnboardingSagaData : ISagaData
{
    public Guid Id { get; set; }
    public int Revision { get; set; }

    public string Email { get; set; }

    public bool WelcomeEmailSent { get; set; }

    public bool FollowUpEmailSent { get; set; }
}
```

Now we can define the `NewsletterOnboardingSaga` class by inheriting from the `Saga` class and implementing the `CorrelateMessages` method.

It's a best practice to use a unique value for correlation.
In our case this will be the `Email`.

You also configure how the `Saga` starts with `IAmInitiatedBy`, and the individual messages the `Saga` handles with `IHandleMessages`.

```csharp
public class NewsletterOnboardingSaga :
    Saga<NewsletterOnboardingSagaData>,
    IAmInitiatedBy<SubscribeToNewsletter>,
    IHandleMessages<WelcomeEmailSent>,
    IHandleMessages<FollowUpEmailSent>
{
    private readonly IBus _bus;

    public NewsletterOnboardingSaga(IBus bus)
    {
        _bus = bus;
    }

    protected override void CorrelateMessages(
        ICorrelationConfig<NewsletterOnboardingSagaData> config)
    {
        config.Correlate<SubscribeToNewsletter>(m => m.Email, d => d.Email);

        config.Correlate<WelcomeEmailSent>(m => m.Email, d => d.Email);

        config.Correlate<FollowUpEmailSent>(m => m.Email, d => d.Email);
    }

    /* Handlers omitted for brevity */
}
```

## Message Types And Naming Conventions

There are two types of messages you send in a **Saga**:

- Commands
- Events

Commands instruct the receiving component what to do.\
Think: **verb, imperative.**

Events notify the Saga which process was just completed.\
Think: **what happened, past tense.**

## Saga Orchestration With Messages

The `NewsletterOnboardingSaga` starts by handling the `SubscribeToNewsletter` command, and publishes a `SendWelcomeEmail` command.

```csharp
public async Task Handle(SubscribeToNewsletter message)
{
    if (!IsNew)
    {
        return;
    }

    await _bus.Send(new SendWelcomeEmail(message.Email));
}
```

The `SendWelcomeEmail` command is handled by a different component, which publishes a `WelcomeEmailSent` event when it completes.

In the `WelcomeEmailSent` handler we update the `Saga` state and publish a **deferred message** by calling `Defer`.
Rebus will persist the `SendFollowUpEmail` command, and publish it when the **timeout expires**.

```csharp
public async Task Handle(WelcomeEmailSent message)
{
    Data.WelcomeEmailSent = true;

    await _bus.Defer(TimeSpan.FromDays(3), new SendFollowUpEmail(message.Email));
}
```

Finally, the `SendFollowUpEmail` command is handled and we publish the `FollowUpEmailSent` event.

We update the `Saga` state again, and also call `MarkAsComplete` to complete the `Saga`.

```csharp
public Task Handle(FollowUpEmailSent message)
{
    Data.FollowUpEmailSent = true;

    MarkAsComplete();

    return Task.CompletedTask;
}
```

Completing the `Saga` will delete it from the database.

## Handling Commands With Rebus

Here's how the `SendWelcomeEmail` command handler looks like.

```csharp
public class SendWelcomeEmailHandler : IHandleMessages<SendWelcomeEmail>
{
    private readonly IEmailService _emailService;
    private readonly IBus _bus;

    public SendWelcomeEmailHandler(IEmailService emailService, IBus bus)
    {
        _emailService = emailService;
        _bus = bus;
    }

    public async Task Handle(SendWelcomeEmail message)
    {
        await _emailService.SendWelcomeEmailAsync(message.Email);

        await _bus.Reply(new WelcomeEmailSent(message.Email));
    }
}
```

The important thing to highlight here is that we're using the `Reply` method to send a message back.
This will reply back to the endpoint specified as the return address on the current message.

## In Summary

**Sagas** are practical for implementing a long-lived process in a distributed system.
Each business process is represented by a local transaction, and publishes a message to trigger the next step in the **Saga**.

Although **Sagas** are very powerful, they are also _complicated to develop, maintain and debug._

We didn't cover a few important topics in this newsletter:

- Error handling
- Message retries
- Compensating transactions

I think you'll have some fun researching these on your own.

Take a look at the [**source code for the example used in this newsletter**](https://github.com/m-jovanovic/newsletter-orchestrated-saga)
if you want to learn more about implementing Sagas.

If you have **Docker** installed, you should be able to run it without a problem and try it out.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

This week's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus Pro is the perfect one-up for serious Rebus users - use Fleet Manager to get Slack alerts when something fails,
and [**retry dead-lettered messages with a click of the mouse.**](https://pro.rebus.fm)

And by [**Uno Platform**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted).
Uno Platfrom is a free and open-source development platform that allows you to build single codebase applications for Windows, iOS, Android, WebAssembly,
macOS, and Linux with C# and XAML.
[**Easily create applications that look and feel native**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted)
to each platform without having to rewrite your codebase from scratch.

</Sponsorships>

**MediatR** is a popular library with a simple **mediator pattern** implementation in .NET.

Here's a definiton taken from MediatR's GitHub: **"In-process messaging with no dependencies."**

With the rise in popularity of the **CQRS pattern**, MediatR became the go-to library to implement commands and queries.

However, MediatR also has support for the **publish-subscribe** pattern using notifications.
You can publish an `INotification` instance and have multiple subscribers handle the published message.

Until recently, the handlers subscribing to an `INotification` message could only execute serially, one by one.

In this week's newsletter, I'll show you how to configure MediatR to **execute the handlers in parallel**.

Let's dive in.

## How Publish-Subscribe Works With MediatR

Before I talk about notification publishing strategies, let's see how **publish-subscribe** works with **MediatR**.

You need a class implementing the `INotification` interface:

```csharp
public record OrderCreated(Guid OrderId) : INotification;
```

Then you need a respective `INotificationHandler` implementation:

```csharp
public class OrderCreatedHandler : INotificationHandler<OrderCreated>
{
    private readonly INotificationService _notificationService;

    public OrderCreatedHandler(INotificationService notificationService)
    {
        _notificationService = notificationService;
    }

    public async Task Handle(
        OrderCreated notification,
        CancellationToken cancellationToken)
    {
        await _notificationService.SendOrderCreatedEmail(
            notification.OrderId,
            cancellationToken);
    }
}
```

And then you simply publish a message using either `IMediator` or `IPublisher`.
I prefer using the `IPublisher` because it's more expressive:

```csharp
await publisher.Publish(new OrderCreated(order.Id), cancellationToken);
```

MediatR will invoke all the respective handlers.

## Introducing Notification Publisher Strategies

Before MediatR v12, the publishing strategy would invoke each handler individually.

However, there's a new interface `INotificationPublisher` controlling how the handlers are called.

The default implementation of this interface is `ForeachAwaitPublisher`:

```csharp
public class ForeachAwaitPublisher : INotificationPublisher
{
    public async Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        foreach (var handler in handlerExecutors)
        {
            await handler
                .HandlerCallback(notification, cancellationToken)
                .ConfigureAwait(false);
        }
    }
}
```

But now you can also use the `TaskWhenAllPublisher`:

```csharp
public class TaskWhenAllPublisher : INotificationPublisher
{
    public Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        var tasks = handlerExecutors
            .Select(handler => handler.HandlerCallback(
                notification,
                cancellationToken))
            .ToArray();

        return Task.WhenAll(tasks);
    }
}
```

Here's a comparison between these two strategies.

`ForeachAwaitPublisher`:

- Invokes each handler one by one
- Fails when an exception occurs in one of the handlers

`TaskWhenAllPublisher`:

- Invokes all the handlers at the same time
- Executes all the handlers regardless of one of them throwing an exception

If you store the task returned by `TaskWhenAllPublisher` you can access the `Task.Exception` property, which will contain an `AggregateException` instance.
You can then implement more robust exception handling.

## Configuring MediatR Notification Publishing Strategy

How do we configure which `INotificationPublisher` strategy MediatR will use?

There's a new way to apply configuration options when calling the `AddMediatR` method.

You supply an `Action<MediatRServiceConfiguration>` delegate and configure the `MediatRServiceConfiguration` instance.

If you want to use the `TaskWhenAllPublisher` strategy, you can either:

- Provide a value for the `NotificationPublisher` property
- Specify the strategy type on the `NotificationPublisherType` property

```csharp
services.AddMediatR(config => {
    config.RegisterServicesFromAssemblyContaining<Program>();

    // Setting the publisher directly will make the instance a Singleton.
    config.NotificationPublisher = new TaskWhenAllPublisher();

    // Seting the publisher type will:
    // 1. Override the value set on NotificationPublisher
    // 2. Use the service lifetime from the ServiceLifetime property below
    config.NotificationPublisherType = typeof(TaskWhenAllPublisher);

    config.ServiceLifetime = ServiceLifetime.Transient;
});
```

You can also implement a custom `INotificationPublisher` instance and use your own implementation instead.

## How Is This Useful?

Being able to **run notification handlers in parallel** provides a significant **performance improvement** over the default behavior.

However, note that all handlers will use the same service scope.

If you have service instances that don't support concurrent access you may run into problems.

Unfortunately, one such service instance is the **EF Core** `DbContext`.

In any case, I think this is a great addition to the already amazing **MediatR** library.

That's all for today.

See you next week.

---



<Sponsorships>

This week's issue is sponsored by [**Localizely.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)
Localizely is the translation management platform that helps you translate texts in your app for targeting multilingual markets.
With Localizely you can collaborate with your team on translations, and reduce translation errors with better context and app screenshots.
[**Learn more here.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

And by [**QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)
QA Wolf gets you to 80% automated end-to-end test coverage in 4 months - and keeps you there.
They will build, run and maintain your test suite so that you don't have to.
[**Chat with QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)

</Sponsorships>

**Data-driven testing** is a testing method where test data is provided through some external source.
Hence it's also known as **parameterized testing**.

A popular testing library in .NET that supports parameterized testing is **xUnit**.
It uses attributes to define test methods.
The `Fact` attribute defines a simple test, and the `Theory` attribute defines a parameterized test.

In this week's newsletter, I'm going to show you four ways to write **parameterized tests** with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

And I'll discuss which approach I think is the best.

Let's dive in.

## Writing Parameterized Tests With InlineData

The simplest way to write parameterized tests with xUnit is using the `InlineData` attribute.
You provide test data by passing in values to the `InlineData` constructor.

Here's how that would look like:

```csharp
[Theory]
[InlineData("test@test.com", "test.com")]
[InlineData("milan@milanjovanovic.tech", "milanjovanovic.tech")]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}
```

In this example, we provide two `string` values to the `InlineData` attribute, which represent the `email` and `expectedDomain` parameters in the test.
We can specify the `InlineData` attribute as many times as we want, to introduce more test cases.

The downside of this approach is that it becomes very verbose when we have many test cases.
And we are limited to only using constant data for the parameters.

## Writing Parameterized Tests With MemberData

With the `MemberData` attribute we have the ability to programmatically provide the test data.
You can load the test data from a static property or member of a type.

Here's an example of using the `MemberData` attribute to load test data from a property:

```csharp
[Theory]
[MemberData(nameof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public static IEnumerable<object[]> EmailTestData => new List<object>
{
    new object[] { "test@test.com", "test.com" },
    new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" }
};
```

You specify the name of the member in the `MemberData` attribute, and it's a best practice to use the `nameof` operator
so that you can rename the property (or method) in the future without breaking your test.

The one constraint is that the property (or method) has to return `IEnumerable<object[]>`, so there is no strong typing.

## Writing Parameterized Tests With ClassData

The `ClassData` attribute allows you to extract test data into its own class.
This is helpful for organizing your test data separately from your tests, and it allows for easier reuse.
You load the test from a class the inherits from `IEnumerable<object[]>` and implements the `GetEnumerator` method.

Here's an example of using the `ClassData` attribute to load test data from a class:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : IEnumerable<object[]>
{

    public IEnumerable<object[]> GetEnumerator()
    {
        yield return new object[] { "test@test.com", "test.com" };
        yield return new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" };
    }

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
};
```

Unfortunately, this approach is complicated because you have to implement the `IEnumerable` interface.

It almost defeats the purpose of separating test data from the actual tests.

And we still suffer from lack of type-safety.

Is there a better solution?

## Writing Parameterized Tests With TheoryData

Let me introduce you to `TheoryData`, which is my preferred way of providing test data for parameterized tests.
Using the `TheoryData` class you can implement a class to provide test data while having the benefit of type-safety.

Here's an example of using `TheoryData` in combination with `ClassData`:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : TheoryData<string, string>
{
    public EmailTestData()
    {
        Add("test@test.com", "test.com");
        Add("milan@milanjovanovic.tech", "milanjovanovic.tech");
    }
};
```

How does `TheoryData` work?

It's a generic class that allows us to specify the types for our parameterized test.

You just call the `Add` method in the `EmailTestData` constructor to provide test data for a single test case.
And introducing more test cases comes down to calling the `Add` method multiple times.

You can also use `TheoryData` in combination with `MemberData`, and return `TheoryData` from a property or method.

## Which Approach Should You Use?

I showed you four approaches to write parameterized tests with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

So which one should you use?

Here's my personal preference that you can follow if you want:

- `InlineData` for simple test cases
- `TheoryData` using `ClassData` for complex test cases

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

[**EF Core is too slow?**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter)
Discover how you can easily insert 14x faster (reducing saving time by 94%).
Boost your performance with our method integrated within EF Core: Bulk Insert, update, delete, and merge.
Join 5,000+ satisfied customers who have trusted our library since 2014.
[**Click here to learn more**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter).

And by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2),
the translation management platform that helps you
[**translate texts in your app for targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

</Sponsorships>

**Entity Framework Core (EF Core)** is a popular ORM in .NET that allows you to work with SQL databases.
EF Core uses a `DbContext`, which represents a session with the database and is responsible for tracking changes,
performing database operations, and managing database connections.

It's common to have only one `DbContext` for the entire application.

But what if you need to have **multiple DbContexts**?

In this week's newsletter we're going to explore:

- When you may want to use multiple DbContexts
- How to create multiple DbContexts
- What are the benefits of using multiple DbContexts

Let's dive in!

## Why Use Multiple DbContexts?

There are a few cases where using **multiple DbContexts can be useful.**

**Multiple Databases**\
Does your application need to work with multiple SQL databases?
Then you're forced to use multiple DbContexts, each one dedicated to a specific SQL database.

**Separating Concerns**\
If the application you're building has a complex domain model, you may see an improvement by separating concerns
between a few DbContexts, where each one is responsible for a specific area of the domain model.

**Modular Monolith**\
When you're building a **Modular Monolith**, using multiple DbContexts can be practical because you can configure
a different **database schema** per `DbContext`, giving you logical separation at the database level.

**Read Replicas**\
You can configure a separate `DbContext` instance to access the read replica of your database, and use that
`DbContext` for read-only queries.
You can also configure `QueryTrackingBehavior.NoTracking` on the `DbContext` level to improve query performance.

## Creating Multiple DbContexts In a Single Application

Here's how you can easily configure multiple DbContexts.
Let's say we have a `CatalogDbContext` and an `OrderDbContext` in our application.
We want to configure them using the following constrainsts:

- Both DbContexts use the **same database**
- Each DbContext has a **separate database schema**

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }
}
```

First we need to configure the `CatalogDbContext` and `OrderDbContext` with the DI container.
You can do this by calling the `AddDbContext` method and specifying which `DbContext` is being configured,
and then using the SQL provider specific method to pass the connection string.
In this case I'm connecting to SQL Server with the `UseSqlServer` method.

```csharp
using Microsoft.EntityFrameworkCore;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));
```

If you just want to use both DbContexts in the same schema, then this is all the configuration you need.
You can now inject the `DbContext` instances in your application and use them.

However, if you want to configure a different schema for each `DbContext` then you also need to override
the `OnModelCreating` method and specify the custom schema with `HasDefaultSchema`.

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("catalog");
    }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("order");
    }
}
```

Limitations with multiple DbContexts:

1. It's not possible to do a join between different `DbContext` instances, because **EF Core** doesn't know if they
   are using the same database
2. Transactions will only work if the DbContexts are using the same database. You have to create a new transaction
   and share it between the DbContexts by [calling the `UseTransaction` method](/blog/working-with-transactions-in-ef-core#using-existing-transactions-with-ef-core)

**Migrations History Table**

If you decide to use different schemas per `DbContext`, you will be unpleasantly surprised to learn
that the default schema doesn't apply to the migrations history table.

You need to configure this by calling the `MigrationsHistoryTable` method and specifying the table name
and schema where the migrations history for that context will live.
I used the `HistoryRepository.DefaultTableName` constant in this example, but you can specify a custom
table name if you want to.

```csharp
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Migrations;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "catalog")));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "order")));
```

## Benefits Of Using Multiple DbContexts

Using multiple DbContexts can offer several benefits to your application:

- Separation of concerns
- Better performance
- More control & security

Each DbContext can be responsible for a specific subset of the application's data, which can help organize the code
and make it more modular.

When you separate data access into multiple DbContexts, the application can reduce the risk of contention and improve
concurrency, and this can improve performance.

And if you're using multiple DbContexts, you can configure more granular access control to improve application security.
You can also optimize performance and resource usage.

## In Summary

Using **multiple EF Core DbContexts** in a single application is straightforward and has many benefits.

For ready-heavy applications you can configure a separate `DbContext` to turn off query tracking by default
and get improved performance.

Also, using multiple DbContexts is practical if you're building a **Modular monolith**.
You can configure the DbContexts to be in **separate database schemas**, giving you logical separation at the database level.

That's all for today.

See you next week.

---



<Sponsorships>

Today's issue is sponsored by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website),
the translation management platform that helps you translate texts in your app for
[**targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website)

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly)
on new or existing SQL Server, enabling teams to ship apps faster.

</Sponsorships>

Although **C#** is an **object-oriented programming** language, it received many new functional features in recent versions.

To mention just a few of these features:

- Pattern matching
- Switch expressions
- Records

You're probably already doing **functional programming** without even knowing it.

Do you use LINQ?
If you do, then you're doing functional programming.
Because LINQ is a functional .NET library.

In today's issue, I will show you how to **refactor** some **imperative code** with **functional programming**.

Let's dive in.

## Benefits Of Functional Programming

Before we take a look at some code, let's see what are the **benefits** of using **functional programming**:

- Emphasis on immutability
- Emphasis on function purity
- Code is easier to reason about
- Less prone to bugs and errors
- Ability to compose functions and create higher-order functions
- Easier to test and debug

From my experience, I think functional programming is more enjoyable once you get used to it.
Starting out, it feels strange because your old object-oriented programming habits will kick in.
But after a while, **functional programming** feels easier to work with than imperative code.

## Starting With Imperative Code

**Imperative programming** is the most basic programming approach.
We describe a step-by-step process to execute a program.
It's easier for beginners to reason with imperative code by following along with the steps in the process.

Here's an example of an `EmailValidator` class written with imperative code:

```csharp
public class EmailValidator
{
    private const int MaxLength = 255;

    public (bool IsValid, string? Error) Validate(string email)
    {
        if (string.IsNullOrEmpty(email))
        {
            return (false, "Email is empty");
        }

        if (email.Length > MaxLength)
        {
            return (false, "Email is too long");
        }

        if (email.Split('@').Length != 2)
        {
            return (false, "Email format is invalid");
        }

        if (Uri.CheckHostName(email.Split('@')[1]) == UriHostNameType.Unknown)
        {
            return (false, "Email domain is invalid");
        }

        return (true, null);
    }
}
```

You can clearly see the distinct steps:

- Check if email is null or empty
- Check if email is not too long
- Check if email format is valid
- Check if email domain is valid

Let's see how we can refactor this using **functional programming**.

## Applying Functional Programming

The basic building block in **functional programming** is - **a function**.
And programs are written by composing function calls.
There are a few other things you need to keep in mind, like keeping your functions pure.
A function is pure if it always returns the same output for the same input.

We can capture each step from the imperative version of `EmailValidator` with a `Func` delegate.
To also capture the respective error message together with the validation check, we can use a tuple.
And since we know all of our validation steps, we can create an array of `Func` delegates to store all of the individual **functions**.

```csharp
public class EmailValidator
{
    const int MaxLength = 255;

    static readonly Func<string, (bool IsValid, string? Error)>[] _validations =
    {
        email => (!string.IsNullOrEmpty(email), "Email is empty"),
        email => (email.Length <= MaxLength, "Email is too long"),
        email => (email.Split('@').Length == 2, "Email format is invalid"),
        email => (
            Uri.CheckHostName(email.Split('@')[1]) != UriHostNameType.Unknown,
            "Email domain is invalid")
    };

    static readonly (bool IsValid, string? Error) _successResult = (true, null);

    public (bool IsValid, string? Error) Validate(string email)
    {
        var validationResult = _validations
            .Select(func => func(email))
            .FirstOrDefault(func => !func.IsValid);

        return validationResult is { IsValid: false, Error: { Length: >0 } } ?
            validationResult : _successResult;
    }
}
```

Notice that this allows us to do all sorts of interesting things with the `_validations` array.
How hard would it be to modify this function to return _all of the errors_ instead of just the first one?

If you're thinking we can use LINQ's `Select` method somehow, you're thinking in the right direction.

## Further Reading

We only scratched the surface of what functional programming is, and what you can do with it.
If you want to learn more, here are some learning materials:

- [Functional Programming in C#, by Enrico Buonanno](https://www.manning.com/books/functional-programming-in-c-sharp)
- [Functional Programming With C# Using Railway-Oriented Programming](https://youtu.be/dDasAmowFts)
- [How Function Composition Can Make Your Code Better](https://youtu.be/zuy2j8vxgYc)
- [Make Your ASP.NET Core API Controllers Incredibly Simple With Functional Programming](https://youtu.be/AVA2mKG4WOc)

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

[**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website) is a translation management platform that
helps you translate texts in your app for targeting multilingual markets.

</Sponsorships>

Working with **Microservices**, or any distributed system for that matter, is difficult.
In a distributed system many things can go wrong, and there are even research papers about this.
If you want to explore this topic further, I suggest that you read about the [fallacies of distributed computing](https://www.se.rit.edu/~se442/doc/fallacies.pdf).

Reducing the surface area for things to go wrong should be one of your goals, as an engineer.
In this week's newsletter, we'll try to achieve exactly that using the **Outbox pattern**.

How can you implement reliable communication between components in a distributed system?

The **Outbox pattern** is an elegant solution to this problem, allowing you to achieve transactional
guarantees in a single service and at-least-once message delivery to external systems.

Let's see how the **Outbox pattern** solves this and how can we implement it.

## What Problem Does The Outbox Pattern Solve?

To understand what problem the **Outbox pattern** solves, first we need a problem, of course.

Here's an example of a user registration flow.
There are a few things going on here:

- Saving the `User` to the database
- Sending a welcome email to the `User`
- Publishing a `UserRegisteredEvent` to a message bus

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    await _unitOfWork.SaveChangesAsync(token);

    await _emailService.SendWelcomeEmailAsync(user, token);

    await _eventBus.PublishAsync(new UserRegisteredEvent(user.Id), token);
}
```

In the happy path, all of the operations complete without any issues and all is well.

But what happens if any one of these operations fail?

- The database is unavailable, and saving the `User` fails
- The email service is down and sending an email crashes
- Publishing an event to the service bus doesn't succeed

Also, imagine a situation where you manage to save a `User` to the database,
send him a welcome email, but fail to publish the `UserRegisteredEvent` to notify other services.
How are you going to recover from this scenario?

The **Outbox pattern** allows you to **atommically** update the database and send messages to the message bus.

## Implementing The Outbox Pattern

The first step is to introduce a table in your database to represent the **Outbox**.
We can call this table `OutboxMessages`, and it's intended to store all messages that need to be delivered.
Now instead of directly making requests to external services, we simply store a message as a new row in the **Outbox** table.
The messages are usually stored as JSON in the database.

The second step is to introduce a **background process** that will periodically poll the `OutboxMessages` table.
If the worker process finds a row with an unprocessed message, it's going to publish that message and mark it as sent.
If publishing the message fails for some reason, the work process can **retry** in the next execution.

Notice that with retries, you now have **at-least-once message delivery** implemented.
The message will be published exactly once for the happy path, and more than one time in case or retries.

We can rewrite the `RegisterUserAsync` method from the previous example, now using an **Outbox**:

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    _outbox.Insert(new UserRegisteredEvent(user.Id));

    await _unitOfWork.SaveChangesAsync(token);
}
```

The **Outbox** is part of the same transaction as our unit of work, so we can atomically save the `User` to the database
and also persist the `OutboxMessage`.
If saving to the database fails, the entire transaction is rolled back and no messages are sent to the message bus.

And since we now moved the publishing of the `UserRegisteredEvent` to the worker process, we need to add a handler
so that we can send the welcome email to the user.
Here's an example of that in the `SendWelcomeEmailHandler` class:

```csharp
public class SendWelcomeEmailHandler : IHandle<UserRegisteredEvent>
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailService _emailService;

    public SendWelcomeEmailHandler(
        IUserRepository userRepository,
        IEmailService emailService)
    {
        _userRepository = userRepository;
        _emailService = emailService;
    }

    public async Task Handle(UserRegisteredEvent message)
    {
        var user = await _userRepository.GetByIdAsync(message.UserId);

        await _emailService.SendWelcomeEmailAsync(user);
    }
}
```

## Architecture Diagram With Outbox

Here's a high level overview of the system architecture with the **Outbox** introduced to the system.
You can see the `Outbox` table in the database.
What changes now is that you store messages to the `Outbox` table in the same transaction along with your entities.

<Image src="blogs/mnw_026/outbox.png" width={1801} height={989} />

## Further Reading

After reading this newsletter you should have a pretty good understanding of what the **Outbox pattern** is
and what problems it solves.
If you need to implement reliable messaging in a distributed system, it's a great solution for your problem.

What's missing is more details about how to implement the **Outbox pattern**, so here are a few videos you can watch:

- [How to use the Domain Events pattern](https://youtu.be/BimfDeDV4yU)
- [How to implement the Outbox pattern](https://youtu.be/XALvnX7MPeo)
- [How to add retries to the Outbox with Polly](https://youtu.be/xajVttkZntU)

Thanks for reading, and have an amazing Saturday.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

**Structured logging** is a practice where you apply the same message format to all of your
application logs.
The end result is that all your logs will have a similar structure, allowing them to be
easily searched and analyzed.

[Serilog](https://serilog.net/) is a popular logging library in .NET, packed with many features.
It provides logging to files, logging to the console, and elsewhere.

However, **Serilog** is unique because it comes with support for **structured logging** out of the box.

Let's see how we can install **Serilog** and configure it an **ASP.NET Core** application.

## Installing Serilog

To install **Serilog** in **ASP.NET Core** you can add the following NuGet package:

```powershell
Install-Package Serilog.AspNetCore
```

This NuGet packages comes with a simple API to integrate **Serilog** into your application.
You can call the `UseSerilog` method on the `HostBuilder` instance to provide a lambda
method to configure **Serilog**.

I think the most flexible way to configure Serilog is through application settings,
which is achieved by calling `ReadFrom.Configuration()`.

You can also call the `UseSerilogRequestLogging()` method to introduce automatic HTTP request logging
in your API.

```csharp
var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((context, configuration) =>
    configuration.ReadFrom.Configuration(context.Configuration));

var app = builder.Build();

app.UseSerilogRequestLogging();

app.Run();
```

The next question is how do you provide the actual configuration values to **Serilog**?

## Configuring Serilog With `appsettings.json`

You need to add a `Serilog` section in your `appsettings.json` file.

Here you can configure, among other things:

- Which **sinks** to use with **Serilog**
- Override default and minimum log levels
- Configure file logging arguments

In this example, we're adding the `Console` and `File` sinks to **Serilog**.
And we're adding some additional configuration for the `File` sink in the `Serilog.WriteTo` configuration section.
We can configure the output path for the log files, the naming format, which formatter to use for the logs and so on.

```json
"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning"
    }
  },
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "/logs/log-.txt",
        "rollingInterval": "Day",
        "rollOnFileSizeLimit": true,
        "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
      }
    }
  ],
  "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ]
}
```

You can get a more detailed overview of what's supported with the `Serilog.Configuration` library in the
[documentation](https://github.com/serilog/serilog-settings-configuration).

## Using Serilog In ASP.NET Core

We managed to successfully install and configure **Serilog**.
But how do we actually use it?

**Serilog** integrates with the `ILogger` interaface coming from the `Microsoft.Extensions.Logging` namespace.
If you're already using `ILogger` for logging, everything will continue working correctly.

Here's a simple example of logging inside of a Minimal API endpoint:

```csharp
app.MapGet("/serilog-is-cool", (ILogger logger) =>
{
    logger.LogInformation("This is a log inside of the Minimal API endpoint.");

    return Results.Ok(new { Message = "success" });
});
```

You just inject an `ILogger` or `ILogger<T>` instance and Serilog will provide its own implementation at runtime.

## Structured Logging Syntax

The idea behind **structured logging** is that you can introduce additional contextual information inside
of your logs.
**Serilog** does this using a message template syntax, where you can specify named parameters and then
pass in their values separately.

Here's an example of what this message template would look like.
You specify parameters inside of curly bracers and provide a name, for example `{NamedParameter}`.
The value provided for the parameter will be serialized as a property inside of the corresponding
structured log.

```csharp
var book = new { Author = "Domain-Driven Design", Title = "Eric Evans" };
var orderNumber = 1;

log.LogInformation(
    "Processing book {@Book}, order number = {@OrderNumber}",
    book,
    orderNumber);
```

There are a few things to unpack here:

- `{@Book}` parameter which accepts an object
- `{OrderNumber}` parameter which accepts a scalar value

The `@` operator in front of `Book` tells Serilog to serialize the object passed in, instead of converting
it using `ToString()`.

## Benefits Of Structured Logging

Lastly, I want to highlight what are some of the benefits of **structured logging** and why you should be using it.

As I said at the beginning, the main idea with **structured logging** is that all log message follow the same
structure.
This structure can be a JSON document for example, or a row in a relational table.
Since structured logs are in a machine-readable format, it's much easier to search through them for
specific information.

When an error occurs, structured logs can provide more context and details about the error, making it easier to
identify the root cause and fix the problem.

It's very easy to start doing structured logging with **Serilog**, and I hope you'll give it a try.

See you next week, and have an excellent Saturday.

---


Previous issues of The .NET Weekly