
<Sponsorships>

Stop coding common elements!
[**Syncfusion**](https://www.syncfusion.com/?utm_source=milanjovanovicnewsletter&utm_medium=text&utm_campaign=esee_milanjovanovic_newsletter)
offers a developer's dream-over 1,800 prebuilt UI components for web, mobile, and desktop platforms.
Focus on your app's unique features, not repetitive UI tasks.
A free community license is also available for individual developers and startups.
[**Try Syncfusion for free today**](https://www.syncfusion.com/?utm_source=milanjovanovicnewsletter&utm_medium=text&utm_campaign=esee_milanjovanovic_newsletter).

Collaborate directly with your external partners in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/)
to help your partners consume your team's APIs and work together on API projects.
[**Check out Postman's Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/),
now available for Professional and Enterprise plans in Postman v11.

</Sponsorships>

If you were to glance at the folder structure of your system, could you tell what the system is about?
And here's a more interesting question.
Could a new developer on your team easily understand what the system does based on the folder structure?

Your architecture should communicate what problems it solves.
Organizing your system around use cases leads to a structure aligned with the business domain.
This approach is called **screaming architecture**.

[Screaming architecture](https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html) is a term coined by Robert Martin (Uncle Bob).
He argues that a software system's structure should communicate what the system is about.
He draws a parallel between looking at a blueprint for a building, where you can tell the purpose of the building based on the blueprint.

In this article, I want to show some practical examples and discuss the benefits of screaming architecture.

## A Use Case Driven Approach

A use case represents a specific interaction or task that a user wants to achieve within your system.
It encapsulates the business logic required to fulfill that task.
A use case is a high-level description of a user's goal.
For example, "reserving an apartment" or "purchasing a ticket".
It focuses on the _what_ of the system's behavior, not the _how_.

When you look at the folder structure and source code files of your system:

- Do they scream: Apartment Booking System or Ticketing System?
- Or do they scream ASP.NET Core?

Here's an example of a folder structure organized around technical concerns:

```powershell
📁 Api/
|__ 📁 Controllers
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Services
    |__ #️⃣ ApartmentService.cs
    |__ #️⃣ BookingService.cs
    |__ ...
|__ 📁 Models
```

Somewhere inside these folders, we'll find concrete classes that contain the system's behavior.
You'll notice that the cohesion with this folder structure is low.

How does screaming architecture help?

A use case driven approach will place the system's use cases as the top-level concept.
I also like to group related use cases into a top-level feature folder.
Inside a use case folder, we may find technical concepts required to implement it.

[**Vertical slice architecture**](vertical-slice-architecture) also approaches this from a similar perspective.

```powershell
📁 Api/
|__ 📁 Apartments
    |__ 📁 ReserveApartment
    |__ ...
|__ 📁 Bookings
    |__ 📁 CancelBooking
    |__ ...
|__ 📁 Payments
|__ 📁 Reviews
|__ 📁 Disputes
|__ 📁 Invoicing
```

The use case driven folder structure helps us better understand user needs and aligns development efforts with business goals.

## Screaming Architecture Benefits

The benefits of organizing our system around use cases are:

- Improved cohesion since related use cases are close together
- High coupling for a single use case and its related use cases
- Low coupling between unrelated use cases
- Easier navigation through the solution

## Bounded Contexts and Vertical Slices

We have many techniques for discovering the high-level modules within our system.
For example, we could use [event storming](https://www.eventstorming.com/) to explore the system's use cases.
Domain exploration happens before we write a single line of code.

The next step is decomposing the larger problem domain into smaller sub-domains and later bounded contexts.
This gives us loosely coupled high-level modules that we can translate into code.

<Image
  src="blogs/mnw_104/bounded_contexts.png"
  width={1060}
  height={187}
  alt="Bounded contexts."
/>

The overarching idea here is thinking about cohesion around functionalities.
We want to organize our system so that the cohesion between the components is high.
Bounded contexts, vertical slices, and screaming architecture are complementary concepts.

Here's a screaming architecture example for this system.
Let's say the `Ticketing` module uses [**Clean Architecture**](clean-architecture-folder-structure) internally.
But we can still organize the system around feature folders and use cases.
An alternative approach could be organizing around [**vertical slices**](vertical-slice-architecture-structuring-vertical-slices), resulting in a less nested folder structure.

```powershell
📁 Modules/
|__ 📁 Attendance
    |__ ...
|__ 📁 Events
    |__ ...
|__ 📁 Ticketing
    |__ 📁 Application
        |__ 📁 Carts
            |__ 📁 AddItemToCart
            |__ 📁 ClearCart
            |__ 📁 GetCart
            |__ 📁 RemoveItemFromCart
        |__ 📁 Orders
            |__ 📁 SubmitOrder
            |__ 📁 CancelOrder
            |__ 📁 GetOrder
        |__ 📁 Payments
            |__ 📁 RefundPayment
        |__ ...
    |__ 📁 Domain
        |__ 📁 Customers
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
    |__ 📁 infrastructure
        |__ 📁 Authentication
        |__ 📁 Customers
        |__ 📁 Database
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
|__ 📁 Users
    |__ ...
```

The example above is a small part of the system I built inside of [**Modular Monolith Architecture**](/modular-monolith-architecture).

## Takeaway

**Screaming Architecture** isn't just a catchy phrase, it's an approach that can profoundly impact how you build software.
By organizing your system around use cases, you align your codebase with the core business domain.
Your system exists to solve the business domain problems.

Remember, the goal is to create a system that communicates its purpose through its structure.
Embrace a use case-driven approach, break down complex domains into bounded contexts.
Build a system that truly "screams" about the problems it solves.

If you want to explore these powerful ideas further, check out [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture).
I share my entire framework for building robust applications from the ground up and organizing the system around use cases.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**IronSecureDoc**](https://ironsoftware.com/enterprise/securedoc/?utm_source=milan&utm_medium=email&utm_campaign=milan) -
the PDF security & compliance server.
Protect your PDFs with IronSecureDoc, the latest from Iron Software.
Redact sensitive data, add digital signatures, and stay compliant with PDF/A & PDF/UA standards.
Easy integration with our REST API.
[**Learn more here**](https://ironsoftware.com/enterprise/securedoc/?utm_source=milan&utm_medium=email&utm_campaign=milan).

Test your API using the [**Collection Runner**](https://learning.postman.com/docs/collections/running-collections/intro-to-collection-runs/).
Use Collection Runner to log the test results for your requests, and use scripts to pass data between requests and change the request workflow.
[**Check it out here**](https://learning.postman.com/docs/collections/running-collections/intro-to-collection-runs/).

</Sponsorships>

Have you ever wondered how large-scale systems handle traffic spikes or maintain performance even when parts of the system are temporarily down?
The answer lies in asynchronous messaging.

Asynchronous messaging is, at its core, about decoupling.
Our components can operate independently and communicate through a message queue or topic.
If one service (component) is temporarily unavailable, the others can continue working.
This improves our system's scalability, resilience, and fault tolerance.

In this article, we'll explore how to use Amazon SQS and SNS for asynchronous messaging in .NET applications.

We'll also see how MassTransit simplifies the process, enabling you to build robust message-driven systems.

Let's dive in.

## What is Amazon SQS?

[Amazon Simple Queue Service](https://aws.amazon.com/sqs/) (SQS) is a fully managed message queueing service.
It facilitates the decoupling and scaling of microservices and distributed systems.

SQS acts as a reliable middleman for asynchronous communication.
It enables different components of your architecture to exchange messages without needing to be online or directly connected at the same time.
Messages are stored in queues and consumed on demand.

SQS offers two distinct queue types depending on your requirements:

- **Standard Queues**: Ideal for high-throughput scenarios.
  Standard Queues provide at-least-once delivery and best-effort ordering.
- **FIFO Queues**: Recommended when maintaining message order is required.
  FIFO Queues guarantee exactly-once processing and preserve the sequence in which messages are sent.

Let's say we have two services - `Stock` and `Reporting`.
When a user creates a purchase order in the `Stock` service, we want to notify the `Reporting` service.

SQS allows us to create decoupled communication between these services.
The `Stock` service sends a message to an SQS queue, and the `Reporting` service can poll from the queue to consume messages.

<Image
  src="blogs/mnw_103/amazon_sqs.png"
  width={1920}
  height={1080}
  alt="Amazon SQS."
/>

It's interesting to highlight that SQS uses a polling mechanism for message consumers.
When a consumer polls for new messages, SQS starts a [visibility timeout](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html).
SQS doesn't automatically delete the messages.
Instead, messages are hidden from other consumers until the timeout expires.

When the consumer successfully processes a message, it's removed from the queue.
But if the visibility timeout expires, the message becomes visible and can be delivered again.
Other consumers can receive this message when polling from SQS.
This is why SQS offers at-least-once delivery (for standard queues).
You will have to implement [**idempotency in the consumer**](idempotent-consumer-handling-duplicate-messages).

## Amazon SQS and Competing Consumers

Let's introduce another service into our system - the `Risk Management` service.
When multiple consumers (services) poll an SQS queue, each wants to retrieve and process messages as they become available.
However, once a message is successfully received and processed by one consumer, it's removed from the queue.

Why is this a problem?

Other consumers who might have been polling will miss out on that specific message.
This is known as [competing consumers](https://learn.microsoft.com/en-us/azure/architecture/patterns/competing-consumers).

Let's consider the example of `Reporting` and `Risk Management` services polling the same queue.
If a new message arrives, only one of these services will "win" the race and retrieve it for processing.
The other service won't find that message even if it polls moments later.

<Image
  src="blogs/mnw_103/amazon_sqs_competing_consumers.png"
  width={1920}
  height={1080}
  alt="Amazon SQS and competing consumers."
/>

So, how can we solve this?

We could introduce a dedicated queue for each service.
However, the producer now needs to publish to multiple queues.
This creates a possibility for some (not so) interesting partial failures.
What happens if we successfully publish to one queue but fail to publish to the other?

You can see how this becomes difficult to scale while maintaining reliability.

Luckily, there's a solution.

## Amazon SNS to The Rescue

[Amazon Simple Notification Service](https://aws.amazon.com/sns/) (SNS) is a fully managed pub/sub messaging service.
It allows publishers to send messages to multiple subscribers (topics) simultaneously.

SNS operates on the principle of publishers and subscribers.
Publishers send messages to an SNS topic, while subscribers express interest in specific topics and receive messages published to those topics.
This decoupled architecture allows you to add or remove subscribers without impacting the publisher or other subscribers.

SNS seamlessly integrates with SQS, allowing you to create a powerful combination where SNS handles the fan-out of messages,
and SQS queues ensure that each message is processed exclusively by a single consumer (service).

Instead of sending a message to the queue, the `Stock` service now publishes to an SNS topic.
Both the `Reporting` and `Risk Management` services create their own SQS queues and subscribe these queues to the SNS topic.
When a new message is published to the SNS topic, SNS delivers it to both SQS queues.
Each queue receives its own copy of the message.

If we want to introduce a new service, we'll create a new SQS queue and subscribe it to the topic.

<Image
  src="blogs/mnw_103/amazon_sns.png"
  width={1920}
  height={1080}
  alt="Amazon SNS."
/>

## MassTransit Integration With SQS and SNS

How can we use SNS and SQS from a .NET application?

You could use the official AWS SDKs.
The benefit is you'll have more control over messaging.
However, you will need to write more code to receive and handle messages successfully.

So, I want to suggest a different approach.

MassTransit is one of the most popular messaging libraries in .NET.
It provides a set of messaging abstractions on top of the supported message transports.

I wrote an article about [**using MassTransit with RabbitMQ and Azure Service Bus**](using-masstransit-with-rabbitmq-and-azure-service-bus).

But we'll focus on using MassTransit with SQS and SNS.

Let's start by installing the NuGet package we'll need:

```powershell
Install-Package MassTransit.AmazonSQS
```

Next, we'll need to configure MassTransit with our .NET applications.

Here's the MassTransit configuration for the `Stock` service:

```csharp
builder.Services.AddMassTransit(configure =>
{
    configure.AddConsumer<PurchaseOrderSentConsumer>().Endpoint(e => e.InstanceId = "stocks");

    configure.UsingAmazonSqs((context, cfg) =>
    {
        cfg.Host("eu-central-1", h =>
        {
            h.AccessKey(builder.Configuration["AmazonSqs:AccessKey"]!);
            h.SecretKey(builder.Configuration["AmazonSqs:SecretKey"]!);

            h.Scope("stocks-platform", scopeTopics: true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("stocks-platform-", false));
    });
});
```

Here are a few things I want to highlight here:

- `UsingAmazonSqs` - Configures SQS (and SNS) as the message transport.
- `Scope` - Adds a prefix to SNS topic names. This helps distinguish topics from other applications and environments.
- `Endpoint` - Sets the `InstanceId`, which is appended to the endpoint (queue) name.
- `ConfigureEndpoints` - Allows us to specify a prefix for endpoint (queue) names. The idea is the same as the topic name prefix.

You'll also need to configure an [IAM policy](https://masstransit.io/documentation/configuration/transports/amazon-sqs#example-iam-policy)
that gives MassTransit the required permissions for AWS resources.

And with this setup in place, you can use MassTransit to publish messages.
Here's an endpoint that accepts a purchase order and publishes a `PurchaseOrderSent` message.

```csharp
app.MapPost("purchase-orders", async (PurchaseOrderRequest request, IPublishEndpoint publishEndpoint) =>
{
    var purchaseOrder = new PurchaseOrder
    {
        Id = Guid.NewGuid(),
        Ticker = request.Ticker,
        LimitPrice = request.LimitPrice,
        Quantity = request.Quantity
    };

    OrdersDb.Add(purchaseOrder);

    await publishEndpoint.Publish(new PurchaseOrderSent(purchaseOrder.Id));

    return Results.Ok(purchaseOrder);
});
```

We will process the purchase order in the `PurchaseOrderSentConsumer`.
If it's successfully processed (filled), we will publish an `OrderFilled` message.

The `Risk Management` service can subscribe to this message using MassTransit.
The configuration is almost identical, with the only difference being the endpoint's `InstanceId`.

```csharp
builder.Services.AddMassTransit(configure =>
{
    configure.AddConsumer<OrderFilledConsumer>().Endpoint(e => e.InstanceId = "risk-management");

    configure.UsingAmazonSqs((context, cfg) =>
    {
        cfg.Host("eu-central-1", h =>
        {
            h.AccessKey(builder.Configuration["AmazonSqs:AccessKey"]!);
            h.SecretKey(builder.Configuration["AmazonSqs:SecretKey"]!);

            h.Scope("stocks-platform", true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("stocks-platform-", false));
    });
});
```

## Broker Topology in AWS

MassTransit will automatically create the required queues, topics, and subscriptions in AWS.
If needed, you can further configure the SQS and SNS resources.

Of course, you can also create the required infrastructure in AWS and tell MassTransit to use it.

But let's keep it simple and allow MassTransit to provision the AWS resources.

By default, MassTransit creates standard SQS queues.
Here's what we get in Amazon SQS.

<div className="bordered">
  <Image
    src="blogs/mnw_103/amazon_sqs_queues.png"
    width={1070}
    height={720}
    alt="Amazon SQS queues created by MassTransit."
  />
</div>

And here's what we get in Amazon SNS:

<div className="bordered">
  <Image
    src="blogs/mnw_103/amazon_sns_topics.png"
    width={1280}
    height={650}
    alt="Amazon SNS topics created by MassTransit."
  />
</div>

MassTransit automatically configures the required subscriptions between the topic and queues.

And we are ready to start publishing and consuming messages.

## In Summary

Asynchronous communication and decoupling are pivotal in achieving scalability, resilience, and fault tolerance.
Amazon SQS and SNS provide the building blocks of message-driven architectures in the AWS cloud.

We explored the core concepts of SQS and SNS, understanding how they enable reliable message delivery and fan-out capabilities.

MassTransit provides an excellent abstraction layer over SQS and SNS, simplifying development.
We can focus on solving the business problems and delivering value to our users.

The combination of Amazon SQS, SNS, and MassTransit gives us robust tools for building modern, event-driven applications.

Thanks for reading, and I'll see you next week!

**P.S.** You can find the source code for this article in [**this repository**](https://github.com/m-jovanovic/aws-tutorials), under the `Amazon SQS and SNS` folder.

---



<Sponsorships>

[**EF Core is too slow?**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter)
Discover how you can easily insert 14x faster (reducing saving time by 94%).
Boost your performance with our method integrated within EF Core: Bulk Insert, update, delete, and merge.
Join 5,000+ satisfied customers who have trusted our library since 2014.
[**Click here to learn more**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter).

[**WPF or .NET MAUI - How to Choose**](https://www.telerik.com/blogs/wpf-net-maui-how-choose?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-general-awareness)?
Creating a desktop application and need to decide between the long-time reliable WPF and the up-and-coming cross-platform .NET MAUI?
[**This blog post**](https://www.telerik.com/blogs/wpf-net-maui-how-choose?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-general-awareness)
will help you explore the features that matter.

</Sponsorships>

Okay, let's be honest.
We all have a million things on our plates, and diving deep into every nook and cranny of EF Core might not be
at the top of your priority list.

But here's the deal: EF Core is powerful, and knowing a few key features can save you lots of time and frustration.

So, I won't bombard you with every single EF Core feature under the sun.

Instead, I've cherry-picked five essential ones that you really need to know.

We'll go through:

- **Query Splitting** - your database's new best friend
- **Bulk Updates and Deletes** - efficiency on steroids
- **Raw SQL Queries** - when you need to go rogue
- **Query Filters** - keeping things nice and tidy
- **Eager Loading** - because lazy isn't so great

Let's get started!

## Query Splitting

[**Query splitting**](how-to-improve-performance-with-ef-core-query-splitting) is one of those EF Core features that you rarely need.
Until one day, you do.
Query splitting is helpful in scenarios where you're eager loading multiple collections.
It helps us avoid the [cartesian explosion](https://learn.microsoft.com/en-us/ef/core/performance/efficient-querying#avoid-cartesian-explosion-when-loading-related-entities) problem.

Let's say we want to retrieve a department with all its teams and employees.
We might write a query like this:

```csharp
Department department =
    context.Departments
        .Include(d => d.Teams)
        .Include(d => d.Employees)
        .Where(d => d.Id == departmentId)
        .First();
```

This translates to a single SQL query with two JOINs.
However, since these `JOIN` statements are on the same level, the database will return a _cross product_.
Each row from `Teams` will be joined with each row `Employees`.
In that case, the database returns many rows, significantly impacting performance.

Here's how we can avoid these performance issues with query splitting:

```csharp
Department department =
    context.Departments
        .Include(d => d.Teams)
        .Include(d => d.Employees)
        .Where(d => d.Id == departmentId)
        .AsSplitQuery()
        .First();
```

With `AsSplitQuery`, EF Core will execute an additional SQL query for each collection navigation.

However, be cautious not to overuse query splitting.
I use split queries when I've _measured_ that they consistently perform better.

Split queries have more round trips to the database, which might be slower if database latency is high.
There is also no consistency guarantee across multiple SQL queries.

## Bulk Updates and Deletes

EF Core 7 added two new APIs for performing [**bulk updates and deletes**](how-to-use-the-new-bulk-update-feature-in-ef-core-7),
`ExecuteUpdate` and `ExecuteDelete`.
They allow you to efficiently update a large number of rows in one round trip to the database.

Here's a practical example.

The company has decided to give a 5% raise to all employees in the "Sales" department.
Without bulk updates, we might iterate through each employee and update their salary individually:

```csharp
var salesEmployees = context.Employees
    .Where(e => e.Department == "Sales")
    .ToList();

foreach (var employee in salesEmployees)
{
    employee.Salary *= 1.05m;
}

context.SaveChanges();
```

This approach involves multiple database roundtrips, which can be inefficient, especially for large datasets.

We can achieve the same in one roundtrip using `ExecuteUpdate`:

```csharp
context.Employees
    .Where(e => e.Department == "Sales")
    .ExecuteUpdate(s => s.SetProperty(e => e.Salary, e => e.Salary * 1.05m));
```

This executes a single SQL `UPDATE` statement, directly modifying the salaries in the database without loading entities into memory, giving us improved performance.

Here's another example.
Let's say an e-commerce platform wants to delete all shopping carts older than one year.

Here's how we could do this with `ExecuteDelete`:

```csharp
context.Carts
    .Where(o => o.CreatedOn < DateTime.Now.AddYears(-1))
    .ExecuteDelete();
```

This results in a single SQL `DELETE` statement, directly removing the old shopping carts from the database.

However, bulk updates bypass the EF change tracker.
This could be problematic, and I wrote about the [**caveats of bulk updates in this article**](what-you-need-to-know-about-ef-core-bulk-updates).

## Raw SQL Queries

EF Core 8 added a new feature that allows us to query unmapped types with raw SQL.

Suppose we want to retrieve data from a database view, stored procedure, or a table that doesn't directly correspond to any of our entity classes.

For example, we want to retrieve a sales summary for each product.
With EF Core 8, we can define a simple `ProductSummary` class representing the structure of the result set and query it directly:

```csharp
public class ProductSummary
{
    public int ProductId { get; set; }
    public string ProductName { get; set; }
    public decimal TotalSales { get; set; }
}

var productSummaries = await context.Database
    .SqlQuery<ProductSummary>(
        @$"""
        SELECT p.ProductId, p.ProductName, SUM(oi.Quantity * oi.UnitPrice) AS TotalSales
        FROM Products p
        JOIN OrderItems oi ON p.ProductId = oi.ProductId
        WHERE p.CategoryId = {categoryId}
        GROUP BY p.ProductId, p.ProductName
        """)
    .ToListAsync();
```

The `SqlQuery` method returns an `IQueryable`, which allows you to compose raw SQL queries with LINQ.
This combines the power of raw SQL with the expressiveness of LINQ.

Remember to use parameterized queries to prevent SQL injection vulnerabilities.
The `SqlQuery` method accepts a `FormattableString`, which means you can safely use an interpolated string.
Each argument is converted to a SQL parameter.

You can learn more about [**raw SQL queries in this article**](ef-core-raw-sql-queries).

## Query Filters

[**Query filters**](how-to-use-global-query-filters-in-ef-core) are like reusable `WHERE` clauses you can apply to your entities.
These filters are automatically added to LINQ queries whenever you retrieve entities of the corresponding type.
This saves you from repeatedly writing the same filtering logic in multiple places within your application.

Query Filters are commonly used for scenarios like:

- [**Soft Deletes**](implementing-soft-delete-with-ef-core): Filter out records marked as deleted.
- [**Multi-tenancy**](multi-tenant-applications-with-ef-core): Filter data based on the current tenant.
- Row-level security: Restrict access to certain records based on user roles or permissions.

In a multi-tenant application, you often need to filter data based on the current tenant.
Query filters allow us to handle this requirement easily:

```csharp
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    // Associate products with tenants
    public int TenantId { get; set; }
}

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    // The current TenantId is set based on the current request/context
    modelBuilder.Entity<Product>().HasQueryFilter(p => p.TenantId == _currentTenantId);
}

// Now, queries automatically filter based on the tenant:
var productsForCurrentTenant = context.Products.ToList();
```

Configuring multiple query filters on the same entity will only apply the last one.
You can combine multiple query filters using `&&` (AND) and `||` (OR) operators.

You can use `IgnoreQueryFilters` to bypass the filters in specific queries when needed.

## Eager Loading

Eager Loading is a feature in EF Core that allows you to load related entities along with your main entity in a single database query.
By fetching all necessary data in a single query, you can improve application performance.
This is especially true when dealing with complex object graphs or when lazy loading would result in many small, inefficient queries.

Here's an example `VerifyEmail` use case.
We want to load an `EmailVerificationToken` and eagerly load a `User` with the `Include` method because we want to modify both entities at the same time.

```csharp {6}
internal sealed class VerifyEmail(AppDbContext context)
{
    public async Task<bool> Handle(Guid tokenId)
    {
        EmailVerificationToken? token = await context.EmailVerificationTokens
            .Include(e => e.User)
            .FirstOrDefaultAsync(e => e.Id == tokenId);

        if (token is null || token.ExpiresOnUtc < DateTime.UtcNow || token.User.EmailVerified)
        {
            return false;
        }

        token.User.EmailVerified = true;

        context.EmailVerificationTokens.Remove(token);

        await context.SaveChangesAsync();

        return true;
    }
}
```

EF Core will generate a single SQL query that joins the `EmailVerificationToken` and `User` tables, retrieving all the necessary data in one go.

Eager loading (and query splitting, which we mentioned earlier) isn't a silver bullet.
Consider using projections if you only need specific properties from related entities to avoid fetching unnecessary data.

## Summary

So, there you have it!
Five EF Core features that, frankly, you can't afford _not_ to know.
Remember, mastering EF Core takes time, but these features provide a solid foundation to build upon.

Another piece of advice is to deeply understand how your database works.
Mastering SQL also allows you to get the most value from EF Core.

While we focused on five key features, there are many other EF Core features worth exploring:

- [**Optimistic concurrency control**](solving-race-conditions-with-ef-core-optimistic-locking)
- [**Database migrations**](efcore-migrations-a-detailed-guide)
- [**Compiled queries**](unleash-ef-core-performance-with-compiled-queries)
- [**Transactions**](working-with-transactions-in-ef-core)
- [**Interceptors**](how-to-use-ef-core-interceptors)

EF Core is continuously evolving, so keep an eye on the latest updates and releases to stay ahead.

Good luck out there, and see you next week.

---


<Sponsorships>

Take Postman wherever you work with the [**Postman VS Code extension**](https://learning.postman.com/docs/getting-started/basics/about-vs-code-extension)!
The Postman VS Code extension enables you to develop and test your APIs in Postman directly from Visual Studio Code and Visual Studio Code Insiders.
[**Try it out**](https://learning.postman.com/docs/getting-started/basics/about-vs-code-extension)!

Are you tired of wasting time and money on complex e-signature solutions?
[**BoldSign**](https://boldsign.com/esignature-api/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign)
offers an easy-to-use .NET SDK that seamlessly integrates e-signatures into your application.
Try our [**free e-signature API**](https://boldsign.com/esignature-api/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign) sandbox today!

</Sponsorships>

Writing good code is important for any software project.
It's also something I deeply care about.
However, it can be hard to spot problems by just reading through everything.

Luckily, there's a tool that can help: **static code analysis**.

It's like having an extra pair of eyes automatically checking your code.
Static code analysis helps you build secure, maintainable, and high-quality C# code.

Here's what we are going to cover in this week's newsletter:

- Static code analysis
- Static analysis in .NET
- Finding security risks

Let's see how static code analysis can help us improve our code quality.

## What is Static Code Analysis?

Static code analysis is a way to examine your code without actually running it.
It reports any issues related to security, performance, coding style, or best practices.

With static code analysis, you can ["shift left"](https://en.wikipedia.org/wiki/Shift-left_testing).
This allows you to find and fix issues early in the development process when they're less expensive to solve.

By writing high-quality code, you'll be able to build systems that are more reliable, scalable, and easier to maintain over time.
Investing in code quality will pay dividends in the later stages of any project.

You can integrate static code analysis into your [**CI pipeline**](how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet) for a quick feedback loop.
We can also pair this with [**architecture testing**](shift-left-with-architecture-testing-in-dotnet) to enforce additional coding standards.

## Static Code Analysis in .NET

.NET has built-in Roslyn analyzers that inspect your C# code for code style and quality issues.
Code analysis is enabled by default if your project targets .NET 5 or later.

The best way I found to configure static code analysis is using `Directory.Build.props`.
It's an XML file where you can configure common project properties.
You can place the `Directory.Build.props` file in the root folder so it will apply to all projects.

You can configure the `TargetFramework`, `ImplicitUsings`, `Nullable` (nullable reference types), etc.
But what we care about is configuring static code analysis.

Here are some properties we can configure:

- `TreatWarningsAsErrors` - Treat all warnings as errors.
- `CodeAnalysisTreatWarningsAsErrors` - Treat code quality (CAxxxx) warnings as errors.
- `EnforceCodeStyleInBuild` - Enables code-style analysis ("IDExxxx") rules.
- `AnalysisLevel` - Specifies which analyzers to enable. The default value is `latest`.
- `AnalysisMode` - Configures the predefined code analysis configuration.

We can also install additional NuGet packages to our projects.
`SonarAnalyzer.CSharp` contains additional code analyzers to help us write clean, safe, and reliable code.
This library comes from the same company that built [SonarQube](https://www.sonarsource.com/products/sonarqube/).

```xml {8-12,16}
<Project>
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    
    <AnalysisLevel>latest</AnalysisLevel>
    <AnalysisMode>All</AnalysisMode>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <CodeAnalysisTreatWarningsAsErrors>true</CodeAnalysisTreatWarningsAsErrors>
    <EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
  </PropertyGroup>

  <ItemGroup Condition="'$(MSBuildProjectExtension)' != '.dcproj'">
    <PackageReference Include="SonarAnalyzer.CSharp" Version="*">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>
        runtime; build; native; contentfiles; analyzers; buildtransitive
      </IncludeAssets>
    </PackageReference>
  </ItemGroup>
</Project>
```

The built-in .NET analyzers and the ones from `SonarAnalyzer.CSharp` can be very helpful.
But they can also make a lot of noise with too many build warnings.

When you encounter code analysis rules that you don't consider helpful, you can turn them off.
You can configure individual code analysis rules in the `.editorconfig` file.

```
# S125: Sections of code should not be commented out
dotnet_diagnostic.S125.severity = none

# S1075: URIs should not be hardcoded
dotnet_diagnostic.S1075.severity = none

# S2094: Classes should not be empty
dotnet_diagnostic.S2094.severity = none

# S3267: Loops should be simplified with "LINQ" expressions
dotnet_diagnostic.S3267.severity = none
```

## Finding (and Fixing) Security Risks

Static code analysis can help you detect potential security vulnerabilities in your code.
Here's an example of a `PasswordHasher` using only `10,000` iterations to generate a password hash.
The `S5344` rule, from `SonarAnalyzer.CSharp`, detects this issue and warns us.
The recommended minimal number of iterations is `100,000`.

You can navigate to the explanation for [S5344](https://rules.sonarsource.com/csharp/RSPEC-5344/) to learn more:

> Weakly hashed password storage poses a significant security risk to software applications.

With `TreatWarningsAsErrors` turned on, your build will fail until you solve this issue.
This reduces the chance of introducing security risks in production.

<div className="bordered">
  <Image
    src="blogs/mnw_101/static_code_analysis.png"
    width={1318}
    height={830}
    alt="Example of static code analysis warning."
  />
</div>

## Conclusion

Static code analysis is a powerful tool I include in all my C# projects.
It helps me catch problems early, leads to more reliable and secure code, and saves time and effort.
While the initial setup and fine-tuning of rules might take some time, the long-term benefits are undeniable.

Remember, static code analysis is a tool that complements your existing development practices.

You can create a robust development process that consistently delivers high-quality software by combining
static code analysis with other techniques like code reviews, unit testing, and continuous integration.

Embrace static code analysis.
Your future self (and your team) will thank you.

That's all for today.

See you next week.

**P.S.** Here's a sample [.editorconfig](https://gist.github.com/m-jovanovic/417b7d0a641d7dd7d1972550fba298db) file you can add to your projects and customize to fit your needs.

---


<Sponsorships>

[**API Collaboration**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/)
has never been easier with comments on collections, folders, and requests using Postman!
Launch into hyper-efficiency using comments to discuss your work with your teammates in Postman.
[**Learn more here**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/).

Build automations with [**UiPath Autopilot™**](https://rebrand.ly/dev-dives-webinar).
Join this free webinar to learn how to build automation workflows, generate code, and create ready-to-use apps by simply describing your needs to the AI-powered Autopilot.
[Register today](https://rebrand.ly/dev-dives-webinar)!

</Sponsorships>

Redis is a popular choice for [**caching data**](caching-in-aspnetcore-improving-application-performance), but its capabilities go far beyond that.
One of its lesser-known features is Pub/Sub support.
Redis channels offer an interesting approach for implementing real-time messaging in your .NET applications.
However, as you'll soon see, channels also have some drawbacks.

In this week's newsletter, we'll explore:

- Basics of Redis channels
- Practical use cases for channels
- Implementing a Pub/Sub example in .NET
- Cache invalidation in distributed systems

Let's dive in.

## Redis Channels

Redis channels are named communication channels that implement the [Publish/Subscribe messaging paradigm](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern).
Each channel is identified by a unique name (e.g., `notifications`, `updates`).
Channels facilitate message delivery from publishers to subscribers.

Publishers use the `PUBLISH` command to send messages to a specific channel.
Subscribers use the `SUBSCRIBE` command to register interest in receiving messages from a channel.

<Image
  src="blogs/mnw_100/redis_channel.png"
  width={2094}
  height={1294}
  alt="Redis channel with publisher and three subscribers."
/>

Redis channels follow a topic-based publish-subscribe model.
Multiple publishers can send messages to a channel, and multiple subscribers can receive messages from that channel.

However, it's crucial to note that Redis channels do not store messages.
If there are no subscribers for a channel when a message is published, that message is immediately discarded.

Redis channels have an **at-most-once delivery** semantics.

## Practical Use Cases

Given that Redis channels operate with **at-most-once delivery** (messages might be lost if there are no subscribers),
they are well-suited for scenarios where occasional message loss is acceptable and real-time or near-real-time communication is desired.

Here are a few possible use cases:

- **Social media feeds**: Broadcasting new posts or updates to users.
- **Live score updates**: Sending live game scores or sports updates to subscribers.
- **Chat applications**: Delivering chat messages in real-time to active participants.
- **Collaborative editing**: Propagating changes in collaborative editing environments.
- **Distributed cache updates**: Invalidating cache entries across multiple servers when data changes. We'll cover this in detail later in the article.

Redis channels aren't the best choice for critical data where message loss is unacceptable.
In such cases, you should consider a more reliable messaging system.

Let's see how we can use Redis channels in .NET.

## Pub/Sub With Redis Channels

We will use the `StackExchange.Redis` library to send messages with Redis channels.

Let's start by installing it:

```powershell
Install-Package StackExchange.Redis
```

You can run [Redis](https://redis.io/) locally in a Docker container.
The default port is `6379`.

```
docker run -it -p 6379:6379 redis
```

Here's a simple background service that'll act as our message `Producer`.

We're creating a `ConnectionMultiplexer` by connecting to our Redis instance.
This allows us to obtain an `ISubscriber` that we can use for pub/sub messaging.
The `ISubscriber` will enable us to publish a message to a channel by specifying the channel name.

```csharp {11,19}
public class Producer(ILogger<Producer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        while (!stoppingToken.IsCancellationRequested)
        {
            var message = new Message(Guid.NewGuid(), DateTime.UtcNow);

            var json = JsonSerializer.Serialize(message);

            await subscriber.PublishAsync(Channel, json);

            logger.LogInformation(
                "Sending message: {Channel} - {@Message}",
                message);

            await Task.Delay(5000, stoppingToken);
        }
    }
}
```

Let's also introduce a separate background service for consuming messages.

The `Consumer` connects to the same Redis instance and obtains an `ISubscriber`.
The `ISubscriber` exposes a `SubscribeAsync` method that we can use to subscribe to messages from a given channel.
This method accepts a callback delegate that we can use to handle the message.

```csharp
public class Consumer(ILogger<Consumer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        await subscriber.SubscribeAsync(Channel, (channel, message) =>
        {
            var message = JsonSerializer.Deserialize<Message>(message);

            logger.LogInformation(
                "Received message: {Channel} - {@Message}",
                channel,
                message);
        });
    }
}
```

Finally, here's what we get when we run both the `Producer` and `Consumer` services:

<Image
  src="blogs/mnw_100/redis_pub_sub.gif"
  width={1920}
  height={500}
  alt="Redis channels publish/subscribe demo."
/>

## Cache Invalidation in Distributed Systems

In a recent project, I tackled a common challenge in distributed systems: keeping the caches in sync.
We were using a two-level caching approach.
First, we had an in-memory cache on each web server for super-fast access.
Second, we had a shared Redis cache to avoid hitting our database too often.

The problem was that when data changed in the database, we needed a way to quickly tell all the web servers to clear their in-memory caches.
This is where Redis Pub/Sub came to the rescue.
We set up a Redis channel specifically for cache invalidation messages.

Each application would run a `CacheInvalidationBackgroundService` that subscribes to messages from the cache invalidation channel.

```csharp
public class CacheInvalidationBackgroundService(
    IServiceProvider serviceProvider)
    : BackgroundService
{
    public const string Channel = "cache-invalidation";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await subscriber.SubscribeAsync(Channel, (channel, key) =>
        {
            var cache = serviceProvider.GetRequiredService<IMemoryCache>();

            cache.Remove(key);

            return Task.CompletedTask;
        });
    }
}
```

Whenever data changes in the database, we publish a message on this channel with the cache key of the updated data.
All the web servers are subscribed to this channel, so they instantly know to remove the old data from their in-memory caches.
Since the in-memory cache is wiped if the application isn't running, losing cache invalidation messages isn't a problem.
This keeps our caches consistent and ensures our users always see the most up-to-date information.

## In Summary

Redis Pub/Sub is not a silver bullet for every messaging need, but its simplicity and speed make it a valuable tool.
Channels allow us to easily implement communication between loosely coupled components.

Redis channels have at-most-once delivery semantics, so they're best suited for cases where the occasional dropped message is acceptable.

I used it to solve the challenge of synchronizing caches across multiple servers.
This allowed our system to serve up-to-date data without sacrificing performance.

**P.S.** When you're ready to dive deeper into creating message-driven systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
I have an entire module dedicated to building reliable distributed messaging and event-driven architecture.

Good luck out there, and see you next week.

---



<Sponsorships>

Streamline API development with [**Package Libary in Postman**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).
Add commonly-used scripts and tests to packages in your team's Package Library, and reuse them in your personal, private, and team workspaces using Postman.
[**Learn more here**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).

The [**IronOCR C# library**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan)
has hit a new milestone with 2 million NuGet downloads!
To celebrate, we're offering a 20% discount on IronOCR team licenses (Lite, Plus, Pro).
Plus, we're excited to introduce AdvancedScan, a powerful new feature that enhances OCR accuracy and Multiple-Language Support.
[**Try it Today**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

Modular monoliths strike a balance between the simplicity of monolithic architecture and the flexibility of microservices.
By breaking down applications into cohesive modules, modular monoliths enable easier development and maintenance.
However, they still have a single codebase and deployment unit.

A critical aspect of the success of a modular monolith is the interaction between its modules.

**System integration testing (SIT)** is an approach to verifying the collaboration of these modules.
It involves testing the integration points and communication mechanisms between modules.

System integration testing allows us to validate the entire system's behavior.
This allows us to catch problems early before they become big headaches.

In this article, we'll learn how to test a modular monolith using system integration testing.

We'll look at real-world examples and discuss why this testing approach is useful.

## What Is System Integration Testing?

System integration testing (SIT) is an approach to testing the interactions between various modules within a single system.
The system could contain many external services, which should also be included during testing.

System integration testing is the perfect testing approach for modular monoliths.
It allows you to mimic the real-world execution of your application (as you'll see later).

The key benefits of system integration testing are:

- **Detecting integration issues**: Discover problems from module integrations that unit testing might miss.
- **Validating business logic**: Confirm that end-to-end business processes function correctly across modules.
- **Ensuring data integrity**: Verify that data flows accurately and consistently between modules.
- **Improving system stability**: Identify and resolve issues early, leading to a more reliable system.
- **Building confidence**: Ensure that the system is well-integrated and ready for deployment.

## Modular Monolith System Example

A modular monolith consists of multiple modules, each with a distinct responsibility.
Modules represent high-level components with well-defined boundaries.
The modules essentially group together related functionalities (use cases).
If you want to learn more about this software architecture, check out this [**introduction to modular monoliths**](what-is-a-modular-monolith).

Let's consider a hypothetical ticketing system that consists of several modules:

- **Users Module**: Handles user administration and authentication.
- **Events Module**: Manages events, scheduling, and ticket availability.
- **Ticketing Module**: Allows users to purchase tickets for various events.
- **Attendance Module**: Allows users to check into events using their tickets.

<figure>
  <Image
    src="blogs/mnw_099/modular_monolith.png"
    width={2506}
    height={1282}
    alt="Modular monolith UML diagram."
  />
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>

During system integration testing, we want to ensure that all these modules interact correctly and
fulfill the business requirements of the ticketing system.

## Testing Modular Monoliths: User Registration

The modules in our ticketing system represent different [**bounded contexts**](monolith-to-microservices-how-a-modular-monolith-helps).
The **Users Module** uses the term `User` as its core entity.
However, the **Ticketing Module** uses the term `Customer` because it's more aligned with its core responsibility of selling tickets.
Conceptually, both entities represent the same person within our system.

Here's the scenario we want to test:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record

<Image
  src="blogs/mnw_099/user_registration.png"
  width={2866}
  height={877}
  alt="User registration flow diagram."
/>

If you haven't figured it out by now, our [**modules communicate asynchronously**](modular-monolith-communication-patterns) using messaging.

Why is this important for our tests?

There is a time delay from when something occurs in one module until the other modules are notified and handle the integration events.
Our system integration tests will have to take into account this delay.

Here's the test for this scenario:

```csharp
public class RegisterUserTests : BaseIntegrationTest
{
    public RegisterUserTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task RegisterUser_Should_PropagateToTicketingModule()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);

                var customerResult = await Sender.Send(query);

                return customerResult;
            });

        // Assert
        customerResult.IsSuccess.Should().BeTrue();
        customerResult.Value.Should().NotBeNull();
    }
}
```

A lot is going on here, so let's unpack the steps:

- We're using the `WebApplicationFactory` to run an in-memory application instance
- Any external dependencies run in a Docker container using [**Testcontainers**](testcontainers-integration-testing-using-docker-in-dotnet)
- We send the `RegisterUserCommand` to register a user in the **Users Module**
- Then, we have to poll the **Ticketing Module** by sending a `GetCustomerQuery`
- The `Poller` allows us to wait until the system eventually becomes consistent

You can learn more about [**integration testing with Testcontainers in this article**](testcontainers-integration-testing-using-docker-in-dotnet).

System integration tests take longer to execute than integration tests for one module.
This is a side effect of testing a bigger slice of the overall system in each test case.

The `Poller` class allows you to execute a delegate until you receive a successful result or a timeout occurs.
You can customize the timeout duration based on the scenario you are testing.

```csharp
internal static class Poller
{
    private static readonly Error Timeout =
        Error.Failure("Poller.Timeout", "The poller has time out");

    internal static async Task<Result<T>> WaitAsync<T>(
        TimeSpan timeout,
        Func<Task<Result<T>>> func)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));

        DateTime endTimeUtc = DateTime.UtcNow.Add(timeout);
        while (DateTime.UtcNow < endTimeUtc &&
               await timer.WaitForNextTickAsync())
        {
            Result<T> result = await func();

            if (result.IsSuccess)
            {
                return result;
            }
        }

        return Result.Failure<T>(Timeout);
    }
}
```

## Testing Modular Monoliths: Adding Ticket to Cart

Here's a more complex scenario testing adding a ticket to the customer's cart:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record
- The **Ticketing Module** creates a dummy event and adds a ticket to the customer's cart

This scenario mimics a user registering with our system, finding a ticket they want to purchase, and adding it to their cart.
We can extend this scenario with more test cases, like ticket purchases.

```csharp
public sealed class AddItemToCartTests : BaseIntegrationTest
{
    private const decimal Quantity = 10;

    public AddItemToCartTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Customer_ShouldBeAbleTo_AddItemToCart()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);
                var customerResult = await Sender.Send(query);
                return customerResult;
            });

        customerResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Add item to cart
        CustomerResponse customer = customerResult.Value;
        var ticketTypeId = Guid.NewGuid();

        await Sender.CreateEventAsync(Guid.NewGuid(), ticketTypeId, Quantity);

        Result result = await Sender.Send(
            new AddItemToCartCommand(customer.Id, ticketTypeId, Quantity));

        // Assert
        result.IsSuccess.Should().BeTrue();
    }
}
```

What I like about system integration testing is that the test cases aren't complicated to write.
The tests execute the use cases of our system and verify the side effects.
Building your system around [**use cases**](building-your-first-use-case-with-clean-architecture) has a few advantages,
the main one being that you can focus on the core business logic.
But it also makes (integration) testing easier.
The use case runs by sending a command or a query.
We can execute the use cases in some logical order and check that we get the expected result.

## In Conclusion

System integration testing is a crucial step in creating a successful modular monolith.
By testing how different modules work together, you can find and fix problems before they become bigger issues.
This makes your software more reliable and saves you time and resources in the long run.

If you want to dive deeper into building modular systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
There's an entire chapter dedicated to testing,
including advanced techniques like system integration testing, using Testcontainers for external services, and automated testing with CI/CD pipelines.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713)

Collaborate directly with your external partners in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/) to help your partners consume your team's APIs and work together on API projects.
Check out Postman's Partner Workspaces, now available for Professional and Enterprise plans in [**Postman v11**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/).

</Sponsorships>

This is a question I often hear: how do I design my use case with Clean Architecture?

I understand the confusion.
Figuring out what to place in the Domain, Application, and Infrastructure layer can seem complicated.
If that's not enough, we also have to decide what makes up a use case and what should be abstracted away.

However, things become simpler if we adhere to the main rule in Clean Architecture — **the Dependency Rule**.
This rule states that source code dependencies can only point inwards.

In this newsletter, we'll explore a practical example of how to apply Clean Architecture principles by building a user registration feature.

## Clean Architecture

[**Clean Architecture**](why-clean-architecture-is-great-for-complex-projects) has emerged as a guiding principle for crafting maintainable, scalable, and testable applications.
At its core, Clean Architecture emphasizes the **separation of concerns** and the **dependency rule**.
The dependency rule dictates that dependencies should point inward toward higher-level modules.
By following this rule, you create a system where the core business logic of your application is decoupled from external dependencies.
This makes it more adaptable to changes and easier to test.

<Image
  src="blogs/mnw_098/clean_architecture.png"
  width={1280}
  height={714}
  alt="Clean Architecture diagram."
/>

The Domain layer encapsulates enterprise-wide business rules.
It contains domain entities, where an entity is typically an object with methods.

The Application layer contains application-specific business rules and encapsulates all of the system's use cases.
A use case orchestrates the flow of data to and from the domain entities and calls the methods exposed by the entities
to achieve its goals.

The Infrastructure and Presentation layers deal with external concerns.
Here, you will implement any abstractions defined in the inner layers.

## Describing The Use Case

What does it mean for a user to register with our application?
It means they reserve an email address (or username) to identify themselves and be able to interact with our system.
The user could provide other information, such as a first and last name, an address, and a phone number.

The first step in building any feature is clearly defining the desired result.

For user registration, this is what the required operations are:

- The user provides an email and password for registration
- Verify that the email was not reserved previously by an existing account
- Hash the password using some cryptographic hash function (e.g., SHA-256, SHA-512)
- Store the user in the database and (optionally) return an access token to the client

We could also consider any domain-specific rules or validations that we must enforce.
A good example is password strength, where we could implement minimum length and complexity requirements.

Now that we have our requirements let's see how to translate them into a use case.

## Implementing the Use Case

With our requirements in place, we can now define the user registration use case.
In [**Clean Architecture**](clean-architecture-and-the-benefits-of-structured-software-design), use cases live in the Application layer and orchestrate the interactions between domain entities and external dependencies.

Let's name our use case `RegisterUser`.
Its input will be a `RegistrationRequest` object containing the user's registration data,
and its output will be a `RegistrationResult` object indicating the outcome of the registration attempt.
Notice that we are using a feature-driven name for the use case.

What about any external dependencies?
If the use case needs to interact with an external system or infrastructure component, we abstract that behind an interface.
Remember, your application's core business logic should be decoupled from external dependencies.

The `RegisterUser` class will use dependency injection to get the necessary dependencies:

- `IUserRepository`: An interface for accessing user data from the database.
- `IPasswordHasher`: An interface for hashing passwords securely.

The `RegisterUser` use case will follow these steps:

1. Validate input data
2. Check for existing `User`
3. Hash the password
4. Create a new `User` entity
5. Save the `User` to the database
6. Return the result

Finally, here's the code for our `RegisterUser` use case:

```csharp
public class RegisterUser(
    IUserRepository userRepository,
    IPasswordHasher passwordHasher)
{
    public async Task<RegistrationResult> Handle(RegistrationRequest request)
    {
        // Validation omitted for brevity

        if (await userRepository.ExistsAsync(request.Email))
        {
            return RegistrationResult.EmailNotUnique;
        }

        var passwordHash = passwordHasher.Hash(request.Password);

        var user = User.Create(
            request.FirstName, request.LastName, request.Email, passwordHash);

        await userRepository.InsertAsync(user);

        return RegistrationResult.Success;
    }
}
```

A big benefit of this approach is that we can immediately write tests for the `RegisterUser` use case.
We can provide mocks for external dependencies in the tests.
We don't need the implementations to exist for this code to compile.
With mocks, we can test our business rules and validate our implementation.

**Action step**: How would you extend the `RegisterUser` use case with more functionality?

Here are two examples:

- Adding an external identity provider
- Implementing email verification

## Where Clean Architecture Becomes Muddled

By designing our application with Clean Architecture, we produce a system independent of external concerns.
We define abstractions in the Application layer and implement them in the Infrastructure layer.
So far, so good.

However, this doesn't mean you can disregard how you integrate with external dependencies.

In theory, we should be able to "swap" the implementation for any external concern and call it a day.
In practice, this couldn't be further from the truth.

Let me give you two practical examples using the user registration flow.

### Race Conditions

The `RegisterUser` use case has a race condition.
Concurrent requests could pass the check for email uniqueness and proceed to register the user.

We could prevent this race condition by introducing a lock before checking for email uniqueness.
That way, only one request will pass the check and proceed to save the user in the database.

```csharp
if (await userRepository.ExistsAsync(request.Email))
{
    return RegistrationResult.EmailNotUnique;
}
```

However, there is a much more elegant way to solve this.
We can introduce a unique index on the `Email` column in the database.
A unique index guarantees that only one transaction can write the unique value to the database.
The losing transaction will return an error.

We can handle this exception on the application side and return an appropriate error message to the user.
The `IUserRepository.InsertAsync` method implementation can encapsulate this logic.

### Changing Hash Functions

Let's say we found a security flaw in the hash function used in the `IPasswordHasher` implementation.
So, we spend a few minutes switching to a more secure hash function.
The tests for the `RegisterUser` use case are all green, and everything seems fine.

The problem? All existing users can no longer log in to the system.

When an existing user tries to log in with their email and password, the new `IPasswordHasher.Hash` implementation
returns a different password hash from the one stored in the database.

The correct approach is to phase out the old password hash for existing users.
We can add a column in the database that says which hash function produced the hash.
We will verify the user's password using the correct hashing function during the login process.

If the user's password hash still uses the old hashing function, we will verify their password first.
Then, we can use the password (which we have in memory) to produce a hash using the new hash function.
We will store the hash in the database and update the hash function column to the new algorithm.

Slowly, we will phase out passwords using the old hash function.

## Conclusion

I hope this was helpful in understanding how to apply Clean Architecture principles to a real-world scenario.
By focusing on the core business logic first (what it means for a user to register), we can define the requirements for our use case.
Translating these requirements into a series of steps within the use case is the easy part.

But Clean Architecture won't save you from bad engineering.
If you don't understand what you are abstracting away, it will become a problem in the long term.

If you want to go deeper, my flagship course, [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture),
takes the guesswork out of structuring your project the right way.
I share my entire framework for building robust applications from the ground up -
from building a rich domain model to creating use cases to getting your application ready for production.

And that's all for this week.

See you next Saturday.

---



<Sponsorships>

Build and test automations with [**UiPath**](https://rebrand.ly/dev-dives-uipath).
Join this free webinar to learn how the UiPath Platform enables you to build C# coded automations and quickly test them using GenAI.
[**Save your seat here**](https://rebrand.ly/dev-dives-uipath)!

Design and mock APIs with [**Postman Mock Servers**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/)!
Build higher-quality APIs faster with Postman mock API servers which simplify design and planning, support split-stack development,
and help you ensure that your API will run the way it's supposed to in production.
[**Learn more here**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/).

</Sponsorships>

Microservices have revolutionized how we build and scale applications.
By breaking down larger systems into smaller, independent services, we gain flexibility, agility, and the ability to adapt to changing requirements quickly.
However, microservices systems are also very dynamic.
Services can come and go, scale up or down, and even move around within your infrastructure.

This dynamic nature presents a significant challenge. How do your services find and communicate with each other reliably?

Hardcoding IP addresses and ports is a recipe for fragility.
If a service instance changes location or a new instance spins up, your entire system could grind to a halt.

Service discovery acts as a central directory for your microservices.
It provides a mechanism for services to register themselves and discover the locations of other services.

In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.

## What is Service Discovery?

Service discovery is a pattern that allows developers to use logical names to refer to external services instead of physical IP addresses and ports.
It provides a centralized location for services to register themselves.
Clients can query the service registry to find out the service's physical address.
This is a common pattern in large-scale distributed systems, such as Netflix and Amazon.

Here's what the service discovery flow looks like:

1. The service will register itself with the service registry
2. The client must query the service registry to get the physical address
3. The client sends the request to the service using the resolved physical address

<Image
  src="blogs/mnw_097/service_discovery_flow.png"
  width={1377}
  height={983}
  alt="Service discovery flow."
/>

The same concept applies when we have multiple services we want to call.
Each service would register itself with the service registry.
The client uses a logical name to reference a service and resolves the physical address from the service registry.

<Image
  src="blogs/mnw_097/service_discovery_microservices.png"
  width={2074}
  height={883}
  alt="Service discovery with multiple microservices."
/>

The most popular solutions for service discovery are Netflix [Eureka](https://github.com/Netflix/eureka) and HashiCorp [Consul](https://www.consul.io/).

There is also a lightweight solution from Microsoft in the `Microsoft.Extensions.ServiceDiscovery` library.
It uses application settings to resolve the physical addresses for services, so some manual work is still required.
However, you can store service locations in [Azure App Configuration](https://azure.microsoft.com/en-us/products/app-configuration) for a centralized service registry.
I will explore this service discovery library in some future articles.

But now I want to show you how to integrate Consul with .NET applications.

## Setting Up the Consul Server

The simplest way to run the Consul server locally is using a Docker container.
You can create a container instance of the `hashicorp/consul` image.

Here's an example of configuring the Consul service as part of the `docker-compose` file:

```yml
consul:
  image: hashicorp/consul:latest
  container_name: Consul
  ports:
    - '8500:8500'
```

If you navigate to `localhost:8500`, you will be greeted by the Consul Dashboard.

<Image
  src="blogs/mnw_097/consul_dashboard.png"
  width={1902}
  height={907}
  alt="Consul dashboard."
/>

Now, let's see how to register our services with Consul.

## Service Registration in .NET With Consul

We'll use the [Steeltoe Discovery](https://docs.steeltoe.io/api/v3/discovery/) library to implement service discovery with Consul.
The Consul client implementation lets your applications register services with a Consul server and discover services registered by other applications.

Let's install the `Steeltoe.Discovery.Consul` library:

```powershell
Install-Package Steeltoe.Discovery.Consul
```

We have to configure some services by calling `AddServiceDiscovery` and explicitly configuring the Consul service discovery client.
The alternative is calling `AddDiscoveryClient` which uses reflection at runtime to determine which service registry is available.

```csharp {6}
using Steeltoe.Discovery.Client;
using Steeltoe.Discovery.Consul;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddServiceDiscovery(o => o.UseConsul());

var app = builder.Build();

app.Run();
```

Finally, our service can register with Consul by configuring the logical service name through application settings.
When the application starts, the `reporting-service` logical name will be added to the Consul service registry.
Consul will store the respective physical address of this service.

```json
{
  "Consul": {
    "Host": "localhost",
    "Port": 8500,
    "Discovery": {
      "ServiceName": "reporting-service",
      "Hostname": "reporting-api",
      "Port": 8080
    }
  }
}
```

When we start the application and open the Consul dashboard, we should be able to see the `reporting-service` and its respective physical address.

<Image
  src="blogs/mnw_097/consul_dashboard_with_service.png"
  width={1902}
  height={907}
  alt="Consul dashboard with registered service."
/>

## Using Service Discovery

We can use service discovery when making HTTP calls with an `HttpClient`.
Service discovery allows us to use a logical name for the service we want to call.
When sending a network request, the service discovery client will replace the logical name with a correct physical address.

In this example, we're configuring the base address of the `ReportingServiceClient` typed client to `http://reporting-service`
and adding service discovery by calling `AddServiceDiscovery`.

Load balancing is an optional step, and we can configure it by calling `AddRoundRobinLoadBalancer` or `AddRandomLoadBalancer`.
You can also configure a custom load balancing strategy by providing an `ILoadBalancer` implementation.

```csharp
builder.Services
    .AddHttpClient<ReportingServiceClient>(client =>
    {
        client.BaseAddress = new Uri("http://reporting-service");
    })
    .AddServiceDiscovery()
    .AddRoundRobinLoadBalancer();
```

We can use the `ReportingServiceClient` typed client like a regular `HttpClient` to make requests.
The service discovery client sends the request to the external service's IP address.

```csharp
app.MapGet("articles/{id}/report",
    async (Guid id, ReportingServiceClient client) =>
    {
        var response = await client
            .GetFromJsonAsync<Response>($"api/reports/article/{id}");

        return response;
    });
```

## Takeaway

Service discovery simplifies the management of microservices by automating service registration and discovery.
This eliminates the need for manual configuration updates, reducing the risk of errors.

Services can discover each other's locations on demand, ensuring that communication channels remain open even as the service landscape evolves.
By enabling services to discover alternative service instances in case of outages or failures, service discovery enhances the overall resilience of the microservices system.

Mastering service discovery gives you a powerful tool to build modern distributed applications.

You can grab the [source code for this example here](https://github.com/m-jovanovic/service-discovery-consul).

Thanks for reading, and I'll see you next week!

---



<Sponsorships>

Store secrets in your [**Postman Vault**](https://learning.postman.com/docs/sending-requests/postman-vault/postman-vault-secrets/)!
Postman Vault enables you to store sensitive data as vault secrets in your local instance of Postman.
This enables you to securely reuse sensitive data in your HTTP collections and requests.
[**Learn more and get started using Vault**](https://learning.postman.com/docs/sending-requests/postman-vault/postman-vault-secrets/)!

[**IronPDF**](https://ironpdf.com/?utm_source=milan&utm_medium=email&utm_campaign=milan):
Your essential C# PDF library for converting modern HTML templates into PDFs effortlessly.
With advanced CSS styling preservation and seamless integration with ASP.NET Core MVC and Razor, IronPDF ensures pixel-perfect results.
[**Try it today**](https://ironpdf.com/?utm_source=milan&utm_medium=email&utm_campaign=milan)!

</Sponsorships>

I'll never forget when I was working on a project that required generating weekly sales reports for a client.
The initial solution involved a clunky process of exporting data, manipulating it in spreadsheets, and manually creating PDFs.
It was tedious, error-prone, and it sucked up way too much of my time.
I knew there had to be a better way.

That's when I discovered the power of combining Razor views with HTML-to-PDF conversion.
You have more control over formatting the document.
You can use modern CSS to style the HTML markup, which will be applied when exporting to a PDF document.
It's also simple to implement in ASP.NET Core.

Here's what we'll cover:

- Understanding Razor views
- Converting Razor views to HTML
- HTML to PDF conversion in .NET
- Putting it all together with Minimal APIs

Let's dive in!

## Razor Views

Razor [views](https://learn.microsoft.com/en-us/aspnet/core/mvc/views)
are an HTML template with embedded [Razor](https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor) markup.
Razor allows you to write and execute .NET code inside a web page.
Views have a special `.cshtml` file extension.
They're commonly used in [ASP.NET Core MVC](https://learn.microsoft.com/en-us/aspnet/core/mvc/overview),
[Razor Pages](https://learn.microsoft.com/en-us/aspnet/core/razor-pages),
and [Blazor](https://learn.microsoft.com/en-us/aspnet/core/blazor).

However, you can define Razor views in a class library or ASP.NET Core Web API project.

You can use the `Model` object to pass in data to a Razor view.
Inside the `.cshtml` file, you'll specify the model type with the `@model` keyword.
In the example below, I'm specifying that the `Invoice` class is the model for this view.
You can access the model instance with the `Model` property in the view.

This is the `InvoiceReport.cshtml` view we'll use to generate a PDF invoice.

You can write CSS in the Razor view inline or reference a stylesheet.
I'm using the [Tailwind CSS](https://tailwindcss.com/) utility framework, which uses inline CSS.
I usually delegate this to a front-end engineer on my team so they can stylize the report as needed.

```csharp {4}
@using System.Globalization
@using HtmlToPdf.Contracts

@model HtmlToPdf.Contracts.Invoice

@{
    IFormatProvider cultureInfo = CultureInfo.CreateSpecificCulture("en-US");
    var subtotal = Model.LineItems.Sum(li => li.Price * li.Quantity).ToString("C", cultureInfo);
    var total = Model.LineItems.Sum(li => li.Price * li.Quantity).ToString("C", cultureInfo);
}

<script src="https://cdn.tailwindcss.com"></script>

<div class="min-w-7xl flex flex-col bg-gray-200 space-y-4 p-10">
    <h1 class="text-2xl font-semibold">Invoice #@Model.Number</h1>

    <p>Issued date: @Model.IssuedDate.ToString("dd/MM/yyyy")</p>
    <p>Due date: @Model.DueDate.ToString("dd/MM/yyyy")</p>

    <div class="flex justify-between space-x-4">
        <div class="bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2">
            <p class="font-medium">Seller:</p>
            <p>@Model.SellerAddress.CompanyName</p>
            <p>@Model.SellerAddress.Street</p>
            <p>@Model.SellerAddress.City</p>
            <p>@Model.SellerAddress.State</p>
            <p>@Model.SellerAddress.Email</p>
        </div>
        <div class="bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2">
            <p class="font-medium">Bill to:</p>
            <p>@Model.CustomerAddress.CompanyName</p>
            <p>@Model.CustomerAddress.Street</p>
            <p>@Model.CustomerAddress.City</p>
            <p>@Model.CustomerAddress.State</p>
            <p>@Model.CustomerAddress.Email</p>
        </div>
    </div>

    <div class="flex flex-col bg-white rounded-lg p-4 space-y-2">
        <h2 class="text-xl font-medium">Items:</h2>
        <div class="">
            <div class="flex space-x-4 font-medium">
                <p class="w-10">#</p>
                <p class="w-52">Name</p>
                <p class="w-20">Price</p>
                <p class="w-20">Quantity</p>
            </div>

            @foreach ((int index, LineItem item) in Model.LineItems.Select((li, i) => (i + 1, li)))
            {
                <div class="flex space-x-4">
                    <p class="w-10">@index</p>
                    <p class="w-52">@item.Name</p>
                    <p class="w-20">@item.Price.ToString("C", cultureInfo)</p>
                    <p class="w-20">@item.Quantity.ToString("N2")</p>
                </div>
            }
        </div>
    </div>

    <div class="flex flex-col items-end bg-gray-50 space-y-2 p-4 rounded-lg">
        <p>Subtotal: @subtotal</p>
        <p>Total: <span class="font-semibold">@total</span></p>
    </div>
</div>
```

## Converting Razor Views to HTML

The next thing we'll need is a way to convert the Razor view into HTML.
We can do this with the [`Razor.Templating.Core`](https://github.com/soundaranbu/Razor.Templating.Core) library.
It provides a simple API to render a `.cshtml` file into a `string`.

```powershell
Install-Package Razor.Templating.Core
```

You can use the `RazorTemplateEngine` static class to call the `RenderAsync` method.
It accepts the path to the Razor view and the model instance that will be passed to the view.

Here's what that will look like:

```csharp
Invoice invoice = invoiceFactory.Create();

string html = await RazorTemplateEngine.RenderAsync(
    "Views/InvoiceReport.cshtml",
    invoice);
```

Alternatively, you can use the `IRazorTemplateEngine` instead of the static class.
In that case, you must call `AddRazorTemplating` to register the required services with DI.
This is also required if you want to use dependency injection inside the Razor views with `@inject`.
It's recommended that you call `AddRazorTemplating` after registering all dependencies.

```csharp
services.AddRazorTemplating();
```

## HTML to PDF conversion

Now that we've converted our Razor view into HTML, we can use it to generate a PDF report.
Many libraries offer this functionality.
The library I've used most often is [IronPDF](https://ironpdf.com/).
It's a paid library (and well worth it), but I know developers also want free options, so I'll list some alternatives at the end.

We can use IronPDF's `ChromePdfRenderer`, which uses an embedded Chrome browser.
The renderer exposes the `RenderHtmlAsPdf` method, which generates a `PdfDocument`.
Once you have the document, you can store it on the file system or export it as binary data.

```csharp
var renderer = new ChromePdfRenderer();

using var pdfDocument = renderer.RenderHtmlAsPdf(html);

pdfDocument.SaveAs($"invoice-{invoice.Number}.pdf");
```

If you're looking for free options, check out [Puppeteer Sharp](https://github.com/hardkoded/puppeteer-sharp).
It's a .NET port of the [Puppeteer](https://github.com/puppeteer/puppeteer) library, which allows you to run a headless Chrome browser.

Another (conditionally) free option to consider is [NReco.PdfGenerator](https://www.nuget.org/packages/NReco.PdfGenerator/).
However, it's only free for single-server deployments.

## Putting It All Together

Let's use everything we discussed to create a Minimal API endpoint to generate an invoice PDF report and return it as a file response.
Here's the code snippet:

```csharp
app.MapGet("invoice-report", async (InvoiceFactory invoiceFactory) =>
{
    Invoice invoice = invoiceFactory.Create();

    var html = await RazorTemplateEngine.RenderAsync(
        "Views/InvoiceReport.cshtml",
        invoice);

    var renderer = new ChromePdfRenderer();

    using var pdfDocument = renderer.RenderHtmlAsPdf(html);

    return Results.File(
        pdfDocument.BinaryData,
        "application/pdf",
        $"invoice-{invoice.Number}.pdf");
});
```

This is what the generated PDF report looks like:

<Image
  src="blogs/mnw_096/invoice.png"
  width={1273}
  height={1937}
  alt="Invoice report."
/>

You can grab the source code for this sample [here](https://github.com/m-jovanovic/razor-view-html-to-pdf).
Feel free to try out a different library for HTML to PDF conversion.

## Summary

In this article, we've explored the power of using Razor views for flexible PDF reporting in .NET.
We've seen how to create report templates with Razor views, convert them to HTML, and then transform that HTML into beautifully formatted PDF documents.

Whether you need to generate invoices, sales reports, or any other kind of structured document, this approach offers a simple and customizable solution.

Here's what you can explore next:

- [PDF Reporting in .NET Using IronPDF and Razor Views](https://youtu.be/XYdcdVWsWos)
- [Creating PDF Reports With QuestPDF](how-to-easily-create-pdf-documents-in-aspnetcore)

That's all for this week. Stay awesome!

---



<Sponsorships>

[**EF Core is too slow?**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter)
Discover how you can easily insert 14x faster (reducing saving time by 94%).
Boost your performance with our method integrated within EF Core: Bulk Insert, update, delete, and merge.
Join 5,000+ satisfied customers who have trusted our library since 2014.
[**Click here to learn more**](https://entityframework-extensions.net/?utm_source=milanjovanovic&utm_medium=newsletter).

Free eBook: [**Blazor Hybrid and Web in One Solution**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-ebook).
Dive into Blazor cross-platform apps, potential roadblocks, testing with Razor components, and more.
[**Get the eBook here**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-hybrid-ebook)!

</Sponsorships>

When you're dealing with thousands or even millions of records, efficiency is king.
That's where [**EF Core bulk update**](how-to-use-the-new-bulk-update-feature-in-ef-core-7) capabilities come into play.

EF Core 7 introduced two powerful new methods, `ExecuteUpdate` and `ExecuteDelete`.
They're designed to simplify bulk updates in your database.
Both methods have their respective async overloads - `ExecuteUpdateAsync` and `ExecuteDeleteAsync`.
[EF bulk updates](https://learn.microsoft.com/en-us/ef/core/saving/execute-insert-update-delete) offer significant performance advantages over traditional approaches.

However, there's an **important caveat**: these bulk operations bypass the EF Core **Change Tracker**.
This disconnect can lead to unexpected behavior if you're not aware of it.

In this week's issue, we'll dive into the details of bulk updates in EF Core.

## Understanding the EF Core ChangeTracker

When you load entities from the database with EF Core, the `ChangeTracker` starts tracking them.
As you update properties, delete entities, or add new ones, the `ChangeTracker` records these changes.

```csharp
using (var context = new AppDbContext())
{
    // Load a product
    var product = context.Products.FirstOrDefault(p => p.Id == 1);
    product.Price = 99.99; // Modify a property

    // At this point, the ChangeTracker knows that 'product' has been modified

    // Add a new product
    var newProduct = new Product { Name = "New Gadget", Price = 129.99 };
    context.Products.Add(newProduct);

    // Delete a product
    context.Products.Remove(product);

    context.SaveChanges(); // Persist all changes to the database
}
```

When you call `SaveChanges`, EF Core uses the `ChangeTracker` to determine which SQL commands to execute.
This ensures that the database is perfectly synchronized with your modifications.
The `ChangeTracker` acts as a bridge between your in-memory object model and your database.

If you're already familiar with how EF Core works, this serves mostly as a reminder.

## Bulk Updates and the ChangeTracker Disconnect

Now, let's focus on how [**bulk updates in EF Core**](how-to-use-the-new-bulk-update-feature-in-ef-core-7)
interact with the `ChangeTracker` - or rather, how they don't interact with it.
This design decision might seem counterintuitive, but there's a solid reason behind it: **performance**.

By directly executing SQL statements against the database, EF Core eliminates the overhead of tracking individual entity modifications.

```csharp
using (var context = new AppDbContext())
{
    // Increase price of all electronics by 10%
    context.Products
        .Where(p => p.Category == "Electronics")
        .ExecuteUpdate(
            s => s.SetProperty(p => p.Price, p => p.Price * 1.10));

    // In-memory Product instances with Category == "Electronics"
    // will STILL have their old price
}
```

In this example, we're increasing the price of all products in the `Electronics` category by 10%.
The `ExecuteUpdate` method efficiently translates the operation into a single SQL `UPDATE` statement.

```sql
UPDATE [p]
SET [p].[Price] = [p].[Price] * 1.10
FROM [Products] as [p];
```

However, if you inspect the `Product` instances that EF Core has already loaded into memory, you'll find that their `Price` properties haven't changed.
This might seem surprising if you aren't aware of how bulk updates interact with the change tracker.

Everything we discussed up to this point also applies to the `ExecuteDelete` method.

[**EF Core interceptors**](how-to-use-ef-core-interceptors) do not trigger for `ExecuteUpdate` and `ExecuteDelete` operations.
If you need to track or modify bulk update operations, you can create database triggers that fire whenever a relevant table is updated or deleted.
This allows you to log details and perform additional actions.

## The Problem: Maintaining Consistency

If `ExecuteUpdate` completes successfully, the changes are directly committed to the database.
This is because bulk operations bypass the `ChangeTracker` and don't participate in the usual transaction managed by `SaveChanges`.

If `SaveChanges` subsequently fails due to an error (e.g., validation error, database constraint violation, connection issue),
you'll be in an inconsistent state.
The changes made by `ExecuteUpdate` are already persisted.
Any changes made "in memory" are lost.

The most reliable way to ensure consistency is to wrap both `ExecuteUpdate` and the operations that lead to `SaveChanges` in a transaction:

```csharp {2}
using (var context = new AppDbContext())
using (var transaction = context.Database.BeginTransaction())
{
    try
    {
        context.Products
            .Where(p => p.Category == "Electronics")
            .ExecuteUpdate(
                s => s.SetProperty(p => p.Price, p => p.Price * 1.10));

        // ... other operations that modify entities

        context.SaveChanges();

        transaction.Commit();
    }
    catch (Exception ex)
    {
        // You could also let the transaction go out of scope.
        // This would automatically rollback any changes.
        transaction.Rollback();

        // Proceed to handle the exception...
    }
}
```

If `SaveChanges` fails, the transaction will be rolled back, reverting the changes made by both `ExecuteUpdate` and any other operations within the transaction.
This keeps your database in a consistent state.

## Summary

EF Core bulk update features, `ExecuteUpdate` and `ExecuteDelete`, are invaluable tools for optimizing performance.
By bypassing the `ChangeTracker` and executing raw SQL directly, they deliver significant speed improvements compared to traditional methods.

However, it's crucial to be mindful of the potential pitfalls associated with this approach.
The disconnect between in-memory entities and the database state can lead to unexpected results if not handled correctly.

My rule of thumb is to create an explicit [**database transaction**](working-with-transactions-in-ef-core) when I want to make additional entity changes.
We can be confident that all the changes will persist in the database or none of them will.

I hope this was helpful, and I'll see you next week.

**P.S.** Get the [source code](https://github.com/m-jovanovic/ef-bulk-updates) and try out the examples from this issue.

---


Previous issues of The .NET Weekly