
<Sponsorships>

Take Part in [**The State of Designer-Developer Collaboration 2024 Survey**](https://progress.co1.qualtrics.com/jfe/form/SV_1H2T9k9UuXgrkMK?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=dsk-awareness-design-dev-collab-survey-2024)!
Are you a web developer, a designer, or someone with a stake in the designer-developer collaboration process?
Then take this newly launched global survey.
[**We need your input**](https://progress.co1.qualtrics.com/jfe/form/SV_1H2T9k9UuXgrkMK?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=dsk-awareness-design-dev-collab-survey-2024)!

Comment on collections, folders, and requests using [**Postman**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/)!
Launch into hyper-efficiency using comments to discuss your work with your teammates in Postman.
[**Learn more here**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/)!

</Sponsorships>

As a .NET developer, I've spent countless hours working with external APIs.
It's a crucial part of modern software development, but let's be honest - it can be a real pain sometimes.

We've all been there, wrestling with `HttpClient`, writing repetitive code, and hoping we didn't miss a parameter or header somewhere.

That's why I want to introduce you to **Refit**, a library that's been a game-changer for me.

Imagine turning your API into a live interface - sounds too good to be true, right?
But that's exactly what Refit does.
It handles all the HTTP heavy lifting, letting you focus on what matters: your application logic.

In this article, I'll explain how Refit can transform the way you work with APIs in your .NET projects.

## What is Refit?

[Refit](https://github.com/reactiveui/refit) is a type-safe REST library for .NET.
It allows you to define your API as an interface, which Refit then implements for you.
This approach reduces boilerplate code and makes your API calls more readable and maintainable.

You describe your API endpoints using method signatures and attributes, and Refit takes care of the rest.

Let me break down why I find Refit so powerful:

- **Automatic serialization and deserialization**:
  You won't have to convert your objects to JSON and back.
  Refit handles all of that for you.
- **Strongly-typed API definitions**:
  Refit helps you catch errors early.
  If you mistype a parameter or use the wrong data type, you'll know at compile time, not when your app crashes in production.
- **Support for various HTTP methods**:
  GET, POST, PUT, PATCH, DELETE - Refit has you covered.
- **Request/response manipulations**:
  You can add custom headers or handle specific content types in a straightforward way.

But what I appreciate most about Refit is how it promotes clean, readable code.
Your API calls become self-documenting.
Anyone reading your code can quickly understand what each method does without diving into implementation details.

## Setting Up and Using Refit in Your Project

Let's set up Refit and see it in action using the [JSONPlaceholder API](https://jsonplaceholder.typicode.com/).
We'll implement a full CRUD interface and demonstrate its usage in a [**Minimal API application**](automatically-register-minimal-apis-in-aspnetcore).

First, install the required NuGet packages:

```powershell
Install-Package Refit
Install-Package Refit.HttpClientFactory
```

Now, let's create our Refit interface:

```csharp
using Refit;

public interface IBlogApi
{
    [Get("/posts/{id}")]
    Task<Post> GetPostAsync(int id);

    [Get("/posts")]
    Task<List<Post>> GetPostsAsync();

    [Post("/posts")]
    Task<Post> CreatePostAsync([Body] Post post);

    [Put("/posts/{id}")]
    Task<Post> UpdatePostAsync(int id, [Body] Post post);

    [Delete("/posts/{id}")]
    Task DeletePostAsync(int id);
}

public class Post
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string Body { get; set; }
    public int UserId { get; set; }
}
```

We define our `IBlogApi` interface with methods for all CRUD operations: GET (single and list), POST, PUT, and DELETE.
The `Post` class represents the structure of our blog posts.

Then you have to register Refit in your dependency injection container:

```csharp {4}
using Refit;

builder.Services
    .AddRefitClient<IBlogApi>()
    .ConfigureHttpClient(c => c.BaseAddress = new Uri("https://jsonplaceholder.typicode.com"));
```

Finally, we can use `IBlogApi` in our Minimal API endpoints:

```csharp
app.MapGet("/posts/{id}", async (int id, IBlogApi api) =>
    await api.GetPostAsync(id));

app.MapGet("/posts", async (IBlogApi api) =>
    await api.GetPostsAsync());

app.MapPost("/posts", async ([FromBody] Post post, IBlogApi api) =>
    await api.CreatePostAsync(post));

app.MapPut("/posts/{id}", async (int id, [FromBody] Post post, IBlogApi api) =>
    await api.UpdatePostAsync(id, post));

app.MapDelete("/posts/{id}", async (int id, IBlogApi api) =>
    await api.DeletePostAsync(id));
```

What I love about this setup is its simplicity.
We've created a fully functional API that communicates with an external service, all in just a few lines of code.
No manual HTTP requests, no raw JSON handling - Refit takes care of all that for us.

## Query Parameters and Route Binding

When working with APIs, you often need to send data as part of the URL, either in the route or as query parameters.
Refit makes this process simple and type-safe.

Let's extend our `IBlogApi` interface with some more complex scenarios:

```csharp {6,8}
public interface IBlogApi
{
    // Other methods omitted for brevity

    [Get("/posts")]
    Task<List<Post>> GetPostsAsync([Query] PostQueryParameters parameters);

    [Get("/users/{userId}/posts")]
    Task<List<Post>> GetUserPostsAsync(int userId);
}

public class PostQueryParameters
{
    public int? UserId { get; set; }
    public string? Title { get; set; }
}
```

Let's break this down:

- `GetPostsAsync` uses an object to represent query parameters.
  This approach is excellent for endpoints with many optional parameters.
  Refit will automatically convert this object into a query string.
- `GetUserPostsAsync` demonstrates passing in route parameters (`userId`) directly.

Using an object for query parameters makes your code type-safe and [**refactoring-friendly**](5-awesome-csharp-refactoring-tips).
If you need to add a new query parameter, you just add a property to `PostQueryParameters`.
Your existing code won't break, and your IDE can help you discover the new options.

## Dynamic Headers and Authentication

Another common requirement when integrating with APIs is including custom headers or authentication tokens with your requests.
Refit provides several ways to handle this, from simple static headers to dynamic, request-specific authentication.

Let's explore some scenarios:

```csharp
public interface IBlogApi
{
    [Headers("User-Agent: MyAwesomeApp/1.0")]
    [Get("/posts")]
    Task<List<Post>> GetPostsAsync();

    [Get("/secure-posts")]
    Task<List<Post>> GetSecurePostsAsync([Header("Authorization")] string bearerToken);

    [Get("/user-posts")]
    Task<List<Post>> GetUserPostsAsync([Authorize(scheme: "Bearer")] string token);
}
```

- You can add a static header to all requests by using the `Headers` attribute
- With the `Header` attribute, you can pass a header value dynamically as a parameter
- The `Authorize` attribute is a convenient way to add Bearer token authentication

But what if you need to add the same dynamic header to all requests?

That's where `DelegatingHandler` comes in handy.

You can learn more about [**using delegating handlers in this article**](extending-httpclient-with-delegating-handlers-in-aspnetcore).

I've found delegating handlers especially helpful in providing API keys, which are typically static.

## JSON Serialization Options

Refit gives you flexibility when choosing and configuring your JSON serializer.
By default, Refit uses `System.Text.Json`, is the built-in JSON serializer in modern .NET versions.

However, you can easily switch to `Newtonsoft.Json` if you need its features.

Here's how you can configure Refit to use it.

First, install the Newtonsoft.Json support package:

```powershell
Install-Package Refit.Newtonsoft.Json
```

Then, configure Refit to use Newtonsoft.Json:

```csharp {7-11}
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using Refit;

builder.Services.AddRefitClient<IBlogApi>(new RefitSettings
{
    ContentSerializer = new NewtonsoftJsonContentSerializer(new JsonSerializerSettings
    {
        ContractResolver = new CamelCasePropertyNamesContractResolver(),
        NullValueHandling = NullValueHandling.Ignore
    })
})
.ConfigureHttpClient(c => c.BaseAddress = new Uri("https://jsonplaceholder.typicode.com"));
```

This setup uses camel case for property names and ignores null values when serializing.

`System.Text.Json` is faster and uses less memory, making it a great default choice.
However, `Newtonsoft.Json` offers more features and might be necessary for compatibility with older systems or specific serialization needs.

## Handling HTTP Responses

While Refit's default behavior of automatically deserializing responses into your defined types is convenient, there are times when you need more control over the HTTP response.

Refit provides two options for these scenarios:
[`HttpResponseMessage`](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpresponsemessage) and `ApiResponse<T>`.

Let's update the `IBlogApi` to use these types:

```csharp
public interface IBlogApi
{
    [Get("/posts/{id}")]
    Task<HttpResponseMessage> GetPostRawAsync(int id);

    [Get("/posts/{id}")]
    Task<ApiResponse<Post>> GetPostWithMetadataAsync(int id);

    [Post("/posts")]
    Task<ApiResponse<Post>> CreatePostAsync([Body] Post post);
}
```

Now, let's break down how to use these, starting with `HttpResponseMessage`:

```csharp {1}
HttpResponseMessage response = await blogApi.GetPostRawAsync(1);

if (response.IsSuccessStatusCode)
{
    var content = await response.Content.ReadAsStringAsync();
    var post = JsonSerializer.Deserialize<Post>(content);
    Console.WriteLine($"Retrieved post: {post.Title}");
}
else
{
    Console.WriteLine($"Error: {response.StatusCode}");
}
```

This approach gives you full control over the HTTP response. You can access status codes, headers, and the raw content.
But you will have to deal with deserialization manually.

`ApiResponse<T>` is a Refit-specific type that wraps the deserialized content and response metadata.
It's a great middle ground when you need the typed response and access to headers or status codes.

Here's a more complex example using `ApiResponse<T>` for creating a post:

```csharp {3}
var newPost = new Post { Title = "New Post", Body = "Content", UserId = 1 };

ApiResponse<Post> createResponse = await blogApi.CreatePostAsync(newPost);

if (createResponse.IsSuccessStatusCode)
{
    var createdPost = createResponse.Content;
    var locationHeader = createResponse.Headers.Location;
    Console.WriteLine($"Created post with ID: {createdPost.Id}");
    Console.WriteLine($"Location: {locationHeader}");
}
else
{
    Console.WriteLine($"Error: {createResponse.Error.Content}");
    Console.WriteLine($"Status: {createResponse.StatusCode}");
}
```

This approach allows you to access the created resource, check specific headers like `Location`, and handle errors gracefully.

## Takeaway

Refit transforms the way we interact with APIs in .NET applications.
Converting your API into a strongly typed interface simplifies your code, enhances type safety, and improves maintainability.

The key Refit benefits we've explored include:

- Simplified API calls with automatic serialization and deserialization
- Flexible parameter handling for complex queries
- Easy management of headers and authentication
- Options for JSON serialization to fit your project's needs
- Granular control over HTTP responses when required

In my experience, Refit shines in projects of all sizes.
I've used it in small applications and large-scale microservices architectures.
It eliminates boilerplate code, reduces the risk of errors, and allows you to focus on your application's core logic rather than the intricacies of HTTP communication.

Remember, while Refit makes API interactions more straightforward, it's not a substitute for understanding the underlying principles of RESTful communication and HTTP.

That's all for today. Stay awesome, and I'll see you next week.

**P.S.** You can find the source code for this example in [**this repository**](https://github.com/m-jovanovic/refit-client-example).

---



<Sponsorships>

[**DDAL/EF**](https://ddal.io/), or **Dynamic Data Analysis Language** for Entity Framework Core,
is a powerful tool and simple language to retrieve formatted JSON data from a relational database.
You can use it to filter your data, retrieve complex lists and grids, and to fetch formatted data for your charts.
No SQL, no joins, no fuss: a flat syntax for simple yet powerful operations.
[**Learn more**](https://ddal.io/).

Reuse commonly used scripts and tests to packages in your team's [**Package Library**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/)
using Postman!
Add commonly-used scripts and tests to packages in your team's Package Library, and reuse them in your personal, private, and team workspaces using Postman!
[**Learn more here**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).

</Sponsorships>

I've been coding in .NET for years, but I never built an event sourced system.
Event sourcing has always intrigued me, though.
The idea of capturing every change and having a complete history of your data - it's fascinating.

So, I decided to dive in.
Not as an expert but as a curious developer.

In this newsletter, I'm sharing my journey into event sourcing.

- What is it really?
- Why does it matter?
- And how might it change the way we think about our .NET apps?

We'll look at the core concepts of event sourcing, potential benefits, and even some practical examples.

## What is Event Sourcing?

> Event Sourcing is an architectural design pattern where changes that occur in a domain are immutably stored as events in an append-only log.

_&mdash; [Event Store](https://www.eventstore.com/event-sourcing)_

When I first encountered event sourcing, it seemed complex.
But stripped down, it's a surprisingly simple idea: store changes, not just the current state.

Think of a bank account or wallet.
Normally, we'd just save the balance.
With event sourcing, we record every deposit and withdrawal.
The balance is then calculated from these events.

This diagram illustrates the difference:

<Image
  src="blogs/mnw_105/event_sourcing.png"
  width={2695}
  height={1467}
  alt="Event sourcing comparison to traditional data storage."
/>

This shift from storing state to storing events is the essence of event sourcing.
It's like keeping a detailed diary of your application's data rather than just a snapshot.
It's not just about where you are but how you got there.
For me, this was a lightbulb moment.

## Why Use Event Sourcing?

As I dug deeper into event sourcing, I kept asking myself: "Why would I use this instead of traditional data storage?"

Here's what I've discovered:

- **Full Audit Trail**: Every change is recorded.
  This is huge for businesses dealing with sensitive data or financial transactions.
  Imagine being able to trace every step of an order's journey or every modification to a user's account.
- **Debugging Time Machine**: With event sourcing, you can reconstruct the state of your application at any point in time.
  As a developer, this feels like a superpower.
  Tracking down bugs becomes less about guesswork and more about replay.
- **Business Insights**: All those stored events?
  They're a goldmine of data.
  You can analyze patterns, user behavior, or system performance in ways that might be impossible with just current-state data.
- **Flexibility**: Need to add a new feature that requires historical data?
  With event sourcing, it's already there.
  This flexibility could have saved me from many "I wish we had kept that information" moments.

Real-world use cases for event sourcing started to make sense:

- E-commerce platforms leveraging it for order tracking and inventory management.
- Financial systems use it for accurate transaction histories.
- IoT applications use it to analyze sensor data over time.

While it's not a silver bullet (what is in programming?), I'm beginning to see why so many developers are excited about event sourcing.
It's not just about storing data; it's about intent and behavior.

## Core Concepts And Practical Examples

As I started to implement event sourcing, understanding the core concepts became much easier with a concrete example.
Let's walk through a simple bank account scenario to see how event sourcing works.

### Events

Events are immutable records of something that happened.
In our bank account example, we might have events like `AccountOpened`, `MoneyDeposited`, and `MoneyWithdrawn`.

Here's how we might define these in C#:

```csharp
public record AccountOpened(Guid AccountId, DateTime OpenedAt);
public record MoneyDeposited(decimal Amount, DateTime DepositedAt);
public record MoneyWithdrawn(decimal Amount, DateTime WithdrawnAt);
```

[**Records**](records-anonymous-types-non-destructive-mutation) are a perfect fit for events, as they are immutable by design.

### State

In event sourcing, the current state is calculated by applying all events in order.

Here's how our `Account` class looks:

```csharp
public class Account
{
    public Guid Id { get; private set; }
    public decimal Balance { get; private set; }

    private List<object> _events = new List<object>();

    public Account(Guid id)
    {
        ApplyEvent(new AccountCreated(id));
    }

    public void Deposit(decimal amount)
    {
        ApplyEvent(new MoneyDeposited(amount));
    }

    public void Withdraw(decimal amount)
    {
        if (Balance >= amount)
        {
            ApplyEvent(new MoneyWithdrawn(amount));
        }
        else
        {
            throw new InvalidOperationException("Insufficient funds");
        }
    }

    private void ApplyEvent(object @event)
    {
        _events.Add(@event);

        switch (@event)
        {
            case AccountCreated e:
                Id = e.AccountId;
                Balance = 0;
                break;
            case MoneyDeposited e:
                Balance += e.Amount;
                break;
            case MoneyWithdrawn e:
                Balance -= e.Amount;
                break;
        }
    }
}
```

Notice how the `Account` class maintains its state.
Each method (`Deposit`, `Withdraw`) doesn't directly modify the balance.
Instead, it creates and applies an event.
The `ApplyEvent` method then updates the state based on these events.

### Event Store

In our simple example, we're using a list (`_events`) to store events.
In a real system, we would persist these events in a database.
The key principle remains: events are appended, never modified.

For production systems, there are specialized event sourcing databases like [EventStoreDB](https://www.eventstore.com/).

There's also [**Marten**](fast-document-database-in-net-with-marten), a .NET library that adds document database and event sourcing capabilities to PostgreSQL.

## Putting It All Together

Here's how we might use our event sourced `Account`:

- An action (like depositing money) triggers the creation of an event.
- The event is stored in the event store (in our simple example, it's just added to the `_events` list).
- The event is applied to update the current state of the `Account`.
- We can rebuild the state by replaying all events in order when needed.

```csharp
var account = new Account(Guid.NewGuid());
account.Deposit(100);
account.Withdraw(30);
account.Deposit(50);

Console.WriteLine($"Final balance: {account.Balance}"); // Output: Final balance: 120
```

We'd store these events in a database in a real event sourcing system.
This allows us to replay the events on demand to produce the current state.

## Challenges and Considerations

Since I started researching event sourcing, I've seen its potential and its hurdles.

Event sourcing itself is a simple idea.
However, the underlying complexity of this approach concerns me.
There's a significant learning curve from event sourcing basics to _applying event sourcing in production_.

It's not just about storing data differently.
It's a fundamental shift in how you model and think about your domain.
This complexity extends to the infrastructure level.
The .NET ecosystem isn't as mature when it comes to event sourcing tools and patterns.

Performance is another consideration that's often overlooked.
While appending events is typically fast, reconstructing the current state from a long history of events can be slow.
Real-world systems often need to implement caching strategies or snapshots to mitigate this.
Event sourcing is also eventually consistent on the read side.

One of the trickiest aspects I've encountered is event schema evolution (event versioning).
As your system grows and changes, so will your events.
Managing these changes without breaking existing event streams is a challenge that requires careful planning and design.
I'm still researching best practices.

## In Summary

Event sourcing has a steep learning curve, even for an experienced developer.
It requires a fundamental shift in how you think about data and system design.

If you want to give it a try, start small.
Implement a simple event-sourced system in a side project.
It's the best way to grapple with the concepts hands-on.
As you do, you might find that Domain-Driven Design (DDD) principles align well with event sourcing.

Remember, the goal isn't to use event sourcing everywhere but to understand where it can add value.

If you're ready to explore this topic further, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
There's an entire chapter on Event-Driven Architecture, which directly complements what you've learned about event sourcing here.

In a future newsletter, we'll explore a more real-world application of event sourcing.

Good luck out there, and I'll see you next week.

---



<Sponsorships>

Stop coding common elements!
[**Syncfusion**](https://www.syncfusion.com/?utm_source=milanjovanovicnewsletter&utm_medium=text&utm_campaign=esee_milanjovanovic_newsletter)
offers a developer's dream-over 1,800 prebuilt UI components for web, mobile, and desktop platforms.
Focus on your app's unique features, not repetitive UI tasks.
A free community license is also available for individual developers and startups.
[**Try Syncfusion for free today**](https://www.syncfusion.com/?utm_source=milanjovanovicnewsletter&utm_medium=text&utm_campaign=esee_milanjovanovic_newsletter).

Collaborate directly with your external partners in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/)
to help your partners consume your team's APIs and work together on API projects.
[**Check out Postman's Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/),
now available for Professional and Enterprise plans in Postman v11.

</Sponsorships>

If you were to glance at the folder structure of your system, could you tell what the system is about?
And here's a more interesting question.
Could a new developer on your team easily understand what the system does based on the folder structure?

Your architecture should communicate what problems it solves.
Organizing your system around use cases leads to a structure aligned with the business domain.
This approach is called **screaming architecture**.

[Screaming architecture](https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html) is a term coined by Robert Martin (Uncle Bob).
He argues that a software system's structure should communicate what the system is about.
He draws a parallel between looking at a blueprint for a building, where you can tell the purpose of the building based on the blueprint.

In this article, I want to show some practical examples and discuss the benefits of screaming architecture.

## A Use Case Driven Approach

A use case represents a specific interaction or task that a user wants to achieve within your system.
It encapsulates the business logic required to fulfill that task.
A use case is a high-level description of a user's goal.
For example, "reserving an apartment" or "purchasing a ticket".
It focuses on the _what_ of the system's behavior, not the _how_.

When you look at the folder structure and source code files of your system:

- Do they scream: Apartment Booking System or Ticketing System?
- Or do they scream ASP.NET Core?

Here's an example of a folder structure organized around technical concerns:

```powershell
📁 Api/
|__ 📁 Controllers
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Services
    |__ #️⃣ ApartmentService.cs
    |__ #️⃣ BookingService.cs
    |__ ...
|__ 📁 Models
```

Somewhere inside these folders, we'll find concrete classes that contain the system's behavior.
You'll notice that the cohesion with this folder structure is low.

How does screaming architecture help?

A use case driven approach will place the system's use cases as the top-level concept.
I also like to group related use cases into a top-level feature folder.
Inside a use case folder, we may find technical concepts required to implement it.

[**Vertical slice architecture**](vertical-slice-architecture) also approaches this from a similar perspective.

```powershell
📁 Api/
|__ 📁 Apartments
    |__ 📁 ReserveApartment
    |__ ...
|__ 📁 Bookings
    |__ 📁 CancelBooking
    |__ ...
|__ 📁 Payments
|__ 📁 Reviews
|__ 📁 Disputes
|__ 📁 Invoicing
```

The use case driven folder structure helps us better understand user needs and aligns development efforts with business goals.

## Screaming Architecture Benefits

The benefits of organizing our system around use cases are:

- Improved cohesion since related use cases are close together
- High coupling for a single use case and its related use cases
- Low coupling between unrelated use cases
- Easier navigation through the solution

## Bounded Contexts and Vertical Slices

We have many techniques for discovering the high-level modules within our system.
For example, we could use [event storming](https://www.eventstorming.com/) to explore the system's use cases.
Domain exploration happens before we write a single line of code.

The next step is decomposing the larger problem domain into smaller sub-domains and later bounded contexts.
This gives us loosely coupled high-level modules that we can translate into code.

<Image
  src="blogs/mnw_104/bounded_contexts.png"
  width={1060}
  height={187}
  alt="Bounded contexts."
/>

The overarching idea here is thinking about cohesion around functionalities.
We want to organize our system so that the cohesion between the components is high.
Bounded contexts, vertical slices, and screaming architecture are complementary concepts.

Here's a screaming architecture example for this system.
Let's say the `Ticketing` module uses [**Clean Architecture**](clean-architecture-folder-structure) internally.
But we can still organize the system around feature folders and use cases.
An alternative approach could be organizing around [**vertical slices**](vertical-slice-architecture-structuring-vertical-slices), resulting in a less nested folder structure.

```powershell
📁 Modules/
|__ 📁 Attendance
    |__ ...
|__ 📁 Events
    |__ ...
|__ 📁 Ticketing
    |__ 📁 Application
        |__ 📁 Carts
            |__ 📁 AddItemToCart
            |__ 📁 ClearCart
            |__ 📁 GetCart
            |__ 📁 RemoveItemFromCart
        |__ 📁 Orders
            |__ 📁 SubmitOrder
            |__ 📁 CancelOrder
            |__ 📁 GetOrder
        |__ 📁 Payments
            |__ 📁 RefundPayment
        |__ ...
    |__ 📁 Domain
        |__ 📁 Customers
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
    |__ 📁 infrastructure
        |__ 📁 Authentication
        |__ 📁 Customers
        |__ 📁 Database
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
|__ 📁 Users
    |__ ...
```

The example above is a small part of the system I built inside of [**Modular Monolith Architecture**](/modular-monolith-architecture).

## Takeaway

**Screaming Architecture** isn't just a catchy phrase, it's an approach that can profoundly impact how you build software.
By organizing your system around use cases, you align your codebase with the core business domain.
Your system exists to solve the business domain problems.

Remember, the goal is to create a system that communicates its purpose through its structure.
Embrace a use case-driven approach, break down complex domains into bounded contexts.
Build a system that truly "screams" about the problems it solves.

If you want to explore these powerful ideas further, check out [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture).
I share my entire framework for building robust applications from the ground up and organizing the system around use cases.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**IronSecureDoc**](https://ironsoftware.com/enterprise/securedoc/?utm_source=milan&utm_medium=email&utm_campaign=milan) -
the PDF security & compliance server.
Protect your PDFs with IronSecureDoc, the latest from Iron Software.
Redact sensitive data, add digital signatures, and stay compliant with PDF/A & PDF/UA standards.
Easy integration with our REST API.
[**Learn more here**](https://ironsoftware.com/enterprise/securedoc/?utm_source=milan&utm_medium=email&utm_campaign=milan).

Test your API using the [**Collection Runner**](https://learning.postman.com/docs/collections/running-collections/intro-to-collection-runs/).
Use Collection Runner to log the test results for your requests, and use scripts to pass data between requests and change the request workflow.
[**Check it out here**](https://learning.postman.com/docs/collections/running-collections/intro-to-collection-runs/).

</Sponsorships>

Have you ever wondered how large-scale systems handle traffic spikes or maintain performance even when parts of the system are temporarily down?
The answer lies in asynchronous messaging.

Asynchronous messaging is, at its core, about decoupling.
Our components can operate independently and communicate through a message queue or topic.
If one service (component) is temporarily unavailable, the others can continue working.
This improves our system's scalability, resilience, and fault tolerance.

In this article, we'll explore how to use Amazon SQS and SNS for asynchronous messaging in .NET applications.

We'll also see how MassTransit simplifies the process, enabling you to build robust message-driven systems.

Let's dive in.

## What is Amazon SQS?

[Amazon Simple Queue Service](https://aws.amazon.com/sqs/) (SQS) is a fully managed message queueing service.
It facilitates the decoupling and scaling of microservices and distributed systems.

SQS acts as a reliable middleman for asynchronous communication.
It enables different components of your architecture to exchange messages without needing to be online or directly connected at the same time.
Messages are stored in queues and consumed on demand.

SQS offers two distinct queue types depending on your requirements:

- **Standard Queues**: Ideal for high-throughput scenarios.
  Standard Queues provide at-least-once delivery and best-effort ordering.
- **FIFO Queues**: Recommended when maintaining message order is required.
  FIFO Queues guarantee exactly-once processing and preserve the sequence in which messages are sent.

Let's say we have two services - `Stock` and `Reporting`.
When a user creates a purchase order in the `Stock` service, we want to notify the `Reporting` service.

SQS allows us to create decoupled communication between these services.
The `Stock` service sends a message to an SQS queue, and the `Reporting` service can poll from the queue to consume messages.

<Image
  src="blogs/mnw_103/amazon_sqs.png"
  width={1920}
  height={1080}
  alt="Amazon SQS."
/>

It's interesting to highlight that SQS uses a polling mechanism for message consumers.
When a consumer polls for new messages, SQS starts a [visibility timeout](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html).
SQS doesn't automatically delete the messages.
Instead, messages are hidden from other consumers until the timeout expires.

When the consumer successfully processes a message, it's removed from the queue.
But if the visibility timeout expires, the message becomes visible and can be delivered again.
Other consumers can receive this message when polling from SQS.
This is why SQS offers at-least-once delivery (for standard queues).
You will have to implement [**idempotency in the consumer**](idempotent-consumer-handling-duplicate-messages).

## Amazon SQS and Competing Consumers

Let's introduce another service into our system - the `Risk Management` service.
When multiple consumers (services) poll an SQS queue, each wants to retrieve and process messages as they become available.
However, once a message is successfully received and processed by one consumer, it's removed from the queue.

Why is this a problem?

Other consumers who might have been polling will miss out on that specific message.
This is known as [competing consumers](https://learn.microsoft.com/en-us/azure/architecture/patterns/competing-consumers).

Let's consider the example of `Reporting` and `Risk Management` services polling the same queue.
If a new message arrives, only one of these services will "win" the race and retrieve it for processing.
The other service won't find that message even if it polls moments later.

<Image
  src="blogs/mnw_103/amazon_sqs_competing_consumers.png"
  width={1920}
  height={1080}
  alt="Amazon SQS and competing consumers."
/>

So, how can we solve this?

We could introduce a dedicated queue for each service.
However, the producer now needs to publish to multiple queues.
This creates a possibility for some (not so) interesting partial failures.
What happens if we successfully publish to one queue but fail to publish to the other?

You can see how this becomes difficult to scale while maintaining reliability.

Luckily, there's a solution.

## Amazon SNS to The Rescue

[Amazon Simple Notification Service](https://aws.amazon.com/sns/) (SNS) is a fully managed pub/sub messaging service.
It allows publishers to send messages to multiple subscribers (topics) simultaneously.

SNS operates on the principle of publishers and subscribers.
Publishers send messages to an SNS topic, while subscribers express interest in specific topics and receive messages published to those topics.
This decoupled architecture allows you to add or remove subscribers without impacting the publisher or other subscribers.

SNS seamlessly integrates with SQS, allowing you to create a powerful combination where SNS handles the fan-out of messages,
and SQS queues ensure that each message is processed exclusively by a single consumer (service).

Instead of sending a message to the queue, the `Stock` service now publishes to an SNS topic.
Both the `Reporting` and `Risk Management` services create their own SQS queues and subscribe these queues to the SNS topic.
When a new message is published to the SNS topic, SNS delivers it to both SQS queues.
Each queue receives its own copy of the message.

If we want to introduce a new service, we'll create a new SQS queue and subscribe it to the topic.

<Image
  src="blogs/mnw_103/amazon_sns.png"
  width={1920}
  height={1080}
  alt="Amazon SNS."
/>

## MassTransit Integration With SQS and SNS

How can we use SNS and SQS from a .NET application?

You could use the official AWS SDKs.
The benefit is you'll have more control over messaging.
However, you will need to write more code to receive and handle messages successfully.

So, I want to suggest a different approach.

MassTransit is one of the most popular messaging libraries in .NET.
It provides a set of messaging abstractions on top of the supported message transports.

I wrote an article about [**using MassTransit with RabbitMQ and Azure Service Bus**](using-masstransit-with-rabbitmq-and-azure-service-bus).

But we'll focus on using MassTransit with SQS and SNS.

Let's start by installing the NuGet package we'll need:

```powershell
Install-Package MassTransit.AmazonSQS
```

Next, we'll need to configure MassTransit with our .NET applications.

Here's the MassTransit configuration for the `Stock` service:

```csharp
builder.Services.AddMassTransit(configure =>
{
    configure.AddConsumer<PurchaseOrderSentConsumer>().Endpoint(e => e.InstanceId = "stocks");

    configure.UsingAmazonSqs((context, cfg) =>
    {
        cfg.Host("eu-central-1", h =>
        {
            h.AccessKey(builder.Configuration["AmazonSqs:AccessKey"]!);
            h.SecretKey(builder.Configuration["AmazonSqs:SecretKey"]!);

            h.Scope("stocks-platform", scopeTopics: true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("stocks-platform-", false));
    });
});
```

Here are a few things I want to highlight here:

- `UsingAmazonSqs` - Configures SQS (and SNS) as the message transport.
- `Scope` - Adds a prefix to SNS topic names. This helps distinguish topics from other applications and environments.
- `Endpoint` - Sets the `InstanceId`, which is appended to the endpoint (queue) name.
- `ConfigureEndpoints` - Allows us to specify a prefix for endpoint (queue) names. The idea is the same as the topic name prefix.

You'll also need to configure an [IAM policy](https://masstransit.io/documentation/configuration/transports/amazon-sqs#example-iam-policy)
that gives MassTransit the required permissions for AWS resources.

And with this setup in place, you can use MassTransit to publish messages.
Here's an endpoint that accepts a purchase order and publishes a `PurchaseOrderSent` message.

```csharp
app.MapPost("purchase-orders", async (PurchaseOrderRequest request, IPublishEndpoint publishEndpoint) =>
{
    var purchaseOrder = new PurchaseOrder
    {
        Id = Guid.NewGuid(),
        Ticker = request.Ticker,
        LimitPrice = request.LimitPrice,
        Quantity = request.Quantity
    };

    OrdersDb.Add(purchaseOrder);

    await publishEndpoint.Publish(new PurchaseOrderSent(purchaseOrder.Id));

    return Results.Ok(purchaseOrder);
});
```

We will process the purchase order in the `PurchaseOrderSentConsumer`.
If it's successfully processed (filled), we will publish an `OrderFilled` message.

The `Risk Management` service can subscribe to this message using MassTransit.
The configuration is almost identical, with the only difference being the endpoint's `InstanceId`.

```csharp
builder.Services.AddMassTransit(configure =>
{
    configure.AddConsumer<OrderFilledConsumer>().Endpoint(e => e.InstanceId = "risk-management");

    configure.UsingAmazonSqs((context, cfg) =>
    {
        cfg.Host("eu-central-1", h =>
        {
            h.AccessKey(builder.Configuration["AmazonSqs:AccessKey"]!);
            h.SecretKey(builder.Configuration["AmazonSqs:SecretKey"]!);

            h.Scope("stocks-platform", true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("stocks-platform-", false));
    });
});
```

## Broker Topology in AWS

MassTransit will automatically create the required queues, topics, and subscriptions in AWS.
If needed, you can further configure the SQS and SNS resources.

Of course, you can also create the required infrastructure in AWS and tell MassTransit to use it.

But let's keep it simple and allow MassTransit to provision the AWS resources.

By default, MassTransit creates standard SQS queues.
Here's what we get in Amazon SQS.

<div className="bordered">
  <Image
    src="blogs/mnw_103/amazon_sqs_queues.png"
    width={1070}
    height={720}
    alt="Amazon SQS queues created by MassTransit."
  />
</div>

And here's what we get in Amazon SNS:

<div className="bordered">
  <Image
    src="blogs/mnw_103/amazon_sns_topics.png"
    width={1280}
    height={650}
    alt="Amazon SNS topics created by MassTransit."
  />
</div>

MassTransit automatically configures the required subscriptions between the topic and queues.

And we are ready to start publishing and consuming messages.

## In Summary

Asynchronous communication and decoupling are pivotal in achieving scalability, resilience, and fault tolerance.
Amazon SQS and SNS provide the building blocks of message-driven architectures in the AWS cloud.

We explored the core concepts of SQS and SNS, understanding how they enable reliable message delivery and fan-out capabilities.

MassTransit provides an excellent abstraction layer over SQS and SNS, simplifying development.
We can focus on solving the business problems and delivering value to our users.

The combination of Amazon SQS, SNS, and MassTransit gives us robust tools for building modern, event-driven applications.

Thanks for reading, and I'll see you next week!

**P.S.** You can find the source code for this article in [**this repository**](https://github.com/m-jovanovic/aws-tutorials), under the `Amazon SQS and SNS` folder.

---



<Sponsorships>

[**WPF or .NET MAUI - How to Choose**](https://www.telerik.com/blogs/wpf-net-maui-how-choose?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-general-awareness)?
Creating a desktop application and need to decide between the long-time reliable WPF and the up-and-coming cross-platform .NET MAUI?
[**This blog post**](https://www.telerik.com/blogs/wpf-net-maui-how-choose?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-general-awareness)
will help you explore the features that matter.

[**Postman v11**](https://blog.postman.com/announcing-postman-v11/) takes your dev collaboration to next level using AI and APIs.
Check out [**Postman v11**](https://blog.postman.com/announcing-postman-v11/) to get started with it's features designed to help bring API collaboration to the next level.

</Sponsorships>

Okay, let's be honest.
We all have a million things on our plates, and diving deep into every nook and cranny of EF Core might not be
at the top of your priority list.

But here's the deal: EF Core is powerful, and knowing a few key features can save you lots of time and frustration.

So, I won't bombard you with every single EF Core feature under the sun.

Instead, I've cherry-picked five essential ones that you really need to know.

We'll go through:

- **Query Splitting** - your database's new best friend
- **Bulk Updates and Deletes** - efficiency on steroids
- **Raw SQL Queries** - when you need to go rogue
- **Query Filters** - keeping things nice and tidy
- **Eager Loading** - because lazy isn't so great

Let's get started!

## Query Splitting

[**Query splitting**](how-to-improve-performance-with-ef-core-query-splitting) is one of those EF Core features that you rarely need.
Until one day, you do.
Query splitting is helpful in scenarios where you're eager loading multiple collections.
It helps us avoid the [cartesian explosion](https://learn.microsoft.com/en-us/ef/core/performance/efficient-querying#avoid-cartesian-explosion-when-loading-related-entities) problem.

Let's say we want to retrieve a department with all its teams and employees.
We might write a query like this:

```csharp
Department department =
    context.Departments
        .Include(d => d.Teams)
        .Include(d => d.Employees)
        .Where(d => d.Id == departmentId)
        .First();
```

This translates to a single SQL query with two JOINs.
However, since these `JOIN` statements are on the same level, the database will return a _cross product_.
Each row from `Teams` will be joined with each row `Employees`.
In that case, the database returns many rows, significantly impacting performance.

Here's how we can avoid these performance issues with query splitting:

```csharp
Department department =
    context.Departments
        .Include(d => d.Teams)
        .Include(d => d.Employees)
        .Where(d => d.Id == departmentId)
        .AsSplitQuery()
        .First();
```

With `AsSplitQuery`, EF Core will execute an additional SQL query for each collection navigation.

However, be cautious not to overuse query splitting.
I use split queries when I've _measured_ that they consistently perform better.

Split queries have more round trips to the database, which might be slower if database latency is high.
There is also no consistency guarantee across multiple SQL queries.

## Bulk Updates and Deletes

EF Core 7 added two new APIs for performing [**bulk updates and deletes**](how-to-use-the-new-bulk-update-feature-in-ef-core-7),
`ExecuteUpdate` and `ExecuteDelete`.
They allow you to efficiently update a large number of rows in one round trip to the database.

Here's a practical example.

The company has decided to give a 5% raise to all employees in the "Sales" department.
Without bulk updates, we might iterate through each employee and update their salary individually:

```csharp
var salesEmployees = context.Employees
    .Where(e => e.Department == "Sales")
    .ToList();

foreach (var employee in salesEmployees)
{
    employee.Salary *= 1.05m;
}

context.SaveChanges();
```

This approach involves multiple database roundtrips, which can be inefficient, especially for large datasets.

We can achieve the same in one roundtrip using `ExecuteUpdate`:

```csharp
context.Employees
    .Where(e => e.Department == "Sales")
    .ExecuteUpdate(s => s.SetProperty(e => e.Salary, e => e.Salary * 1.05m));
```

This executes a single SQL `UPDATE` statement, directly modifying the salaries in the database without loading entities into memory, giving us improved performance.

Here's another example.
Let's say an e-commerce platform wants to delete all shopping carts older than one year.

Here's how we could do this with `ExecuteDelete`:

```csharp
context.Carts
    .Where(o => o.CreatedOn < DateTime.Now.AddYears(-1))
    .ExecuteDelete();
```

This results in a single SQL `DELETE` statement, directly removing the old shopping carts from the database.

However, bulk updates bypass the EF change tracker.
This could be problematic, and I wrote about the [**caveats of bulk updates in this article**](what-you-need-to-know-about-ef-core-bulk-updates).

## Raw SQL Queries

EF Core 8 added a new feature that allows us to query unmapped types with raw SQL.

Suppose we want to retrieve data from a database view, stored procedure, or a table that doesn't directly correspond to any of our entity classes.

For example, we want to retrieve a sales summary for each product.
With EF Core 8, we can define a simple `ProductSummary` class representing the structure of the result set and query it directly:

```csharp
public class ProductSummary
{
    public int ProductId { get; set; }
    public string ProductName { get; set; }
    public decimal TotalSales { get; set; }
}

var productSummaries = await context.Database
    .SqlQuery<ProductSummary>(
        @$"""
        SELECT p.ProductId, p.ProductName, SUM(oi.Quantity * oi.UnitPrice) AS TotalSales
        FROM Products p
        JOIN OrderItems oi ON p.ProductId = oi.ProductId
        WHERE p.CategoryId = {categoryId}
        GROUP BY p.ProductId, p.ProductName
        """)
    .ToListAsync();
```

The `SqlQuery` method returns an `IQueryable`, which allows you to compose raw SQL queries with LINQ.
This combines the power of raw SQL with the expressiveness of LINQ.

Remember to use parameterized queries to prevent SQL injection vulnerabilities.
The `SqlQuery` method accepts a `FormattableString`, which means you can safely use an interpolated string.
Each argument is converted to a SQL parameter.

You can learn more about [**raw SQL queries in this article**](ef-core-raw-sql-queries).

## Query Filters

[**Query filters**](how-to-use-global-query-filters-in-ef-core) are like reusable `WHERE` clauses you can apply to your entities.
These filters are automatically added to LINQ queries whenever you retrieve entities of the corresponding type.
This saves you from repeatedly writing the same filtering logic in multiple places within your application.

Query Filters are commonly used for scenarios like:

- [**Soft Deletes**](implementing-soft-delete-with-ef-core): Filter out records marked as deleted.
- [**Multi-tenancy**](multi-tenant-applications-with-ef-core): Filter data based on the current tenant.
- Row-level security: Restrict access to certain records based on user roles or permissions.

In a multi-tenant application, you often need to filter data based on the current tenant.
Query filters allow us to handle this requirement easily:

```csharp
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    // Associate products with tenants
    public int TenantId { get; set; }
}

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    // The current TenantId is set based on the current request/context
    modelBuilder.Entity<Product>().HasQueryFilter(p => p.TenantId == _currentTenantId);
}

// Now, queries automatically filter based on the tenant:
var productsForCurrentTenant = context.Products.ToList();
```

Configuring multiple query filters on the same entity will only apply the last one.
You can combine multiple query filters using `&&` (AND) and `||` (OR) operators.

You can use `IgnoreQueryFilters` to bypass the filters in specific queries when needed.

## Eager Loading

Eager Loading is a feature in EF Core that allows you to load related entities along with your main entity in a single database query.
By fetching all necessary data in a single query, you can improve application performance.
This is especially true when dealing with complex object graphs or when lazy loading would result in many small, inefficient queries.

Here's an example `VerifyEmail` use case.
We want to load an `EmailVerificationToken` and eagerly load a `User` with the `Include` method because we want to modify both entities at the same time.

```csharp {6}
internal sealed class VerifyEmail(AppDbContext context)
{
    public async Task<bool> Handle(Guid tokenId)
    {
        EmailVerificationToken? token = await context.EmailVerificationTokens
            .Include(e => e.User)
            .FirstOrDefaultAsync(e => e.Id == tokenId);

        if (token is null || token.ExpiresOnUtc < DateTime.UtcNow || token.User.EmailVerified)
        {
            return false;
        }

        token.User.EmailVerified = true;

        context.EmailVerificationTokens.Remove(token);

        await context.SaveChangesAsync();

        return true;
    }
}
```

EF Core will generate a single SQL query that joins the `EmailVerificationToken` and `User` tables, retrieving all the necessary data in one go.

Eager loading (and query splitting, which we mentioned earlier) isn't a silver bullet.
Consider using projections if you only need specific properties from related entities to avoid fetching unnecessary data.

## Summary

So, there you have it!
Five EF Core features that, frankly, you can't afford _not_ to know.
Remember, mastering EF Core takes time, but these features provide a solid foundation to build upon.

Another piece of advice is to deeply understand how your database works.
Mastering SQL also allows you to get the most value from EF Core.

While we focused on five key features, there are many other EF Core features worth exploring:

- [**Optimistic concurrency control**](solving-race-conditions-with-ef-core-optimistic-locking)
- [**Database migrations**](efcore-migrations-a-detailed-guide)
- [**Compiled queries**](unleash-ef-core-performance-with-compiled-queries)
- [**Transactions**](working-with-transactions-in-ef-core)
- [**Interceptors**](how-to-use-ef-core-interceptors)

EF Core is continuously evolving, so keep an eye on the latest updates and releases to stay ahead.

Good luck out there, and see you next week.

---


<Sponsorships>

Take Postman wherever you work with the [**Postman VS Code extension**](https://learning.postman.com/docs/getting-started/basics/about-vs-code-extension)!
The Postman VS Code extension enables you to develop and test your APIs in Postman directly from Visual Studio Code and Visual Studio Code Insiders.
[**Try it out**](https://learning.postman.com/docs/getting-started/basics/about-vs-code-extension)!

Are you tired of wasting time and money on complex e-signature solutions?
[**BoldSign**](https://boldsign.com/esignature-api/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign)
offers an easy-to-use .NET SDK that seamlessly integrates e-signatures into your application.
Try our [**free e-signature API**](https://boldsign.com/esignature-api/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign) sandbox today!

</Sponsorships>

Writing good code is important for any software project.
It's also something I deeply care about.
However, it can be hard to spot problems by just reading through everything.

Luckily, there's a tool that can help: **static code analysis**.

It's like having an extra pair of eyes automatically checking your code.
Static code analysis helps you build secure, maintainable, and high-quality C# code.

Here's what we are going to cover in this week's newsletter:

- Static code analysis
- Static analysis in .NET
- Finding security risks

Let's see how static code analysis can help us improve our code quality.

## What is Static Code Analysis?

Static code analysis is a way to examine your code without actually running it.
It reports any issues related to security, performance, coding style, or best practices.

With static code analysis, you can ["shift left"](https://en.wikipedia.org/wiki/Shift-left_testing).
This allows you to find and fix issues early in the development process when they're less expensive to solve.

By writing high-quality code, you'll be able to build systems that are more reliable, scalable, and easier to maintain over time.
Investing in code quality will pay dividends in the later stages of any project.

You can integrate static code analysis into your [**CI pipeline**](how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet) for a quick feedback loop.
We can also pair this with [**architecture testing**](shift-left-with-architecture-testing-in-dotnet) to enforce additional coding standards.

## Static Code Analysis in .NET

.NET has built-in Roslyn analyzers that inspect your C# code for code style and quality issues.
Code analysis is enabled by default if your project targets .NET 5 or later.

The best way I found to configure static code analysis is using `Directory.Build.props`.
It's an XML file where you can configure common project properties.
You can place the `Directory.Build.props` file in the root folder so it will apply to all projects.

You can configure the `TargetFramework`, `ImplicitUsings`, `Nullable` (nullable reference types), etc.
But what we care about is configuring static code analysis.

Here are some properties we can configure:

- `TreatWarningsAsErrors` - Treat all warnings as errors.
- `CodeAnalysisTreatWarningsAsErrors` - Treat code quality (CAxxxx) warnings as errors.
- `EnforceCodeStyleInBuild` - Enables code-style analysis ("IDExxxx") rules.
- `AnalysisLevel` - Specifies which analyzers to enable. The default value is `latest`.
- `AnalysisMode` - Configures the predefined code analysis configuration.

We can also install additional NuGet packages to our projects.
`SonarAnalyzer.CSharp` contains additional code analyzers to help us write clean, safe, and reliable code.
This library comes from the same company that built [SonarQube](https://www.sonarsource.com/products/sonarqube/).

```xml {8-12,16}
<Project>
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    
    <AnalysisLevel>latest</AnalysisLevel>
    <AnalysisMode>All</AnalysisMode>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <CodeAnalysisTreatWarningsAsErrors>true</CodeAnalysisTreatWarningsAsErrors>
    <EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
  </PropertyGroup>

  <ItemGroup Condition="'$(MSBuildProjectExtension)' != '.dcproj'">
    <PackageReference Include="SonarAnalyzer.CSharp" Version="*">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>
        runtime; build; native; contentfiles; analyzers; buildtransitive
      </IncludeAssets>
    </PackageReference>
  </ItemGroup>
</Project>
```

The built-in .NET analyzers and the ones from `SonarAnalyzer.CSharp` can be very helpful.
But they can also make a lot of noise with too many build warnings.

When you encounter code analysis rules that you don't consider helpful, you can turn them off.
You can configure individual code analysis rules in the `.editorconfig` file.

```
# S125: Sections of code should not be commented out
dotnet_diagnostic.S125.severity = none

# S1075: URIs should not be hardcoded
dotnet_diagnostic.S1075.severity = none

# S2094: Classes should not be empty
dotnet_diagnostic.S2094.severity = none

# S3267: Loops should be simplified with "LINQ" expressions
dotnet_diagnostic.S3267.severity = none
```

## Finding (and Fixing) Security Risks

Static code analysis can help you detect potential security vulnerabilities in your code.
Here's an example of a `PasswordHasher` using only `10,000` iterations to generate a password hash.
The `S5344` rule, from `SonarAnalyzer.CSharp`, detects this issue and warns us.
The recommended minimal number of iterations is `100,000`.

You can navigate to the explanation for [S5344](https://rules.sonarsource.com/csharp/RSPEC-5344/) to learn more:

> Weakly hashed password storage poses a significant security risk to software applications.

With `TreatWarningsAsErrors` turned on, your build will fail until you solve this issue.
This reduces the chance of introducing security risks in production.

<div className="bordered">
  <Image
    src="blogs/mnw_101/static_code_analysis.png"
    width={1318}
    height={830}
    alt="Example of static code analysis warning."
  />
</div>

## Conclusion

Static code analysis is a powerful tool I include in all my C# projects.
It helps me catch problems early, leads to more reliable and secure code, and saves time and effort.
While the initial setup and fine-tuning of rules might take some time, the long-term benefits are undeniable.

Remember, static code analysis is a tool that complements your existing development practices.

You can create a robust development process that consistently delivers high-quality software by combining
static code analysis with other techniques like code reviews, unit testing, and continuous integration.

Embrace static code analysis.
Your future self (and your team) will thank you.

That's all for today.

See you next week.

**P.S.** Here's a sample [.editorconfig](https://gist.github.com/m-jovanovic/417b7d0a641d7dd7d1972550fba298db) file you can add to your projects and customize to fit your needs.

---


<Sponsorships>

[**API Collaboration**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/)
has never been easier with comments on collections, folders, and requests using Postman!
Launch into hyper-efficiency using comments to discuss your work with your teammates in Postman.
[**Learn more here**](https://learning.postman.com/docs/collaborating-in-postman/working-with-your-team/discussing-your-work/).

Build automations with [**UiPath Autopilot™**](https://rebrand.ly/dev-dives-webinar).
Join this free webinar to learn how to build automation workflows, generate code, and create ready-to-use apps by simply describing your needs to the AI-powered Autopilot.
[Register today](https://rebrand.ly/dev-dives-webinar)!

</Sponsorships>

Redis is a popular choice for [**caching data**](caching-in-aspnetcore-improving-application-performance), but its capabilities go far beyond that.
One of its lesser-known features is Pub/Sub support.
Redis channels offer an interesting approach for implementing real-time messaging in your .NET applications.
However, as you'll soon see, channels also have some drawbacks.

In this week's newsletter, we'll explore:

- Basics of Redis channels
- Practical use cases for channels
- Implementing a Pub/Sub example in .NET
- Cache invalidation in distributed systems

Let's dive in.

## Redis Channels

Redis channels are named communication channels that implement the [Publish/Subscribe messaging paradigm](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern).
Each channel is identified by a unique name (e.g., `notifications`, `updates`).
Channels facilitate message delivery from publishers to subscribers.

Publishers use the `PUBLISH` command to send messages to a specific channel.
Subscribers use the `SUBSCRIBE` command to register interest in receiving messages from a channel.

<Image
  src="blogs/mnw_100/redis_channel.png"
  width={2094}
  height={1294}
  alt="Redis channel with publisher and three subscribers."
/>

Redis channels follow a topic-based publish-subscribe model.
Multiple publishers can send messages to a channel, and multiple subscribers can receive messages from that channel.

However, it's crucial to note that Redis channels do not store messages.
If there are no subscribers for a channel when a message is published, that message is immediately discarded.

Redis channels have an **at-most-once delivery** semantics.

## Practical Use Cases

Given that Redis channels operate with **at-most-once delivery** (messages might be lost if there are no subscribers),
they are well-suited for scenarios where occasional message loss is acceptable and real-time or near-real-time communication is desired.

Here are a few possible use cases:

- **Social media feeds**: Broadcasting new posts or updates to users.
- **Live score updates**: Sending live game scores or sports updates to subscribers.
- **Chat applications**: Delivering chat messages in real-time to active participants.
- **Collaborative editing**: Propagating changes in collaborative editing environments.
- **Distributed cache updates**: Invalidating cache entries across multiple servers when data changes. We'll cover this in detail later in the article.

Redis channels aren't the best choice for critical data where message loss is unacceptable.
In such cases, you should consider a more reliable messaging system.

Let's see how we can use Redis channels in .NET.

## Pub/Sub With Redis Channels

We will use the `StackExchange.Redis` library to send messages with Redis channels.

Let's start by installing it:

```powershell
Install-Package StackExchange.Redis
```

You can run [Redis](https://redis.io/) locally in a Docker container.
The default port is `6379`.

```
docker run -it -p 6379:6379 redis
```

Here's a simple background service that'll act as our message `Producer`.

We're creating a `ConnectionMultiplexer` by connecting to our Redis instance.
This allows us to obtain an `ISubscriber` that we can use for pub/sub messaging.
The `ISubscriber` will enable us to publish a message to a channel by specifying the channel name.

```csharp {11,19}
public class Producer(ILogger<Producer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        while (!stoppingToken.IsCancellationRequested)
        {
            var message = new Message(Guid.NewGuid(), DateTime.UtcNow);

            var json = JsonSerializer.Serialize(message);

            await subscriber.PublishAsync(Channel, json);

            logger.LogInformation(
                "Sending message: {Channel} - {@Message}",
                message);

            await Task.Delay(5000, stoppingToken);
        }
    }
}
```

Let's also introduce a separate background service for consuming messages.

The `Consumer` connects to the same Redis instance and obtains an `ISubscriber`.
The `ISubscriber` exposes a `SubscribeAsync` method that we can use to subscribe to messages from a given channel.
This method accepts a callback delegate that we can use to handle the message.

```csharp
public class Consumer(ILogger<Consumer> logger) : BackgroundService
{
    private static readonly string ConnectionString = "localhost:6379";
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = "messages";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        await subscriber.SubscribeAsync(Channel, (channel, message) =>
        {
            var message = JsonSerializer.Deserialize<Message>(message);

            logger.LogInformation(
                "Received message: {Channel} - {@Message}",
                channel,
                message);
        });
    }
}
```

Finally, here's what we get when we run both the `Producer` and `Consumer` services:

<Image
  src="blogs/mnw_100/redis_pub_sub.gif"
  width={1920}
  height={500}
  alt="Redis channels publish/subscribe demo."
/>

## Cache Invalidation in Distributed Systems

In a recent project, I tackled a common challenge in distributed systems: keeping the caches in sync.
We were using a two-level caching approach.
First, we had an in-memory cache on each web server for super-fast access.
Second, we had a shared Redis cache to avoid hitting our database too often.

The problem was that when data changed in the database, we needed a way to quickly tell all the web servers to clear their in-memory caches.
This is where Redis Pub/Sub came to the rescue.
We set up a Redis channel specifically for cache invalidation messages.

Each application would run a `CacheInvalidationBackgroundService` that subscribes to messages from the cache invalidation channel.

```csharp
public class CacheInvalidationBackgroundService(
    IServiceProvider serviceProvider)
    : BackgroundService
{
    public const string Channel = "cache-invalidation";

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await subscriber.SubscribeAsync(Channel, (channel, key) =>
        {
            var cache = serviceProvider.GetRequiredService<IMemoryCache>();

            cache.Remove(key);

            return Task.CompletedTask;
        });
    }
}
```

Whenever data changes in the database, we publish a message on this channel with the cache key of the updated data.
All the web servers are subscribed to this channel, so they instantly know to remove the old data from their in-memory caches.
Since the in-memory cache is wiped if the application isn't running, losing cache invalidation messages isn't a problem.
This keeps our caches consistent and ensures our users always see the most up-to-date information.

## In Summary

Redis Pub/Sub is not a silver bullet for every messaging need, but its simplicity and speed make it a valuable tool.
Channels allow us to easily implement communication between loosely coupled components.

Redis channels have at-most-once delivery semantics, so they're best suited for cases where the occasional dropped message is acceptable.

I used it to solve the challenge of synchronizing caches across multiple servers.
This allowed our system to serve up-to-date data without sacrificing performance.

**P.S.** When you're ready to dive deeper into creating message-driven systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
I have an entire module dedicated to building reliable distributed messaging and event-driven architecture.

Good luck out there, and see you next week.

---



<Sponsorships>

Streamline API development with [**Package Libary in Postman**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).
Add commonly-used scripts and tests to packages in your team's Package Library, and reuse them in your personal, private, and team workspaces using Postman.
[**Learn more here**](https://learning.postman.com/docs/tests-and-scripts/write-scripts/package-library/).

The [**IronOCR C# library**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan)
has hit a new milestone with 2 million NuGet downloads!
To celebrate, we're offering a 20% discount on IronOCR team licenses (Lite, Plus, Pro).
Plus, we're excited to introduce AdvancedScan, a powerful new feature that enhances OCR accuracy and Multiple-Language Support.
[**Try it Today**](https://ironsoftware.com/csharp/ocr/how-to/read-specific-document/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

Modular monoliths strike a balance between the simplicity of monolithic architecture and the flexibility of microservices.
By breaking down applications into cohesive modules, modular monoliths enable easier development and maintenance.
However, they still have a single codebase and deployment unit.

A critical aspect of the success of a modular monolith is the interaction between its modules.

**System integration testing (SIT)** is an approach to verifying the collaboration of these modules.
It involves testing the integration points and communication mechanisms between modules.

System integration testing allows us to validate the entire system's behavior.
This allows us to catch problems early before they become big headaches.

In this article, we'll learn how to test a modular monolith using system integration testing.

We'll look at real-world examples and discuss why this testing approach is useful.

## What Is System Integration Testing?

System integration testing (SIT) is an approach to testing the interactions between various modules within a single system.
The system could contain many external services, which should also be included during testing.

System integration testing is the perfect testing approach for modular monoliths.
It allows you to mimic the real-world execution of your application (as you'll see later).

The key benefits of system integration testing are:

- **Detecting integration issues**: Discover problems from module integrations that unit testing might miss.
- **Validating business logic**: Confirm that end-to-end business processes function correctly across modules.
- **Ensuring data integrity**: Verify that data flows accurately and consistently between modules.
- **Improving system stability**: Identify and resolve issues early, leading to a more reliable system.
- **Building confidence**: Ensure that the system is well-integrated and ready for deployment.

## Modular Monolith System Example

A modular monolith consists of multiple modules, each with a distinct responsibility.
Modules represent high-level components with well-defined boundaries.
The modules essentially group together related functionalities (use cases).
If you want to learn more about this software architecture, check out this [**introduction to modular monoliths**](what-is-a-modular-monolith).

Let's consider a hypothetical ticketing system that consists of several modules:

- **Users Module**: Handles user administration and authentication.
- **Events Module**: Manages events, scheduling, and ticket availability.
- **Ticketing Module**: Allows users to purchase tickets for various events.
- **Attendance Module**: Allows users to check into events using their tickets.

<figure>
  <Image
    src="blogs/mnw_099/modular_monolith.png"
    width={2506}
    height={1282}
    alt="Modular monolith UML diagram."
  />
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>

During system integration testing, we want to ensure that all these modules interact correctly and
fulfill the business requirements of the ticketing system.

## Testing Modular Monoliths: User Registration

The modules in our ticketing system represent different [**bounded contexts**](monolith-to-microservices-how-a-modular-monolith-helps).
The **Users Module** uses the term `User` as its core entity.
However, the **Ticketing Module** uses the term `Customer` because it's more aligned with its core responsibility of selling tickets.
Conceptually, both entities represent the same person within our system.

Here's the scenario we want to test:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record

<Image
  src="blogs/mnw_099/user_registration.png"
  width={2866}
  height={877}
  alt="User registration flow diagram."
/>

If you haven't figured it out by now, our [**modules communicate asynchronously**](modular-monolith-communication-patterns) using messaging.

Why is this important for our tests?

There is a time delay from when something occurs in one module until the other modules are notified and handle the integration events.
Our system integration tests will have to take into account this delay.

Here's the test for this scenario:

```csharp
public class RegisterUserTests : BaseIntegrationTest
{
    public RegisterUserTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task RegisterUser_Should_PropagateToTicketingModule()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);

                var customerResult = await Sender.Send(query);

                return customerResult;
            });

        // Assert
        customerResult.IsSuccess.Should().BeTrue();
        customerResult.Value.Should().NotBeNull();
    }
}
```

A lot is going on here, so let's unpack the steps:

- We're using the `WebApplicationFactory` to run an in-memory application instance
- Any external dependencies run in a Docker container using [**Testcontainers**](testcontainers-integration-testing-using-docker-in-dotnet)
- We send the `RegisterUserCommand` to register a user in the **Users Module**
- Then, we have to poll the **Ticketing Module** by sending a `GetCustomerQuery`
- The `Poller` allows us to wait until the system eventually becomes consistent

You can learn more about [**integration testing with Testcontainers in this article**](testcontainers-integration-testing-using-docker-in-dotnet).

System integration tests take longer to execute than integration tests for one module.
This is a side effect of testing a bigger slice of the overall system in each test case.

The `Poller` class allows you to execute a delegate until you receive a successful result or a timeout occurs.
You can customize the timeout duration based on the scenario you are testing.

```csharp
internal static class Poller
{
    private static readonly Error Timeout =
        Error.Failure("Poller.Timeout", "The poller has time out");

    internal static async Task<Result<T>> WaitAsync<T>(
        TimeSpan timeout,
        Func<Task<Result<T>>> func)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));

        DateTime endTimeUtc = DateTime.UtcNow.Add(timeout);
        while (DateTime.UtcNow < endTimeUtc &&
               await timer.WaitForNextTickAsync())
        {
            Result<T> result = await func();

            if (result.IsSuccess)
            {
                return result;
            }
        }

        return Result.Failure<T>(Timeout);
    }
}
```

## Testing Modular Monoliths: Adding Ticket to Cart

Here's a more complex scenario testing adding a ticket to the customer's cart:

- A user registers with our application through the **Users Module**
- The **Users Module** publishes an integration event to notify other modules
- The **Ticketing Module** handles the integration event and creates a customer record
- The **Ticketing Module** creates a dummy event and adds a ticket to the customer's cart

This scenario mimics a user registering with our system, finding a ticket they want to purchase, and adding it to their cart.
We can extend this scenario with more test cases, like ticket purchases.

```csharp
public sealed class AddItemToCartTests : BaseIntegrationTest
{
    private const decimal Quantity = 10;

    public AddItemToCartTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Customer_ShouldBeAbleTo_AddItemToCart()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result<Guid> userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result<CustomerResponse> customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =>
            {
                var query = new GetCustomerQuery(userResult.Value);
                var customerResult = await Sender.Send(query);
                return customerResult;
            });

        customerResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Add item to cart
        CustomerResponse customer = customerResult.Value;
        var ticketTypeId = Guid.NewGuid();

        await Sender.CreateEventAsync(Guid.NewGuid(), ticketTypeId, Quantity);

        Result result = await Sender.Send(
            new AddItemToCartCommand(customer.Id, ticketTypeId, Quantity));

        // Assert
        result.IsSuccess.Should().BeTrue();
    }
}
```

What I like about system integration testing is that the test cases aren't complicated to write.
The tests execute the use cases of our system and verify the side effects.
Building your system around [**use cases**](building-your-first-use-case-with-clean-architecture) has a few advantages,
the main one being that you can focus on the core business logic.
But it also makes (integration) testing easier.
The use case runs by sending a command or a query.
We can execute the use cases in some logical order and check that we get the expected result.

## In Conclusion

System integration testing is a crucial step in creating a successful modular monolith.
By testing how different modules work together, you can find and fix problems before they become bigger issues.
This makes your software more reliable and saves you time and resources in the long run.

If you want to dive deeper into building modular systems, check out [**Modular Monolith Architecture**](/modular-monolith-architecture).
There's an entire chapter dedicated to testing,
including advanced techniques like system integration testing, using Testcontainers for external services, and automated testing with CI/CD pipelines.

That's all for today.

See you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240713)

Collaborate directly with your external partners in Postman!
You can use [**Partner Workspaces**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/) to help your partners consume your team's APIs and work together on API projects.
Check out Postman's Partner Workspaces, now available for Professional and Enterprise plans in [**Postman v11**](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/partner-workspaces/).

</Sponsorships>

This is a question I often hear: how do I design my use case with Clean Architecture?

I understand the confusion.
Figuring out what to place in the Domain, Application, and Infrastructure layer can seem complicated.
If that's not enough, we also have to decide what makes up a use case and what should be abstracted away.

However, things become simpler if we adhere to the main rule in Clean Architecture — **the Dependency Rule**.
This rule states that source code dependencies can only point inwards.

In this newsletter, we'll explore a practical example of how to apply Clean Architecture principles by building a user registration feature.

## Clean Architecture

[**Clean Architecture**](why-clean-architecture-is-great-for-complex-projects) has emerged as a guiding principle for crafting maintainable, scalable, and testable applications.
At its core, Clean Architecture emphasizes the **separation of concerns** and the **dependency rule**.
The dependency rule dictates that dependencies should point inward toward higher-level modules.
By following this rule, you create a system where the core business logic of your application is decoupled from external dependencies.
This makes it more adaptable to changes and easier to test.

<Image
  src="blogs/mnw_098/clean_architecture.png"
  width={1280}
  height={714}
  alt="Clean Architecture diagram."
/>

The Domain layer encapsulates enterprise-wide business rules.
It contains domain entities, where an entity is typically an object with methods.

The Application layer contains application-specific business rules and encapsulates all of the system's use cases.
A use case orchestrates the flow of data to and from the domain entities and calls the methods exposed by the entities
to achieve its goals.

The Infrastructure and Presentation layers deal with external concerns.
Here, you will implement any abstractions defined in the inner layers.

## Describing The Use Case

What does it mean for a user to register with our application?
It means they reserve an email address (or username) to identify themselves and be able to interact with our system.
The user could provide other information, such as a first and last name, an address, and a phone number.

The first step in building any feature is clearly defining the desired result.

For user registration, this is what the required operations are:

- The user provides an email and password for registration
- Verify that the email was not reserved previously by an existing account
- Hash the password using some cryptographic hash function (e.g., SHA-256, SHA-512)
- Store the user in the database and (optionally) return an access token to the client

We could also consider any domain-specific rules or validations that we must enforce.
A good example is password strength, where we could implement minimum length and complexity requirements.

Now that we have our requirements let's see how to translate them into a use case.

## Implementing the Use Case

With our requirements in place, we can now define the user registration use case.
In [**Clean Architecture**](clean-architecture-and-the-benefits-of-structured-software-design), use cases live in the Application layer and orchestrate the interactions between domain entities and external dependencies.

Let's name our use case `RegisterUser`.
Its input will be a `RegistrationRequest` object containing the user's registration data,
and its output will be a `RegistrationResult` object indicating the outcome of the registration attempt.
Notice that we are using a feature-driven name for the use case.

What about any external dependencies?
If the use case needs to interact with an external system or infrastructure component, we abstract that behind an interface.
Remember, your application's core business logic should be decoupled from external dependencies.

The `RegisterUser` class will use dependency injection to get the necessary dependencies:

- `IUserRepository`: An interface for accessing user data from the database.
- `IPasswordHasher`: An interface for hashing passwords securely.

The `RegisterUser` use case will follow these steps:

1. Validate input data
2. Check for existing `User`
3. Hash the password
4. Create a new `User` entity
5. Save the `User` to the database
6. Return the result

Finally, here's the code for our `RegisterUser` use case:

```csharp
public class RegisterUser(
    IUserRepository userRepository,
    IPasswordHasher passwordHasher)
{
    public async Task<RegistrationResult> Handle(RegistrationRequest request)
    {
        // Validation omitted for brevity

        if (await userRepository.ExistsAsync(request.Email))
        {
            return RegistrationResult.EmailNotUnique;
        }

        var passwordHash = passwordHasher.Hash(request.Password);

        var user = User.Create(
            request.FirstName, request.LastName, request.Email, passwordHash);

        await userRepository.InsertAsync(user);

        return RegistrationResult.Success;
    }
}
```

A big benefit of this approach is that we can immediately write tests for the `RegisterUser` use case.
We can provide mocks for external dependencies in the tests.
We don't need the implementations to exist for this code to compile.
With mocks, we can test our business rules and validate our implementation.

**Action step**: How would you extend the `RegisterUser` use case with more functionality?

Here are two examples:

- Adding an external identity provider
- Implementing email verification

## Where Clean Architecture Becomes Muddled

By designing our application with Clean Architecture, we produce a system independent of external concerns.
We define abstractions in the Application layer and implement them in the Infrastructure layer.
So far, so good.

However, this doesn't mean you can disregard how you integrate with external dependencies.

In theory, we should be able to "swap" the implementation for any external concern and call it a day.
In practice, this couldn't be further from the truth.

Let me give you two practical examples using the user registration flow.

### Race Conditions

The `RegisterUser` use case has a race condition.
Concurrent requests could pass the check for email uniqueness and proceed to register the user.

We could prevent this race condition by introducing a lock before checking for email uniqueness.
That way, only one request will pass the check and proceed to save the user in the database.

```csharp
if (await userRepository.ExistsAsync(request.Email))
{
    return RegistrationResult.EmailNotUnique;
}
```

However, there is a much more elegant way to solve this.
We can introduce a unique index on the `Email` column in the database.
A unique index guarantees that only one transaction can write the unique value to the database.
The losing transaction will return an error.

We can handle this exception on the application side and return an appropriate error message to the user.
The `IUserRepository.InsertAsync` method implementation can encapsulate this logic.

### Changing Hash Functions

Let's say we found a security flaw in the hash function used in the `IPasswordHasher` implementation.
So, we spend a few minutes switching to a more secure hash function.
The tests for the `RegisterUser` use case are all green, and everything seems fine.

The problem? All existing users can no longer log in to the system.

When an existing user tries to log in with their email and password, the new `IPasswordHasher.Hash` implementation
returns a different password hash from the one stored in the database.

The correct approach is to phase out the old password hash for existing users.
We can add a column in the database that says which hash function produced the hash.
We will verify the user's password using the correct hashing function during the login process.

If the user's password hash still uses the old hashing function, we will verify their password first.
Then, we can use the password (which we have in memory) to produce a hash using the new hash function.
We will store the hash in the database and update the hash function column to the new algorithm.

Slowly, we will phase out passwords using the old hash function.

## Conclusion

I hope this was helpful in understanding how to apply Clean Architecture principles to a real-world scenario.
By focusing on the core business logic first (what it means for a user to register), we can define the requirements for our use case.
Translating these requirements into a series of steps within the use case is the easy part.

But Clean Architecture won't save you from bad engineering.
If you don't understand what you are abstracting away, it will become a problem in the long term.

If you want to go deeper, my flagship course, [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture),
takes the guesswork out of structuring your project the right way.
I share my entire framework for building robust applications from the ground up -
from building a rich domain model to creating use cases to getting your application ready for production.

And that's all for this week.

See you next Saturday.

---



<Sponsorships>

Build and test automations with [**UiPath**](https://rebrand.ly/dev-dives-uipath).
Join this free webinar to learn how the UiPath Platform enables you to build C# coded automations and quickly test them using GenAI.
[**Save your seat here**](https://rebrand.ly/dev-dives-uipath)!

Design and mock APIs with [**Postman Mock Servers**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/)!
Build higher-quality APIs faster with Postman mock API servers which simplify design and planning, support split-stack development,
and help you ensure that your API will run the way it's supposed to in production.
[**Learn more here**](https://learning.postman.com/docs/designing-and-developing-your-api/mocking-data/setting-up-mock/).

</Sponsorships>

Microservices have revolutionized how we build and scale applications.
By breaking down larger systems into smaller, independent services, we gain flexibility, agility, and the ability to adapt to changing requirements quickly.
However, microservices systems are also very dynamic.
Services can come and go, scale up or down, and even move around within your infrastructure.

This dynamic nature presents a significant challenge. How do your services find and communicate with each other reliably?

Hardcoding IP addresses and ports is a recipe for fragility.
If a service instance changes location or a new instance spins up, your entire system could grind to a halt.

Service discovery acts as a central directory for your microservices.
It provides a mechanism for services to register themselves and discover the locations of other services.

In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.

## What is Service Discovery?

Service discovery is a pattern that allows developers to use logical names to refer to external services instead of physical IP addresses and ports.
It provides a centralized location for services to register themselves.
Clients can query the service registry to find out the service's physical address.
This is a common pattern in large-scale distributed systems, such as Netflix and Amazon.

Here's what the service discovery flow looks like:

1. The service will register itself with the service registry
2. The client must query the service registry to get the physical address
3. The client sends the request to the service using the resolved physical address

<Image
  src="blogs/mnw_097/service_discovery_flow.png"
  width={1377}
  height={983}
  alt="Service discovery flow."
/>

The same concept applies when we have multiple services we want to call.
Each service would register itself with the service registry.
The client uses a logical name to reference a service and resolves the physical address from the service registry.

<Image
  src="blogs/mnw_097/service_discovery_microservices.png"
  width={2074}
  height={883}
  alt="Service discovery with multiple microservices."
/>

The most popular solutions for service discovery are Netflix [Eureka](https://github.com/Netflix/eureka) and HashiCorp [Consul](https://www.consul.io/).

There is also a lightweight solution from Microsoft in the `Microsoft.Extensions.ServiceDiscovery` library.
It uses application settings to resolve the physical addresses for services, so some manual work is still required.
However, you can store service locations in [Azure App Configuration](https://azure.microsoft.com/en-us/products/app-configuration) for a centralized service registry.
I will explore this service discovery library in some future articles.

But now I want to show you how to integrate Consul with .NET applications.

## Setting Up the Consul Server

The simplest way to run the Consul server locally is using a Docker container.
You can create a container instance of the `hashicorp/consul` image.

Here's an example of configuring the Consul service as part of the `docker-compose` file:

```yml
consul:
  image: hashicorp/consul:latest
  container_name: Consul
  ports:
    - '8500:8500'
```

If you navigate to `localhost:8500`, you will be greeted by the Consul Dashboard.

<Image
  src="blogs/mnw_097/consul_dashboard.png"
  width={1902}
  height={907}
  alt="Consul dashboard."
/>

Now, let's see how to register our services with Consul.

## Service Registration in .NET With Consul

We'll use the [Steeltoe Discovery](https://docs.steeltoe.io/api/v3/discovery/) library to implement service discovery with Consul.
The Consul client implementation lets your applications register services with a Consul server and discover services registered by other applications.

Let's install the `Steeltoe.Discovery.Consul` library:

```powershell
Install-Package Steeltoe.Discovery.Consul
```

We have to configure some services by calling `AddServiceDiscovery` and explicitly configuring the Consul service discovery client.
The alternative is calling `AddDiscoveryClient` which uses reflection at runtime to determine which service registry is available.

```csharp {6}
using Steeltoe.Discovery.Client;
using Steeltoe.Discovery.Consul;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddServiceDiscovery(o => o.UseConsul());

var app = builder.Build();

app.Run();
```

Finally, our service can register with Consul by configuring the logical service name through application settings.
When the application starts, the `reporting-service` logical name will be added to the Consul service registry.
Consul will store the respective physical address of this service.

```json
{
  "Consul": {
    "Host": "localhost",
    "Port": 8500,
    "Discovery": {
      "ServiceName": "reporting-service",
      "Hostname": "reporting-api",
      "Port": 8080
    }
  }
}
```

When we start the application and open the Consul dashboard, we should be able to see the `reporting-service` and its respective physical address.

<Image
  src="blogs/mnw_097/consul_dashboard_with_service.png"
  width={1902}
  height={907}
  alt="Consul dashboard with registered service."
/>

## Using Service Discovery

We can use service discovery when making HTTP calls with an `HttpClient`.
Service discovery allows us to use a logical name for the service we want to call.
When sending a network request, the service discovery client will replace the logical name with a correct physical address.

In this example, we're configuring the base address of the `ReportingServiceClient` typed client to `http://reporting-service`
and adding service discovery by calling `AddServiceDiscovery`.

Load balancing is an optional step, and we can configure it by calling `AddRoundRobinLoadBalancer` or `AddRandomLoadBalancer`.
You can also configure a custom load balancing strategy by providing an `ILoadBalancer` implementation.

```csharp
builder.Services
    .AddHttpClient<ReportingServiceClient>(client =>
    {
        client.BaseAddress = new Uri("http://reporting-service");
    })
    .AddServiceDiscovery()
    .AddRoundRobinLoadBalancer();
```

We can use the `ReportingServiceClient` typed client like a regular `HttpClient` to make requests.
The service discovery client sends the request to the external service's IP address.

```csharp
app.MapGet("articles/{id}/report",
    async (Guid id, ReportingServiceClient client) =>
    {
        var response = await client
            .GetFromJsonAsync<Response>($"api/reports/article/{id}");

        return response;
    });
```

## Takeaway

Service discovery simplifies the management of microservices by automating service registration and discovery.
This eliminates the need for manual configuration updates, reducing the risk of errors.

Services can discover each other's locations on demand, ensuring that communication channels remain open even as the service landscape evolves.
By enabling services to discover alternative service instances in case of outages or failures, service discovery enhances the overall resilience of the microservices system.

Mastering service discovery gives you a powerful tool to build modern distributed applications.

You can grab the [source code for this example here](https://github.com/m-jovanovic/service-discovery-consul).

Thanks for reading, and I'll see you next week!

---


Previous issues of The .NET Weekly