
**A big thank you to our sponsors who keep this newsletter free to the reader:**

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

---

If you're building a scalable application, it's a common requirement to offload some work in your application to a **background job**.

A few examples of that are:
- Publishing email notifications
- Generating reports
- Updating a cache
- Image processing 

How can you create a recurring **background job** in .NET?

[**Quartz.NET**](https://www.quartz-scheduler.net/) is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

There are three concepts you need to understand in **Quartz.NET**:
- **Job** - the actual background task you want to run
- **Trigger** - the trigger controlling when a job runs
- **Scheduler** - responsible for coordinating jobs and triggers

Let's see how we can use **Quartz.NET** to create and schedule **background jobs**.

## Adding The Quartz.NET Hosted Service

The first thing we need to do is install the **Quartz.NET** NuGet package.
There are a few to pick from, but we're going to install the `Quartz.Extensions.Hosting` library:

```powershell
Install-Package Quartz.Extensions.Hosting
```

The reason we're using this library is because it integrates nicely with .NET using an `IHostedService` instance.

To get the **Quartz.NET** hosted service up and running, we need two things:
- Add the required services with the DI container
- Add the hosted service

```csharp
services.AddQuartz(configure =>
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});
```

**Quartz.NET** will create jobs by fetching them from the DI container.
This also means you can use scoped services in your jobs, not just singleton or transient services.

Setting the `WaitForJobsToComplete` option to `true` will ensure that **Quartz.NET** waits for the jobs to complete gracefully before exiting.

## Creating Background Jobs With `IJob`

To crate a background job with **Quartz.NET** you need to implement the `IJob` interface.

It only exposes a single method - `Execute` - where you can place the code for your background job.

A few things worth noting here:
- We're using DI to inject the `ApplicationDbContext` and `IPublisher` services
- The job is decorated with `DisallowConcurrentExecution` to prevent running the same job concurrently

```csharp
[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List<OutboxMessage> messages = await _dbContext
            .Set<OutboxMessage>()
            .Where(m => m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject<IDomainEvent>(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}
```

Now that the **background job** is ready, we need to register it with the **DI** container and add a trigger that will run the job.

## Configuring the Job

I mentioned at the start that there are three key concepts in **Quartz.NET**:
- Job
- Trigger
- Scheduler

We already implemented the `ProcessOutboxMessagesJob` background job in the previous section.

The **Quartz.NET** library will take care of the scheduler.

And this leaves us with configuring the **trigger** for our `ProcessOutboxMessagesJob`.

```csharp
services.AddQuartz(configure =>
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob<ProcessOutboxMessagesJob>(jobKey)
        .AddTrigger(
            trigger => trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule => schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});
```

We need to uniquely identify our **background job** with a `JobKey`.
I like to keep it simple and use the job name.

Calling `AddJob` will register the `ProcessOutboxMessagesJob` with DI and also with Quartz.

After that we configure a trigger for this job by calling `AddTrigger`.
You need to associate the job with the trigger by calling `ForJob`, and then you configure the schedule for the background job.
In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.

**Quartz** also has support for configuring triggers using [cron expressions.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html)

## Job Persistence

By default, Quartz configures all jobs using the `RAMJobStore` which is the most performant because it keeps all of its data in RAM.
However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.

It could be useful to have a persistent job store in some scenarios and there's a built in `AdoJobStore` which works with SQL databases.
You need to create a set of database tables for Quartz.NET to use.

You can learn more about this in the [job stores documentation.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html)

## Takeaway

**Quartz.NET** makes running **background jobs** in .NET easy, and you can use all the power of DI in your **background jobs**.
It's also flexible for various scheduling requirements with configuration via code or using cron expressions.

There's some room for improvement to make scheduling jobs easier and reduce boilerplate:
- Add an extension method to simplify configuring jobs with a simple schedule
- Add an extension method to simplify configuring jobs with a cron schedule from application settings

If you want to see a tutorial on using **Quartz.NET**, I made an in-depth video about [**using Quartz for processing Outbox messages**](https://youtu.be/XALvnX7MPeo).

That's all for this week.

See you next Saturday.

---



**A big thank you to our sponsors who keep this newsletter free to the reader:**

Today's issue is sponsored by [**Tower.**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)
Try Tower, a [**powerful Git client for Windows**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly) focused on productivity.
Make the complex simple with helpful features like Undo, Drag and Drop and Remote Services Integration!

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

---

Do you want to streamline your software development process and accelerate your release cycles?

Imagine being able to automatically build, test, and deploy your .NET applications with every code change.

With **CI/CD**, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.

And it's never been easier to get started with **CI/CD**.

**GitHub Actions** are completely free and simple to use.

So here's what we'll cover:
- Introduction to **CI/CD** & **GitHub Actions**
- Creating a **build & test pipeline** for **.NET**
- Creating a **deployment pipeline** for **Azure App Service**

Let's dive in!

## What Is Continuous Integration And Delivery?

I'll try to briefly explain what **CI/CD** is, before we take a look at **GitHub Actions**.

**CI/CD** is a method to increase the frequency of delivering new features by adding automation to your software development workflow.

**Continuous Integration ("CI")** refers to the automation process of syncing new code to a repository.
Any new changes to the application code are immediately built, tested and merged.

**Continuous Delivery, or Deployment, ("CD")** refers to the process of automating the deployment part of the workflow.
When you make a change which gets merged to the repository, this step takes care of deploying those changes to the production environment (or any other environment).

## Continuous Integration With GitHub Actions

If you're using **GitHub**, getting started with **Continuous Integration** has never been easier.

You can use [**GitHub Actions**](https://github.com/features/actions) to automate your build, test, and deployment **pipeline**.
You can create workflows that build and test every commit to your repository, or deploy to production when a new tag is created.

To create a **GitHub Action**, you write a _workflow_ to be triggered when some _event_ occurs in your repository.
An example event is a commit to the main branch, creation of a tag, or you can manually run the workflow.

Here's a **GitHub Actions** workflow to build and test a .NET project:

```yaml
name: Build & Test 🧪

on:
  push:
    branches:
    - main

env:
  DOTNET_VERSION: "7.0.x"

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Install dependencies 📂
        run: dotnet restore WebApi

      - name: Build 🧱
        run: dotnet build WebApi --configuration Release --no-restore

      - name: Test 🧪
        run: dotnet test WebApi --configuration Release --no-build
```

Let's unwrap what is happening here:
- Defining an event to trigger the workflow
- Setting up the **.NET SDK** with the version from `env.DOTNET_VERSION`
- Restoring, building and testing the project using the `dotnet` CLI tool

You can add this to your **GitHub** repository today, and start getting instant feedback when you commit code to the repository.

When a workflow run fails due to a build error or a failed test, you'll get an email notification.

## Continuous Delivery To Azure With GitHub Actions

**Continuous Integration** is a great way to start out with **CI/CD**, but the real value lies in **automating** your **deployment** process.

Imagine this:
- You make a change to your codebase
- The commit triggers the **deployment pipeline**
- A few minutes later your changes are live in production

Usually it's a little more nuanced, because we need to think about configuration, database migrations, etc.
But try to see the big picture here.

If you're running your application in the cloud, for example on **Azure**, chances are there's an existing **GitHub Action** you can use.

Here's a **deployment pipeline** I use to publish my application to an **Azure App Service** instance:

```yaml
name: Publish 🚀

on:
  push:
    branches:
    - main

env:
  AZURE_WEBAPP_NAME: web-api
  AZURE_WEBAPP_PACKAGE_PATH: "./publish"
  DOTNET_VERSION: "7.0.x"

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Build and Publish 📂
        run: |
          dotnet restore WebApi
          dotnet build WebApi -c Release --no-restore
          dotnet publish WebApi -c Release --no-build
            --output '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'

      - name: Deploy to Azure 🌌
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          publish-profile: ${{ secrets.AZURE_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
```

This workflow is pretty similar to the previous one, with the differences being:
- Adding a publish step and configuring the output path
- Using the `azure/webapps-deploy@v2` action to deploy to **Azure**

If you need to safely and securely expose secret values in your workflows, you can use **GitHub secrets**.
You can define the secrets on GitHub, and use them in actions without having to add them to source control. 

In the deployment workflow I'm using `secrets.AZURE_PUBLISH_PROFILE` to access my publish profile for the App Service instance.

## In Summary

**Continuous Integration and Delivery** can transform your development process by increasing the speed at which you release changes.

Try adding up how much time you spend on deployments.
I'm pretty sure you'll be surprised by the time savings potential of automating them.

And the good part is you will typically set up your **build and deployment pipelines** once, and then continue benefiting from them for the lifetime of your project.

If you want a step by step guide, I made a video showing [**how to implement a CI/CD pipeline from scratch.**](https://youtu.be/QP0pi7xe24s)
And the **source code** is also public, so you can add the **GitHub Actions** workflow file to your project.

Thanks for reading.

Hope that was helpful!

---



**A big thank you to our sponsors who keep this newsletter free to the reader:**

Today's issue is sponsored by [**Refactoring.**](https://refactoring.club/?utm_source=milan)
Refactoring is a weekly newsletter [**about making great software and working well with humans.**](https://refactoring.club/?utm_source=milan)
It is read every week by 35,000+ developers, managers and leaders at companies like Google, Meta, and Amazon.
Check it out [**here.**](https://refactoring.club/?utm_source=milan)

And by [**Jam.**](https://jam.dev/)
Reduce time spent on bug reporting with [**Jam**](https://jam.dev/) by 60-80%.
Auto-capture dev logs & network requests in one link, access Instant Replays for hard-to-repro bugs and integrate with Jira, Slack & more.
Free Team Tier for the rest of 2023.
[**Get Jam Now.**](https://jam.dev/)

---

Most software applications today are built around the concept of **multi-tenancy**.

One application serves multiple customers, while keeping their data **isolated**.

You can approach **multi-tenancy** in two ways:
- **Single database** and **logical isolation** of tenants
- **Multiple databases** and **physical isolation** of tenants

Which option you decide to use will depend mostly on your requirements.
Some industries like healthcare require a high degree of **data isolation**, and using a **database per tenant** is a must.

So how are we going to **implement multi-tenancy** support with **EF Core**?

We can use **Query Filters** to apply a **tenant filter** to all database queries.

Implement it once, and you can almost forget about it.

Let's see what are some of the problems we need to solve.

## How To Use EF Core Query Filters

If you want an in-depth dive into **Query filters**, take a look at the newsletter issue where I talked about
[**using query filters with EF Core.**](how-to-use-global-query-filters-in-ef-core)

Here's a quick refresher on **Query filters**:
- Configure the **query filter** by calling `HasQueryFilter` for your entity
- **EF** will apply it to all queries for that entity
- You can turn it off with `IgnoreQueryFilters`
- Only **one** query filter **per entity** is allowed

And here's a simple example:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

All queries to the `Order` table will include an `IsDeleted = FALSE` condition.

## Single Database Multi-Tenancy With EF Core

You will need two things to **implement multi-tenancy** on a **single database**:
- A way to know **who** the **current tenant** is
- A way to **filter** the **data** for that **tenant** only

The typical approach for **multi-tenancy** on a **single database** is having a `TenantId` column in your tables.
And then filtering on that column when querying the database.

You can use the **Query filters** feature in **EF Core** to apply a **global filter** for some entity.

Inside of the `OnModelCreating` method we configure the query filter on the `Order` entity:

```csharp
public class OrdersDbContext : DbContext
{
    private readonly string _tenantId;

    public OrdersDbContext(
        DbContextOptions<OrdersDbContext> options,
        TenantProvider tenantProvider)
        : base(options)
    {
        _tenantId = tenantProvider.TenantId;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder
            .Entity<Order>
            .HasQueryFilter(o => o.TenantId == _tenantId);
    }
}
```

We're using the `TenantProvider` class to get the **current tenant** value.

Here's what the `TenantProvider` implementation looks like:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;

    public TenantProvider(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];
}
```

The `TenantId` is coming from the HTTP request header in this example.

A few other options to get the `TenantId` are:
- Query string - `api/orders?tenantId=example-tenant-id`
- JWT Claim
- API Key

If you want a more **secure implementation** you should go with **JWT Claims** or **API Keys** to provide the `TenantId` value.

## Separate Databases Multi-Tenancy With EF Core

What if we want to **isolate** each tenant to a **separate database**?

Here are the changes we need to make:
- Applying different **connection string per tenant**
- Resolving the connection string for each tenant _somehow_

You can't use **Query filters** here, since we are working with different databases.

So you will need to store the **tenant information** and **connection strings** somewhere.

A simple example would be store them in the **application settings**:

```json
"Tenants": {
    { "Id": "tenant-1", "ConnectionString": "Host=tenant1.db;Database=tenant1" },
    { "Id": "tenant-2", "ConnectionString": "Host=tenant2.db;Database=tenant2" }
}
```

You can then register an `IOptions` instance with a list of `Tenant` objects.

And we need to slightly modify the `TenantProvider` class to return a **connection string** for the current tenant:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;
    private readonly TenantSettings _tenantSettings;

    public TenantProvider(
        IHttpContextAccessor httpContextAccessor,
        IOptions<TenantSettings> tenantsOptions)
    {
        _httpContextAccessor = httpContextAccessor;
        _tenants = tenantsOptions.Value;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];

    public string GetConnectionString()
    {
        return _tenantSettings.Tenants.Single(t => t.Id == TenantId);
    }
}
```

And the last part is registering your `DbContext` to **dynamically resolve** the **connection string** for the current **tenant**.

```csharp
builder.Services.AddDbContext<OrdersDbContext>((sp, o) =>
{
    var tenantProvider = sp.GetRequiredService<TenantProvider>();

    var connectionString = tenantProvider.GetConnectionString();

    o.UseSqlServer(connectionString);
});
```

On every request, we create a new `OrdersDbContext` and connect to the appropriate database for that tenant.

You should definitely consider storing the tenant **connection strings** in a **secure** place like **Azure Key Vault**.

## Closing Thoughts

I hope you now have a better understanding of how to build a **multi-tenant system** with **EF Core**.

I showed you the bare bones implementation, which you can improve to make it more robust and secure.

Building **multi-tenant systems** isn't easy, but when you understand the basic principles it shouldn't be too difficult either.

That's all for this week.

See you next Saturday.

---


Milan Jovanović

Software Architect•Microsoft MVP•Entrepreneur

Latest On YouTube

Let's connect

Milan Jovanović

Software Architect•Microsoft MVP•Entrepreneur

Recent Issues of Milan's .NET Weekly

Latest On YouTube

Let's connect