Pull Requests

A pull request (PR) is a social contract between developers, architects, and technical leads. So what are the major parts of a pull request?

Code

In software development, most of the time, PR is about code. Keep the changes as small as possible. Do not try to include too many changes in a pull request.

Description

I have seen many pull requests with poor descriptions. When I read them, they do not give me any helpful information. What should be included in a PR’s descriptions?

Bugs: Root Cause – Solution – Impact

Root Cause: It is a critical part. Developers must explain why and how it occurred. A common understanding between developers and reviewers is established once they read and agree upon the root cause. When the root cause is wrong, the rest does not matter.

Solution: Explain how you have fixed the problem. Did you make any tradeoffs? And why did you choose that approach? Depending on the problem, the detail differs.

Impact: Some changes might have significant consequences, such as infrastructure changes, changes in frameworks, changes in common flows. Some changes are local. Developers must be aware of them, and they must write them down.

Features: Problem/Requirement – Solution – Impact

The steps are the same as bugs, except the first one is "Problem/Requirement," a sentence explaining the outcome of the PR.

Commit Logs

Many pull requests have only one poor description commit. And it includes every change in the PR. IMO, it is a waste of commit feature. Commit logs should tell the process of solving the problem, a series of small changes. It reflects the implementation flow.

  1. A commit should capture a single "logical" change if possible. It is the same as the Single Responsibility Principle in software development.
  2. A commit should have a "good" commit message. At least it should help you, the developer, understand what has been implemented in that commit.
  3. The commit logs (all commits in a PR) should form an implementation flow, which is very important. It is like a checklist for both developers and reviewers. One can figure out what was missing by reading the commit logs.

There is pushback. What is a single "logical" change? Well, there is no definition. Developers have to practice and improve their judgment over time. It is also the magic of this practice. It makes developers better.

Resolve Code Reviews

If possible, resolve a comment in a commit. It is much easier to track what has been resolved and how – by looking at the commit detail.

Small practices can make a big difference.

ASPNET Core User Secrets

One of things I recently know: ASPNET Core user secrets.

I took over another ASPNET Core project. As usual, I did a normal routine

  1. Check out the code from repository
  2. Compile and hit F5

Oops! It compiled but an exception was thrown at runtime. Usually, the configuration was wrong because the code was now running on my machine. I checked the source and looked for the appsettings.json or appsettings.development.json file. Bad news! They were not there.

How could the other developers run it locally? Were there any secrets? Apparently, yes there was a User Secret.

<PropertyGroup>
  <TargetFramework>netcoreapp3.1</TargetFramework>
  <UserSecretsId>79a3edd0-2092-40a2-a04d-dcb46d5ca9ed</UserSecretsId>
</PropertyGroup>

And here is the complete article from Microsoft Docs. Once I read it, Aha! It has been around for a while. Just that I was behind the bar.

The purpose is clear. It enforces security by preventing accidental commits to repository. There are connection strings, credentials, passwords, … all kind of sensitive information. It is so easy to make a mistake and push them into the remote repositories.

However, I had a secret problem. There are more than 15 settings. Adding them one by one is not a pleasure task. Note that there was a configuration file with all default settings. If every developer names their database, port, folder, … consistently, the configuration is the same. I had that file by asking other developers. How do I import them into my secrets?

  1. Open the secret folder at %appdata%\Microsoft\UserSecrets\
  2. Find the folder with the secret ID
  3. Copy the appsettings.development.json file to it and rename to secrets.json. Delete the existing one if exists

And it works perfectly. If there are differences in configuration between branches, be aware of updating the secrets.

The secrets are revealed!

Decouple systems with ASP.NET Core Hosted Service

The story of Hosted Service and Channel to build decoupled systems.

I have been working on a project for around two and a half years. The high architecture is the Event Sourcing-Like. I do not dare to claim it Event Sourcing because it will lead to many debates.

Here is the overview. There are 2 data stores. The primary one is an SQL database storing all events. This is the fact, the truth of the system. The second one is a Cosmos database. This stores the projected views from the events. So the writes go to the Event Store. The reads go to the Cosmos store.

Architecture

The write side composes of Application Services. The read side composes of Projections. We have to send events to the Projections to project them and store the normalized views in Cosmos. The communication has to be asynchronous.

The normal reaction is some kind of a queue. However, we do not want to go that far if we can handle everything inside a single Web Application. Our application is not at StackOverflow scale so a single Web Application instance should be enough. There is a lot of data for sure, but the request rates are reasonable.

Projections as Background Service

The Projections must be run as a background service alongside with the WebApi. With the ASP.NET Core, it is fairly easy with Hosted Service. All good stuffs are documented by Microsoft. Go and check it out. Good stuff! Guaranteed!

We also use Hosted Service for other purposes such as Deleting Files.

Ok, we have the WebApi (the application services) and a background service, how are they communicating to each other?

Channel

It is well-documented, well-explained here DevBlog Channel, or a wonderful blog from Steve Gordon. These links cover everything you need to know about Channel.

To solve the communication issue, we use Channel to create a channel from Application Services to Projections. It is a multi-writers, single-reader channel. If we need a two-way ticket, another channel is created. We also have it for the scenario where the Application Services have to wait for the Projections.

It works beautifully.

Code

So it is time to see some code. All the code is on GitHub

I started a new project with the .NET 6.0 (the latest one). It is cool to use the latest even thought I do not use any special feature in the project.

Let start with a simple stupid API

[ApiController]
[Route("[controller]")]
public class CountersController : ControllerBase
{
    private readonly CountersService _service;

    public CountersController(CountersService service)
    {
        _service = service;
    }

    [HttpPost]
    [Route("/increase")]
    public async Task<IActionResult> Increase()
    {
        await _service.IncreaseCounter();
        return Ok("Have a good day!");
    }
}

The API exposes an endpoint that will make some changes to the system. The controller then dispatches the actual work to a service CountersService. This approach keeps the controller simple, stupid, and clean.

The CountersService handles the logic and finally publishes events.

public class CountersService
{
    private readonly EventChannel _eventChannel;

    public CountersService(EventChannel eventChannel)
    {
        _eventChannel = eventChannel;
    }
    public  Task IncreaseCounter()
    {
        // Execute whatever the business logic here.
        // And finally, write an event package into the event channel.
        // Imagine there is an actual framework here which will publish events.
        // Those events are then packaged into EventPackage and dispatch
        return _eventChannel.Write(new EventPackage
        {
            EventName = "CounterIncreased"
        });
    }
}

That is the end of a service. It does a bunch of stuff and finally publish events (EventPackage).

Who is going to handle those package asynchronously? The Projection Service

public class ProjectionBackgroundService : BackgroundService
{
    private readonly ILogger<ProjectionBackgroundService> _logger;
    private readonly EventChannel _eventChannel;

    public ProjectionBackgroundService(
        ILogger<ProjectionBackgroundService> logger,
        EventChannel eventChannel)
    {
        _logger = logger;
        _eventChannel = eventChannel;
    }
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        // Keep running until asked to stop
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await foreach (var package in _eventChannel.ReadAllAsync(stoppingToken))
                {
                    // Include the logic to proceed a package here.
                    // A package typical contains the actual event (or events) that the projection needs to project
                    _logger.LogInformation($"Package received: {package.EventName}");
                }
            }
            catch (Exception e)
            {
                _logger.LogError(e, "Bad thing happened");
            }
        }
    }
}

It picks up event packages from EventChannel one by one and proceeds.

The final piece that connects them is the EventChannel

public class EventChannel
{
    private readonly Channel<EventPackage> _channel;

    public EventChannel()
    {
        var options = new BoundedChannelOptions(10_000)
        {
            SingleReader = true
        };
        _channel = Channel.CreateBounded<EventPackage>(options);
    }

    public IAsyncEnumerable<EventPackage> ReadAllAsync(CancellationToken cancellationToken)
        => _channel.Reader.ReadAllAsync(cancellationToken);

    public async Task Write(EventPackage eventPackage)
    {
        if (await _channel.Writer.WaitToWriteAsync())
        {
            await _channel.Writer.WriteAsync(eventPackage);
        }
    }
}

public class EventPackage
{
    public string EventName { get; init; }
}

And finally, register those components with ServiceCollection (DI framework in .NET Core)

public static class ServiceCollectionExtensions
{
    public static void RegisterComponents(this IServiceCollection services)
    {
        services.AddHostedService<ProjectionBackgroundService>();
        services.AddScoped<CountersService>();

        services.AddSingleton<EventChannel>();
    }
}

Once run, you should see this in the console

And that is it. I have a decouple architecture that will work.

The Neglect in the Technical Details and Why It Matters

In software development organizations, what are the common things shared in the discussion of managers? In Agile/Scrum conferences, what are they discussing? Processes, people management, autonomy, … but rarely mentioned technical details.

In one Agile conference, they talked about adapt to changes. I asked: "But how?". A hole bunch of Agile theory came out. None of them actually solved the problem. None of them talked about architecture, the design that allows the team to adapt changes quickly and less expensive.

I did not understand why back then. I just did not like the answer and the way they understood and presented the Agile. It might work for them.

"Clients change their mind often"–it all started from there. If everyone knows what they want and do not change their mind, there is no problem, no job in the world. The software developer job is less interesting, less challenging. I heard they said about "educate clients", "educate other parties". Oh really? How does it work? How many percent of success in that?

The way I see it is that we shift the blame to the clients. We shift our focus to the things we have so little control over. And we ignore the most important part: Build the software, write the code.

Put aside all the contractual agreement, the cost, the time, change requests are eventually changing the code to adapt the new requests.

"How easy is it to change?" – That should be the most important question, IMO.

Most of the time we deal with the problems we created. Why? Because the bad architecture, it is extremely hard to change something in the code.

MVP, Minimum Viable Product, sounds cool than dangerous without careful design and architecture. How hard is it to add new feature after the first MVP? Is it possible to append or rewrite? It all depends on how well the MVP is defined, scope and implemented.

Eventually, regardless of methodologies, what matters most is the code. It is the fundamental block of a software. I really do not want to ignore it in my discussions.

An Opinionated Usage of Interface

Every C# developer knows what interface is. From the code perspective it is interface ISomething. There is no requirement for the I in the interface name. But that is just the naming convention. And it is good to know it is an interface by looking at the name. A more interesting question is when do we use interface?. And I guess each developer might have their own answer and reasons behind them.

Back in time, about 10 years ago, I started to use interface a lot. Back then the obvious reason was mocking and unit testing. And then the dependency injection came to my developer life. I read somewhere that you should inject interfaces instead of concrete implementations.

Have you ever heard of these?

  • Depend on interfaces instead of concrete implementations
  • It is easier for you to change the implementation later
  • It helps you mocked the implementation in the unit test
  • Your code looks clean and professional

In some codebases, I ended up with interfaces everywhere. The unit test was a bunch of mocks and mocks, everywhere. It looked good in the beginning. However, after a while, it was a pain.

  • It was hard to do the refactoring. For example, when I did not want to move a piece of code from this class to another class without changing the outcome behavior, the unit test failed. Because the expected interface was no longer there. The unit test knew too much about the implementation detail. I have the habit of refactoring code quite often. And I expect the unit test to catch my mistake if I accidentally change the outcome behavior. With mocking, I failed to archive that
  • Had to test at every layer. Basically, there were behavior test with mocking and there were tests for actual implementation. There were too much test code to maintain. Well, that was a waste of time and effort and error prone in the test
  • The chances of actually changing an implementation was rare

Ok, so is interface useful? Of course yes it is. And here are my opinions when to use it.

Code Contract

The interface tells the consumers the functionalities it supports. A class might have 10 methods. But not all consumers use or care 10 methods. They might be interested in 2 or 3 methods. It is fine to inject the class. However, the consumer is confused and might misuse.

Let take an example of an imagination log service. Here is the concrete implementation

public class SimpleLogService
{
    public void WriteLog(string logMessage)
    {

    }

    public IList<string> GetLogs()
    {
        return new List<string>();
    }
}

// API Controller to read the log
public class LogController : Controller
{
    private readonly SimpleLogService _logService;
    public LogController(SimpleLogService logService)
    {
        _logService = logService;
    }

    public IActionResult Get()
    {
        return OK(_logService.GetLogs());
    }
}

There is nothing wrong with the above code. However, I do not want the LogController to see/use the WriteLog method. That method is used by another controllers or services. And the SimpleLogService class might grow in size over the time. More and more methods are developed.

To solve that problem, I want to create a contract to tell LogController what it can use.

public interface ILogReaderService
{
    public IList<string> GetLogs();
}

public class SimpleLogService : ILogReaderService
{
    public void WriteLog(string logMessage)
    {

    }

    public IList<string> GetLogs()
    {
        return new List<string>();
    }
}

// API Controller to read the log
public class LogController : Controller
{
    private readonly ILogReaderService _logService;
    public LogController(ILogReaderService logService)
    {
        _logService = logService;
    }

    public IActionResult Get()
    {
        return OK(_logService.GetLogs());
    }
}

And I can do the same for the WriteLog part if necessary.

Decouple Implementation Dependency

In many projects, there is data involve. There are databases. And then comes the concept of Repository. And if the repository implementation is easy and that the database is ready. A developer can write a complete feature from API layer down to database layer. But I am doubt that is the reality. So the situation might look like this

  • One developer takes care of front end development
  • One developer takes care of the API (controller) implementation
  • One developer takes care of designing database, writing the repository. This might be the same developer that implements the API

The API layer depends on the Repository. However, we also want to see the flow and speed up the development. Let’s see some code

public class UserController : Controller
{
    private readonly IUserRepository _repository;

    public UserController(IUserRepository repository)
    {
        _repository = repository;
    }

    public async Task<IActionResult> GetUsers()
    {
        var users = await _repository.GetAllUsers();

        return Ok(user);
    }
}

The IUserRepository is the Code Contract between API and the repository implementation. To unlock the development flow, a simple in memory repository implementation is introduced

public class InMemoryUserRepository : IUserRepository
{
    public async Task<IList<User>> GetAllUsers()
    {
        await Task.CompletedTask();

        return new List<User>{
            new User("Elsa"),
            new User("John"),
            new User("Anna")
        };
    }
}

And the API can function. This releases the dependency on the actual repository implementation. When such an implementation is ready, switch to use it.

However, do not overuse it. Otherwise, you end up with interfaces everywhere and each developer gives their own temporary implementations. Choosing the right dependencies is an art and context matters a lot.

Conclusion

I rarely create interfaces with the purpose of unit testing in mind. Rather, it is the outcome of writing code, refactoring from a concrete implementation and then extracting into interfaces where they make the most sense. When I do, I pay close attention to its meaning. If I can avoid an interface, I will do it.

Code Contract and Decouple Implementation Dependency are the 2 big benefits from having proper interfaces. There are other reasons to use interfaces. They are all valid depending on the context. Sometime, it is the way the project architect wants it.

What are yours?