Category Archives: software

.net core: could not load file or assembly runtime error

Shortly after posting about my tips to fix broken builds, I was in for another bug hunt that took a few hours to resolve. This time it wasn’t the build itself that failed, but the package that came out of it. For some reason, the application crashed instantly on a missing DLL error you know and hate in .NET.
Usually it means a Nuget package conflict. Cleaning the build sometimes helps, but not in this case. We are talking about system DLL’s here, so you know you’re in trouble when that happens.

I was getting the following exception at runtime on a .NET Core app. It wasn’t always the same DLL. But every time it was a system DLL like System.Collections, System.Collection.Generic, System.Linq, etc.

Unhandled exception. System.IO.FileLoadException: Could not load file or assembly 'System.Collections, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
File name: 'System.Collections, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at Microsoft.Extensions.DependencyInjection.ServiceCollection..ctor()

When checking the package files, the wrong version of the DLL was in the output, compared to the version number I was seeing in the solution, so the error message was correct. The problem was, why does the build pick the wrong DLL version? My local build output was fine however, so that just adds to the mystery.

The special bit about this project is that it also used a .NET Framework DLL. It was doing some .NET Standard magic in the build.

The symptoms

  • The project has a mix of .NET Core / .NET Standard / .NET Framework projects or Nuget packages.
  • You are building a self contained application (so all DLL’s are copied in the output folder).
  • Seemingly random 0x80131040 errors when I try to run my deployed .NET Core app on system DLL’s.
  • Different builds give errors on different DLL’s, but they are always system DLL’s.
  • A local dotnet publish works, but the output from the build server doesn’t run.

The fix

The fix turned out to be stupefyingly simple. I was build the .sln file for my project in the CI pipeline, instead of the .csproj file for the console application itself.
Apparently things go wrong at build time and the wrong DLL’s are copied into your output folder, giving you the exceptions above on startup.
The reason why the same dotnet publish command works locally is probably because I have version 5 of the dotnet tool on my local machine, due to Visual Studio. On the build server version 3.1 is installed, from the 3.1 SDK. My guess is that the 3.1 version handles mixed solutions differently than the dotnet 5 version.

tips on fixing a broken build

When builds break on the build server, it can be hard to figure out what the problem is. I’ve been doing this a lot of times, and so here’s a collection of tips that can help you in fixing that weird error that keeps breaking the build. Build systems are pretty similar, so whether you’re using TeamCity, Azure Devops, Gitlab, Jenkins, Circle CI or whatever, these should pretty much work everywhere.

Make sure it builds locally.

This has to be the “Did you try turning it on and off again?” of build fixing. Sometimes that build really is broken. A file might not have been checked in, or a merge might have gone wrong, but what you have on your local machine, might not be what’s in source control.
Get a clean copy of your repo in a separate folder, and build that to check if it builds OK. At least you’ll be 100% sure it’s nothing trivial before you start your investigation.

Try a clean build.

Another “try turning it on and off again” tip, I guess. On most build servers, you can opt to run a clean build. This means the agent will set up a fresh checkout folder for the new build. This way you make sure no leftovers from a previous build are causing the problem. Sometimes things go wrong with source control and getting the latest version, messing with your build. This also fixes that issue.

Does it fail everywhere?

Sometimes build agents are not installed in exactly the same way, and builds might break on a single or a few build agents only. This is good. Now you know you can always get a build on a good agent, so you aren’t stuck when you want to produce some build artifacts, and you know there’s something on that machine that’s not quite right where it’s failing.
The error will usually give you a clue on what’s missing, but most of the time you’ll only understand why after you resolved the issue. Ah, the joys of software development.
My tip here is to compare the machines and see what build tools are missing or different from the one where it does work. Make sure your SDK and build tool versions are the same, and can be found. Check your shell versions, like PowerShell and if they are running in x32 or x64 mode, and if all PS modules are installed in the mode you’re using.
If you have an automated way to quickly produce a build agent, you might want to ditch the failing one, in favor of a brand new one if your build works there.

Check the logs.

It sounds trivial, but the log files of your build server contain a lot of details you’re not looking at when things are working. Now that things are broken, it’s time to dig in and see if you can find a few clues. Most build systems also have settings to change how detailed those logs are. Setting this to verbose can be what you need to find that final clue to solve the puzzle.

Get into the machine.

When things should work, but appear broken on the build server, you’ll sometimes have to dial into the build agent and see what things look like in there. Paths might not be what is expected, files might actually be missing, and tools might be different. There you can open a shell, check if anything is missing, check if build tools are available, or try and run the same command that’s failing yourself. This doesn’t always work as build systems sometimes wrap your SDK calls, and you’re not always sure you’re running the same statement, but it might give you a hint of what’s going wrong anyway.
Look around on the machine, and you might find the problem. Can’t access the machine? Try the next tip.

Get into the machine, without getting into the machine.

You might not be able to access the build agent itself. Pesky admins, or it might even be a cloud machine you don’t own. Luckily you can edit the build itself right? In most cases, you can run some arbitrary shell commands as a build step for whatever fancy stuff that isn’t covered by default.
Well, nothing is stopping you from running an ls or type nuget.config command to find out what is really happening during your build. The output of those commands will normally appear in your build logs. You might have to check the more detailed full log files, but it should be there. This can be a tedious process, but it saved my butt a few times when things didn’t work for an at the time unknown reason.

wake up Windows automatically using a scheduled task

This seems easy at first, but it turns out to be a lot harder than expected, due to various OS and power settings that get in the way. The idea is simple: you want to have your Windows 10 machine start up at a specific time using a task set in the task scheduler.

When you create a new task in the Task Scheduler, there’s a setting for this, so it looks so easy. It’s on the conditions tab. You activate the option to wake the computer to execute your task, and you’re done. Right? Well, if you’re really lucky that might work straight out of the box.
In case it doesn’t work, here are some things you can fiddle with to try and get it going.

  1. Create a wake-up tasks that doesn’t really do anything. Just run a command like this:
    cmd /c echo %date%
    For a more detailed guide on how to set up a task and the wake timers check this excellent article.
  2. Create a second task that does what you want it to do in a batch script but 10 minutes later than the first. That gives your computer ample time to start up, install any potential updates, or do whatever it sometimes does that takes so damn long.
  3. Deactivate Fast Startup. It messes with your hibernation mode and causes it not to startup again afterwards.
  4. Always hibernate your machine. If you just do a plain shutdown it doesn’t seem to automatically start up again. You can do this with the command below. This is also handy if you want to shut your machine down again, after your task has finished:
    shutdown /h
  5. Enable wake timers in your Power Options advanced settings. See the link below for instructions.
  6. For laptops make sure your Power Options are configured correctly. Mind closing the lid in that case. A closed lid will prevent the laptop from starting up in my case for example. Laptops will also not startup when they are running on the battery by default.
  7. Reboot your machine after fiddling with these settings if it still doesn’t work. This makes sure your settings stick.
  8. If you tried everything and it still doesn’t work, check your BIOS settings and see if there might be a power option in there that might prevent it from starting up automatically.

Give your machine a few minutes if you’re testing this. Yes, it’s annoying to wait, but if you set it too fast, it might not be powered down fast enough to be able to start up again, and miss the timer. That way you won’t be able to tell if it’s actually working.

Hopefully you can now use your desktop or laptop to run some nightly scheduled jobs, without having to have a dedicated always-on machine around. Saves you time and money on power consumption, hardware and maintenance!

automatically delete emails with IMAP Cleanup

Let’s say you have this IOT device like a motion detection camera. Which sends you emails. Emails you keep in a separate IMAP mailbox. Lots of emails. So you want to delete those emails in some automated fashion, because doing that daily is oh so boring (remember, lots of emails).

Wouldn’t it be great if there was like some handy command line tool that would delete the oldest emails and keep like a 1.000 or 500 of them only? Well yes, that way I could script that annoying job and run it daily.

I didn’t find something that already did this. So I figured I’d be able to whip something up in an hour or so using PowerShell, or maybe a small .NET console application using an existing IMAP library.

Well, 3 different IMAP libraries and about 4 hours later I did have something rudimentary that finally did what it was supposed to do. Delete the oldest email, and leave the most recent 1000 (or whatever number you want) behind. That took longer than expected, so to regain as much time spent on this as possible I threw the ImapCleanup tool on GitHub, including binary downloads. I hope someone else will find this useful as well.

Beware though. This tool deletes emails. Be careful which mailbox you point this at, and make sure you test it in advance on a dummy mailbox. Maybe your email server behaves differently than mine, and important emails get digitally shredded by mistake.