Partial Database Migrations with DbUp

We started using DbUp for database migrations several years ago around the same time that we started using Octopus. This works reasonably well so long as the are deployed in the same order that they are created. Unfortunately, with many different projects, there are many times where this is not the case. We handle this by using a development and release branch. Once a project has passed the development milestones, it is promoted to the release branch and deployed to a pre-production environment for final testing. Unfortunately again, this doesn’t always go as planned, and some projects end up in limbo awaiting final release, while other projects move forward. This creates situations where there are multiple sets of scripts in a release, when only one set is intended.

We are currently working around this by using the following workflow:

  • Run the DbUp project with a flag that just has it list the pending scripts
  • Manual Intervention Step. This halts the deployment so that someone can verify the list of scripts.
  • The developer, when preparing the release, includes a list of scripts so this verification can happen.
  • If the lists match, the deployment is allowed to move forward. If they don’t, the deployment is cancelled.
  • If the deployment is cancelled, the developer works to create a new build which only contains the necessary scripts, and a new release is created.

This system works, but it has its fair share of shortcomings:

  • The process is manual and error-prone:
    • The developer can make a mistake when creating the list.
    • The user verifying the list can make a mistake.
    • It’s common that a deployment is left in the “waiting” state rather than being cancelled.
  • Creating a new build that only contains the needed scripts is a messy process that potentially creates other problems later.
  • Having a manual process attached to an automated deployment process seems wrong, at least when we’re not dealing with an exception.

Let’s set aside the idea of having another “merge gate” via another branch that could help solve that. There are cultural issues at play there that make that difficult to easily move forward on.

So my current idea involves moving the manual aspects of this process into an automated process. I’d like to be able to include a file with the release that contains the list of scripts to be executed. That list of scripts should be evaluated against the list of scripts to be executed by DbUp. If they match, the scripts are executed without restriction. If they don’t match, but the DbUp list contains all of the scripts in the release list, execute the scripts filtered to that release list. If the DbUp list doesn’t contain all of the scripts from the release list, fail the deployment.

I was considering using artifacts for the purpose of providing the release list, but I don’t think that would work the way that I’m thinking. I would like the release list to be separate from the build if possible, as that would add some flexibility that I think would be useful for our purposes.

Any help, ideas, etc. are appreciated.

Hi Eric,

Thanks for getting in touch. I think we’ve boiled your question down to I would like the release list to be separate from the build if possible, so it sounds like you’re trying to compare 2 lists of migrations, and act based on that comparison, and store is somewhere?

Our first idea is could you leverage the built in package feed to store your migration list/decision data?

Could your build pipeline or Octopus create a package with the list of migrations, that way it is stored and reusable?

If we didn’t get a good grasp of what you’re trying to do please let us know.

Regards,
Nick