I was able to reproduce this behavior in my local test instance. As a result, I have created this GitHub issue which you may subscribe to here. As a workaround, simply delete or rename the TentacleBundle folder on your Octopus Server.
Let me know if the workaround allows you to upgrade your Tentacles as expected once you get a chance to try.