( https://pypi.python.org/pypi/octopus-http/0.6.4 ) Python, a general purpose programming language with an unparalleled level of expressiveness coupled with ridiculous productivity, and Octopus Deploy - easily the coolest way to deploy web applications. I won’t say which one is the steak and which one is the peanut butter. That hardly matters. These are two technologies that are not inherently designed to work together. Octopus Deploy specializes in deploying Microsoft .NET stacks. This isn’t to say (obviously) that it can’t do other stacks, but its initial raison d’etre was Microsoft deployment and we used it at Visual Storage Intelligence in that capacity. The results were superb. We went from deploying at most, monthly to sometimes deploying new features several times per day.
But with the upcoming April release of VSI 4, we changed our stack from purely Microsoft .NET to incorporate Python. This makes sense because at our core, we are a business intelligence service offering, and with all the buzz around BI, charting, big data, numpy, and pandas, with R and Python in the eye of the storm, well it seemed natural that we should give Python a whirl.
We reorganized our code, replaced some of our C# code with Python Tutoria PDF, and we changed our site from the aging Microsoft MVC 3 framework to use the Python / Flask microframework. We also took the opportunity update the way the app looks since we haven’t really updated the GUI’s appearance in the last several years.
The net result is that we reduced complexity by orders of magnitude moving several hundred thousand lines of C# into a very modest “thousand or so” lines of Python. Now we have fewer moving parts and a considerably faster turn-around time for new features and device support. To prove the point, we’ll be releasing support for NetApp cluster mode; a feature we were able to implement in a few days versus a few months which was typical before we reorganized. VSI 4 users will also see a whole suite of performance, enterprise reporting and business unit functionality that has never been available in our on-line service offering.
But how can we deploy with the same efficacy we did with the pure Microsoft stack? We didn’t change all of our stack, only part, and so I really wanted to keep Octopus Deploy as our continuous delivery server if at all possible. It turns out its super easy to use Octopus with Python.
We use Jet Brain’s Team City as our build server, though really you don’t need a “build” with Python given its interpreted nature. Nevertheless, TeamCity does check out our code and run the normal gamut of unit and Selenium UI tests. It does a great job of spotting dependency problems where developers (OK, I’ll own it - where I… ) forget to update the dependency file, requirements.txt, when I add third party modules to our stack.
If all that is green, we send it over to Octopus Deploy. This is done using NuGet packages, which are again, normally associated with .NET technology stacks. Normally you’d simply incorporate Octopack, a tool that is part of Octopus Deploy which utilizes Visual Studio’s project structure, and its incorporation into MS Build, to generate a NuGet package with no work on the developer’s part.
Since we don’t have a Visual Studio project file, and since by extension we don’t use MS Build, I didn’t consider Octopack a viable solution for package generation. Instead I used TeamCity’s built in NuGet Pack command. First I had to upload a copy of nuget.exe to TeamCity which it, in turn, distributes to its build agents. Next I set a build step that runs NuGet Pack. This requires a “nuspec” file, which is just a JSON formatted text file. I copied an example from NuGet.org and just replaced the basics with my project’s specifics, checked it into my subversion repo, and TeamCity can now generate my NuGet packages which it automatically publishes as artifacts to its internal NuGet feed server.
NuGet feed servers are basically like RSS or Atom servers. They syndicate content using an XML format. TeamCity has one built in, but if you want to use it, you have to turn it on in the administration options. Alternatively you can use Octopus Deploy Server’s NuGet feed. It doesn’t really matter which one you pick, but I found that since TeamCity automatically publishes its artifacts to its NuGet server, I was saving a step versus packing up my code and then explicitly sending it to Octopus’ nuget server.
Once Octopus has it, the process was not much different than it was for an ASP.NET app, but with Python apps we often rely on PyPy packages. This is the equivalent of publicly available NuGet or Chocolatey packages in the .NET world.
We use the pip tool to manage our packages, so I added a step after deployment to execute pip’s install command. This works against a simple text file conventionally called requirements.txt. Each time a developer adds a package dependency, for example support for SQL Server or perhaps a module for exporting to Excel, we update the requirements.txt file with a pip command called freeze. This generates the requirements.txt file which is then checked into our repo.
At the tail end of each deployment we configured Octopus Deploy’s tentacle to execute pip against this file. If the deployment target matches the existing requirements.txt file, pretty much nothing happens. If we’ve added something and it isn’t there, pip will download and install it.
And just like magic, we have a wonderful deployment dashboard that tells us exactly which version of our code is on each of our various environments.
Updating each environment is as easy as pushing a button. It is so easy to use we actually let make the boss do the deployments 
So while Octopus and Python might sound like a bad sushi night, these two technologies blend beautifully with just a little coaxing.