Project Groups purpose and role

Legacy Uncategorised
1

I know there are a number of questions regarding project groups, but they all seem to either just end or suggest they’ve been addressed in a particular release or suggest that a particular role of it has been replace by something like lifecycles, and I’m just trying to figure out whether there is an easy way to do things that currently seem a bit problematic, and whether the way I’m doing things is the way things are intended to be used (or whether they have been deprecated in favor of other methods).

GUI

Currently I have about 40 projects in Octopus across a smattering of teams (if you’re looking to break them down probably 4 distinct teams). I have since the beginning created Project Groups to segment these in the GUI, and they show just fine both on the main dashboard and in the projects area. That is assuming you stick with the “All” option. If I have someone select fewer items because they aren’t interested in what the other 4 teams is doing, if I were to add a project to a group they would have interest in (because say someone has developed a new Windows service for that project), they have no way to know that it has been added (without happening to reconfigure the dashboard). To be clear the “Project Group” does show up in the multi-select, but there is no mechanism to select that project group. Or maybe putting it more succinctly, it SEEMS that by the design of the interface that there should be a mechanism to select a project group but there isn’t. And it appears it has been that way for a long time so that it hasn’t been an added feature or hasn’t been a issue to get upvoted makes it feel as though there is another way around it. This of course presents a similar issue in users/teams when assigning permissions except the interface is different and you have to remember what you had in each project group. With only 40 projects, I’m just looking for advice/suggestions on how people who have been using this for longer (or who have more projects) securely manage this without forgetting things.

Deployment

That was in the GUI, which there is no “easy” way for me to address (at least for the view maybe I could rewrite the permissions but then I’m managing it in two places). Now when it comes to deployments, looking at the API, I think I should be able a way to deploy all of a project group. However when I was reading other discussions I thought (although now I can’t find it), that the mention was that this was going to be handled by lifecycles going forward. Which looking at how I’ve applied lifecycles would mean I need to rethink my application of them.

I don’t want to write something and find out there was a different intent (or something perhaps I’m missing). Project Groups seems like a great idea to help manage things that are connected, but from what I’ve been there for quite a while and some of the basic reasons for it being there haven’t been fleshed out. And it seems like it would be at a spot where it should be a pain point for others (it is for me) but hasn’t generally been seen as a problem by the majority, so it feels like I’m missing something like maybe a different way to do the same thing?

2

I should also add that I find myself staring at times at my Octopus dashboard and wondering. For a couple of the development groups, we have a fix release project on top of the normal release (different lifecycles if nothing else). Now if use Project Groups for anything other than how they display on the screen I may want to create a different project group for the fix instead of putting it in the regular project group. I realize that there are ways of passing variables from the build system which maybe could get around this, but the point is if I were to use project groups for deployment I shouldn’t have the fix project in the same project group as the normal release.

Anyways, I know this is all long winded. Sorry.

3

Hi Mike,

Thanks for getting in touch! Wow, I’ve taken a bit of time to read through and imagine your scenario and your thoughts about Project Groups. It’s good to have the feedback. Now let me see if I can do justice to your questions in my response!

Background

Project Groups started their life as a simple way to group Projects on the Dashboard and Projects list. Over time they got some more features like controlling Retention Policies, but we found that a much better model was to introduce Lifecycles. This way you get fine-grained control over the Lifecycle of each Project in a Project Group which may be different from each other.

Nowadays

Project Groups still act as a grouping mechanism. They also loosely help with Project Permissions, but not quite as well as we’d like.

There is also a suggestion on User Voice to provide for collapse and ordering etc.

Configure Your Dashboard

Oh boy, wouldn’t it be nice if you could select a Project Group for your Dashboard? And whilst that page makes it look like you could move a Project Group it is Project by Project. There is a similar suggestion on User Voice about a tabbed Dashboard where you could add your thoughts, or another on a Virtual Dashboard, or feel free to create a new Suggestion, but it’s usually better to jump on an already popular suggestion.

Deploy Project Group

Sorry to be the bearer of bad news here, but we are not likely to implement this at any time because of the variability between how Customers group Projects. Many of our customers find the API is really good for building your own super-orchestrations over the top of Octopus. In your case you could build a simple application that lets you build up a coordinated release of several Projects in a Project Group.

Lifecycles won’t help with this either unless I’m misunderstanding your scenario. The primary role of Lifecycle is to allow for you to design how a Project flows through Environments, and the Retention Policies at each Phase.

Maybe you’ll get some traction from the upcoming support for what we’re calling Channels in Octopus 3.2 (due for pre-release in the next couple of weeks).

Fix Project

I must admit, you lost me there. Again, the new Channels support may be interesting here where you will be able to modify the Lifecycle, Variables, or the actual Deployment Process for a Project based on what Channel you select. This will address things like Feature Branching, HotFix Deployments, etc.

Now who won? The Question or the Answer? :wink:

Hope that helps!
Mike

4

Thanks for the reply. The answers make sense, but let me clarify a couple of things that may be helpful.

Fix Projects – So after a release typically what we have seen is we have part of our R&D team assigned to the project continuing on with new requirements and there is another part of the group who works on any “hot fixes” that need to get addressed in the previous release. And testing may have to be addressed with both at the same time. So the way we’ve dealt with this is to have two different projects where for dev and QA they may have different settings (the settings are certainly different for IIS or Tomcat site so they can have two sites up at the same time). Once we get to Stage and Production they get pushed to the same location. Because they are ultimately for the same Project (big P), it initially seemed to make sense to include them in the same project group. There probably is a saner way to handle this, but I haven’t figured out how to do it, without maintaining variables in a different location and having the build system push them. Without having a view into how versioning will work, it does sound like it still might help address this, but we’ll see.

GUI – But as one works with the tool, and starts handing responsibility for things over to other team members, I have found that there are things that I did because of how I thought pieces would come together that might not make sense (now). Explaining to someone on your (“your” means “mine” here) team that any time you add a new project to a project group you still have to go back and adjust permissions for the teams you’ve created, invariably has them asking “Why?”. You also want to make sure as you hand the pieces off that you haven’t missed something that will make the job of those you are handing it off to easier (and also help them not to miss things). In the GUI, where there was obviously the understanding that you’d want to group teams together so you didn’t have to manage permissions for each user on each project, it (to me) seems counterintuitive when you’ve got a mechanism such as “Project Groups” (and a way to get to them in the API), not to have a way to assign permissions on a group in some way or limit the dashboard based on the group(s) you are part of. However from what I’ve seen of the user voice items, it seemed like there is either hesitancy (or lack of interest) from doing it that way. So I was thinking maybe I was missing something. Sounds like it’s mostly lack of interest from other users for this feature. Which means I’m still stuck with really wishing I knew how people with many projects successfully keep track of permissions. But, that’s not something you can necessarily answer.

Deployments - One of the things I was tasked with when I came in to my current company was figuring out a deployment mechanism. I looked at a bunch of different options, and even temporarily home-rolled our own because I hated doing things manually in the meantime (and keeping track of it all). From a GUI standpoint it was basic (and honestly not having to maintain the GUI and all the permissions/logging around that is one of the things that really attracted me to Octopus), and had only a multi-select that you used to choose which projects you wanted to push at the same time and environments to push to. It again seemed like in Octopus it would at least be possible to say “Deploy this group of things” since from reading use cases in the support forums that does seem like something most people are doing. Now I realized that maybe not everyone thought like I did. And as I’m not the creator of this or even an early adopter, maybe I didn’t use project groups the same way, or maybe there was something else that treated the grouping of display different than actual deployment (say lifecycles) or maybe even there was the possibility I should use a “super” project which simply contains a list of deployment steps which could point to other projects. I don’t/didn’t really care too much as long as I didn’t have to maintain it, but again it seemed like something more than just one person (me) would want to do, but I don’t see all that much in the way of questions or push for this, so I thought I would ask if I was missing something because that it would seem like something most others would be interested in, but that there weren’t that many suggestions on how to address it would seem like I was missing something. :slight_smile:

It sounds like that isn’t the case and I should look at home rolling something (but then of course I’m back in the business of maintaining permissions and possibly logging again). Not the end of the day, and I certainly appreciate that API wise what I need is probably exposed, but I was hoping NOT to have to reinvent the wheel or at least figure out some way to plug it into the GUI that already existed so I don’t have to handle things Octopus already generally does well.

-Mike

5

Hi Mike,

Thanks for your response, and I won’t try to compete on length this time! :slight_smile:

Fix Projects
As it turns out this will be the perfect fit for the upcoming Channels feature we’re shipping in 3.2 (with a pre-release coming very soon). This will allow you to support multiple release versions among others. In this case you’ll easily be able to deploy bug fixes to 3.0.x while you push out test releases of 3.1.x, and introduce new/different steps along the way, all in the scope of the same Project. Hopefully this will help.

Project Groups and Permissions
Yep, I agree in your situation (and I’m sure many others) Project Group would be a very convenient place to manage permissions. My best suggestion would be to try and get votes moving up on User Voice as I’d mentioned.

Deploying Project Groups
I agree, I also wouldn’t want to go and be re-implementing the entire security model. The idea behind a Project is to model all of the things that need to be deployed at the same time, which is natural when there are <10 things, but starts to fall apart when there are >10 things.

People have been hyping about Microservices recently and this is where we are seeing people asking for the ability to deploy an entire Project Group of “Microservices”. Done right, each service should be versioned and deployed independently to other services, potentially on their own cadence, so they should really be modeled as individual Projects in Octopus. Given that independence the version numbers won’t line up across services, so it makes the concept of deploying a Project Group just as difficult as deploying each Project.

To get a feel for your situation, how many Project Groups do you have and what are their average number of Projects? Also how many Package steps do each of your Projects have?

Hope that helps!
Mike

6

Fix Projects

I’ll be curious to test out Channels. I think my use case might be a slight deviant from what you are thinking in that I would want the fix site to go to IIS as “SiteFix” rather than “Site”, while continuing development would go to “Site” (rather than “SiteFix”). Now with some DNS hacking I can probably get around that (site1 vs. site2 or some other incrementing strategy), but knowing the developers and QA they like knowing where things are going (without digging), so forgetting the technical aspect that may not be acceptable to them. But I’d be all for it. :slight_smile:

Project Groups and Permissions

OK, I’ll go down that route. Hopefully I pick the one with the most momentum. :slight_smile:

Deploying Multiple Projects

So this is a bit of a tricky question in that we’ve played around a little bit with combining projects we are assuming will always be deployed together as well as install steps that may not normally be part of a deploy (we just had an instance where fonts had to be installed… now if we deploy to a new box by adding the roles, we may want it to add fonts, or we may not because we’ve moved the configuration elsewhere so not including it as part of the normal deployment seemed to make sense). A perhaps better example is if we are creating a new service connecting to an existing DB, just as we want the service to install with that user, we also need to make sure that something adds the user to the appropriate DB. Having that as a separate project would make sense in that we don’t always want it to run, but would want it as part of a deploy sequence. (P.S. One other item that has really come up lately. It would be nice to be able to run a step template from the Tasks page, especially for things like stopping all Windows services that are part of a project group. :slight_smile: But I’ll do a search for the step template thing in the support suggestions.)

All of that is somewhat tangental, except that understanding at what point in the process the Octopus team might be looking to glue things together is helpful in understanding the best way for me (us) to structure projects.

Anyways, so to answer your question count what I’m going to do is give you number of packages we create because each one is either a web service, web site, a windows service, a set of plugins or a DB deployment. For one of our project groups that I currently see being deployed as a group > 75% of the time. What you’ll see is I’ve had a hard time figuring out the best logical way to group them already. :wink:

Project One
Project Group One (Web Sites/Services)

  1. We have 4 web services, and one set of 1 plugin (currently this is one project)
  2. 2 additional web services/sites. Each are a separate project.

Project Group two (Windows Services)

  1. Currently have 9 windows services (but I’m expecting more). Each of these are their own project

Project Group Three (DB)

  1. One set of DB scripts

Project Group Four (Documentation —> Help)

  1. One help site

So if you count all of those up, currently 15 different projects will get pushed, and 18 different nuget packages. That’s assuming we don’t push any phone components (which we might), and we don’t have any additional installation material (which again may not be suited for Octopus, but maybe it is).

Project Two
I’m not going to quite the level of detail above, as it is much more straight forward. Two Project Groups (one is again “docs”) —> 7 projects —> 7 sites/services, (7 packages).

Others
The rest of our deployments would group into two or less projects (for a normal deployment).

7

Hi Mike,

Just thought I’d touch base and see how you were going? Have you had a chance to test out Channels and see if it suits your situation? Feel free to get in touch if you want to discuss anything further.

Hope that helps!
Michael

8

We finally got upgraded to 3.x (3.2.3 as it turns out) about 2 weeks ago. Since that time, I have been trying to address some items that are also tactical. Trying to find a happy balance with variables (global/project specific) and site/service creation/promotion on new hardware (permissions on folders/databases/app pools with different values than what we can with the deployment). So unfortunately I haven’t had a chance to try channels yet.

-Mike