Hi,
We are facing an issue with Octopus where even if we select an environment and lab combination, the corresponding deployment target doesn’t load which is why when we click on “Deploy” the deployment doesn’t get triggered.
Current version: 2022.4.8335
If we add one more tenant to it then the deployment targets for both tenants load and we are then able to deploy.
If we remove the second tenant from the list then the deployment target for the first tenant is visible.
Thanks for pointing this out, unfortunately I cannot seem to replicate it, I am on 2022.4.8344 so will spin up your instance version in a moment but I am only one version above you.
I set up a new project with one run a script step in and attached 2 tenants to that, I have two deployment targets connected to one of those tenants. When I go to deploy to one environment if I chose one tenant it loads the tenant preview fine and shows the two deployment targets associated with that tenant:
I then tried on a different browser just to rule out some cache issue and its the same, it still loads for me. I will test this on 2022.4.8335 and see if I get the same. Would you be able to replicate this issue again but this time when you select the one tenant can you grab us the HAR file when you load that releases page and then select the tenant to deploy.
We have a GitHub issue in for Content Security Policy issues which results in parts of the Octopus UI to not load (its environment specific too it seems) and I am just spit balling here and wondering if you are hitting that issue as it seems to affect random areas of Octopus. You can check the HAR file for the error below:
Feel free to send that HAR file to us if you can see that error and I can take a look at it for you. I have generated a secure link here for you to upload it to, as always let us know once you have uploaded it as we are not notified when customers upload to our secure site.
I will go ahead now and test this on your version and will let you know the outcomes, if you can take a look at that HAR file and see if you are getting a security policy error that would rule that GitHub issue out.
Is there anything you notice in your testing such as does this issue occur if you deploy to a certain environment or is it all environments you see this issue, is it certain tenants, so if you were to select a different tenant at the create release page does it show the preview or not?
My instances are pretty bare as they are only used for testing, my main instance on .8344 has tenant tags and they are connected to the deployment targets, I changed it to tenants not tenant tags but I still see the deployment targets on the deploy release page when I select a tenant.
I am hoping the HAR file gives us some more information, I can definitely see you have deployment targets associated with that tenant, it would still show the preview if you had not though, it just wouldn’t show any targets:
The CSP issue could definitely be a possibility here so until we can get a look at that HAR file we wont really know whats going on.
Does this happen for other users on other machines? That CSP issue does seem to be machine dependant so I am wondering if any of your other users are seeing the same issue?
I look forward to seeing that HAR file and carrying on troubleshooting with you,
Kind Regards,
Clare
Hi @clare.martin,
I have uploaded the HAR file.
Only one user has highlighted the issue and as far as I know its all environments that they see the issue.
Sorry been busy this morning but wanted you to know I have gotten your HAR file and will take a look at it after I had my lunch, I had a quick look and cannot see anything obvious, no CSP errors which rules out that GitHub issue.
I also find it interesting only one user is having the issue, does that mean you are not?
Is that HAR file from the user that is having the issue and do you know of any team scoping that could affect this users ability to not see that tenant preview. I realise they can see it when they add tenant 2 but I am wondering is there is a bug I have not yet managed to find because of permission scoping.
I will take a closer look at that HAR file in an hour once I have had some lunch and will test out some user permission scoping. If you can confirm that HAR file was from that users machine whilst that user was logged in and if they have different permissions to other users that would be great.
Also, daft question really but do they see the same issue if they use another browser or login in a private window (if you are using username/password to authenticate).
I look forward to hearing from you,
Kind Regards,
Clare
Sorry for the double post here but my testing has not brought back anything so I brought this up at our daily support meeting to see if the team had any other ideas.
If none of the above fixes the issue can you get your user to logon to a different machine and try this out so we can rule out a machine issue.
Also you mentioned that all environments selected reproduced the issue but did not mention if it was all tenants or just a select few, does it happen with all tenants?
I look forward to hearing from you, this is an interesting issue so I am keen to get to the bottom of why it is happening.
Hi @clare.martin,
The issue is reproducible across multiple environments and tenants.
More than one user has also faced the same issue. so its not limited to a single machine, single env or a single tenant.
Thanks for confirming that, the HAR file you sent over, was that from a user having the issue? If not are you able to send one over from a user that is having the issue please so I can take a look.
I am going to have a play again and see if I can get this to break with my test instance.
I will let you know if I get anywhere with this, no other users have reported this so far so it does seem environmental but I will have a play with some scoping and see if I can reproduce the issue.
I spent a few hours on this this morning and still cannot replicate the issue but I have noticed the HAR file from your user has significantly longer loading times than mine does for the tenant tags and deployment process preview, here is yours (loading time for each call is shown in the last column on the right):
And here is mine for a project with 13 steps all individually scoped to the tenant tags, with a channel scoped to the same tags which, from what I can tell, is how you have this project set up.
As you can see the first tasks calls are Octopus loading the tenant tags that are scoped to the channel and steps on the deploy release page, the recordportaloperation is the loading and then viewing of the deployment preview and customise section.
You can see in your HAR file its over 500ms to 1.2 seconds to load each tenant tag whereas mine is under 20ms for each call, this may explain what is going on here but my test instance is bare bones so I do not have a lot of data, it may be that its the data that is the issue here.
We changed the process about a year ago so that the deploy button on the right of the deploy release page does not go green until it loads all it needs to deploy that release, this includes all of the tenant information. From your initial first screenshot it looks like the deploy button is green (indicating all of the data is there) but it has not loaded the preview.
I am going to get this in front of engineering as it may be a bug, its possible I just dont have a large enough data set to replicate the issue but our engineers will.
To further help investigations am I able to get a few more things from you if you are not too busy please, they can all be sent via that secure link I put in here as it is still active:
A screen recording of a user with the issue clicking on that deploy release page and not seeing the preview then selecting another tenant (it would be beneficial to see what that deploy button is doing at certain stages of the user clicking on tenants).
If you cant get a recoding no worries, it would just be useful, I will pass this onto our engineers now with the HAR file and if you could let me know when you have uploaded the other files that would be great and I can pass those on too.
Hi @clare.martin,
Thanks for the detailed reply.
As you mention that it could be a data issue, that’s makes sense.
The space in question have a huge amount of data which they are reducing now because there are some 504 errors which are caused by the high volume of data. Also no other space has raised the issue.
As they are reducing the amount of data we will monitor the issue and it may improve as they delete data.
Thanks for getting back to us and confirming you do have a lot of data in that space, its also good to know only one space has this issue too, which further supports more of a mass data issue than a bug.
Would you be happy to wait until some data has been cleaned out of that space or would you like this further investigating, forums posts auto-close within I think its 28 days of inactivity but you are free to either reply to this within the 28 days or re-open another forum post and reference this one if you find reducing the dataset does not fix the issue.
I have informed our engineers of your response, if you are not happy to wait we are happy to investigate this further, we do have customers with huge datasets and have not seen this issue come up with them so it may be beneficial to get the engineers eyes on some of your logs.
I will say though, the way you have that specific project set up there is a lot of scoping to tenant tags you have going on, you have 13 steps in that project and all of them are individually scoped to it looks like three tenant tags and you also have the channel scoped to tenant tags too. I did notice all of your 13 steps are scoped to those same three tenant tags.
I am not sure if you have this setup like that for a reason, it would make sense if you had your channel scoped to say three tenant tags and you want 6 of those steps to only execute on two tenant tags and the other 7 steps to execute on all three tenant tags but it may be beneficial for you to take the scoping off of those steps if you want them all to execute on the same three tenant tags your channel is scoped to. I think because there is so much scoping going on in that project its adding to the slowness of Octopus figuring out what tenants it is allowed to deploy to.
I did set my test project up like that and did not experience the same issue you did but again I have 4 tenants, 4 tenant tags and one channel so my dataset is tiny.
If you are happy to wait we are happy to investigate further later down the line if you find reducing the dataset does not make a difference.
