v2022.01 (Build 2133) - The proxy configuration is not being honored, the connectivity check, therefore, does not traverse our proxy and in turn, the request gets blocked by our firewall.
I don’t have many options left to try, I could create a Tentacle, either locally to the Octopus Server I suppose as a workaround, however, this problem should be addressed in this scenario as well as the one mentioned in the linked topic and issue.
Thanks for reaching out, welcome to the Octopus community!
I’ll run some tests to see if we have regressed on that previous issue or if there could be something else going on here. Would you be able to please send through the Task log for a failed Health Check? You should be able to upload it securely here, let me know if there are any issues.
I’ll let you know if I have any updates or questions, please feel free to do the same!
Thanks for sending that through, confirming I have received it ok!
Unfortunately I haven’t been able to reproduce any issues on my end. My WebApp’s Health Check traffic when run directly on the Octopus Server is appearing in my proxy’s access logs ok.
I’d still like to help figure out what’s going on for your setup however, would you be willing to share how you have configured the proxy in both Octopus and Windows?
The changes that were introduced with v2021.1 regarding proxies are outlined in this pull request if you are interested in reading about it, however it mentions that we weren’t always honouring the HTTP_PROXY environment variables. I’d just like to confirm that you don’t happen to have an environment variable set for NO_PROXY by any chance?
This is set as a Custom Proxy in the Octopus Server Manager.
Proxying is not configured for Windows i.e. either of the mentioned environment variables exist nor is anything set in IE > Connections, and as far as I can see there are no differences between the 2 servers i.e. the one running v2019.6.7, wherein the features work fine and the one running v2022.01 (Build 2133) which doesn’t pick up the proxy settings and therefore gets blocked by our firewall; I can see it being blocked by the firewall attempting to contact Azure on 443 directly, instead of via our proxy on our custom port.
I am going to set HTTPS_PROXY and report back, however, I want to make it clear that this is not set on the server running v2019.6.7.
I was able to successfully reproduce an issue with the ‘Custom’ proxy option in Octopus as you’ve described, where I can see that it’s not respecting the proxy port configured for this health check.
I’ve kicked off a discussion about this with the engineers and will keep you posted with any updates, or a public Github issue for tracking the resolution. In the meantime as a potential workaround, the ‘Proxy configured in Internet Explorer’ option should use the correct proxy port, I found it was only the ‘Custom Proxy’ option that was effected.
Thanks for raising this with us, feel free to reach out if you have any questions!
I will have to explore the options, as we explicitly do not allow web browsing traffic from this network zone, therefore, I suspect that I would end up attempting to create a cocktail of conflicting settings to get the behavior that we want in the short term but in the longer term, this is obviously far from ideal.
Thank you for getting back to us with that information, I will leave this one to Finnian if that is ok as he has the reproduction available. I think this will end up as a GitHub issue and will be looked at by our engineers as a bug.
If you can try one of the workarounds Finnian suggested to get you up and running for now, Finnian will see your response but I will also message him to make him aware there is an update to this ticket.
If a GitHub issue is created then our engineers will look at a permanent fix for you so you will not have to keep the workaround on your instance.
I hope that helps alleviate any concerns you have with regards to implementing a workaround, we always look at permanent fixes for customers as workarounds are supposed to be temporary so rest assured we would aim to produce a valid permanent fix for this issue.
Apologies, I created an issue yesterday but forgot to share it!
The devs have a good idea about the cause of the issue and so a fix should be available in the near future. You can follow along here for updates, it will even post the version number that will contain the fix once it’s resolved.
To prevent having to change any network/firewall settings, I would recommend (as you also suggested earlier) temporarily configuring an External Worker locally on the Octopus Instance to perform the health check. We do typically recommend to install Workers on external instances as it can impact performance, but if it’s only performing this health check, then it shouldn’t have too much of an impact.
Feel free to reach out if you had any questions at all, thanks again for pointing this out to us!
I’ll report back here with an update once I know which way the wind is going to take me, I suspect that I will do nothing for a while (probably until a new build is available) but indeed in our situation certainly presently the performance impact would be very negligible.
Just keeping this open, as of now, ~4 weeks later we have chosen to not worry about a workaround for the issue, I am looking forward to a solution or information to advise which release this will be resolved in so that I can decide if we are going to build our upcoming future processes for Azure and AWS around Octopus Deploy or not.
