Urgent: Use a custom proxy server don't work with https feeds

Legacy Uncategorised
1

Hi,

We have started to roll out Octopus to our On-Premise customers and find that most of them requires the use of a proxy to get access to our NuGet Octopus feed on Internet. That should all be fine with the new “Use a custom proxy server” setting in the Octopus Manager. But it turns out that this does not work.

My setup for testing this is like this:
I have a proxy server with the an address like this http://myproxyserver:8888. It requires authentication
In Octopus manager I then configure:
Proxy Host = myproxyserver
Proxy Port = 8888
Proxy Login and Password

I then add the feed in Octopus Library: https://myofficalserver/mynugetfeed
This feed then always fail when I test it with this exception

NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://myofficalserver/mynugetfeed. ---> System.NotSupportedException: The ServicePointManager does not support proxies with the https scheme.
   at System.Net.ServicePointManager.FindServicePointHelper(Uri address, Boolean isProxyServicePoint)
   at System.Net.HttpWebRequest.FindServicePoint(Boolean forceFind)
   at System.Net.HttpWebRequest.set_Proxy(IWebProxy value)
   at System.Net.Http.HttpClientHandler.SetDefaultOptions(HttpWebRequest webRequest)
   at System.Net.Http.HttpClientHandler.CreateAndPrepareWebRequest(HttpRequestMessage request)
   at System.Net.Http.HttpClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.HttpHandlerResourceV3Provider.CredentialPromptWebRequestHandler.<SendAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.HttpRetryHandler.<SendAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.HttpSource.<SendWithCredentialSupportAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.HttpSource.<GetThrottled>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.HttpSource.<SendAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<CreateODataServiceDocumentResourceV2>d__9.MoveNext()
   --- End of inner exception stack trace ---
   at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<CreateODataServiceDocumentResourceV2>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<TryCreate>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.Types.SourceRepository.<GetResourceAsync>d__11`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.PackageSearchResourceV2FeedProvider.<TryCreate>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.Types.SourceRepository.<GetResourceAsync>d__11`1.MoveNext()

I’ve verified that the proxy works fine by also configuring the use of the same proxy with chocolatey. This works fine with https feeds. Other .Net code I have that use WebClient also works fine with the proxy and https download urls.

This looks like a bug. The error message suggests that it tries to use https for the proxy url. It’s the feed url that is https not the proxy.

I hope you can help me find a quick resolution/bug fix for this issue

Regards
Jan

2

Hi Jan,

I was able to replicate the issue and I’ve posted a bug here.

https://github.com/OctopusDeploy/Issues/issues/2926

When setting up the proxy server in Octopus Manager, you can try entering “http://[proxy]”, but the scheme will be automatically removed. However there’s good news - if you edit the OctopusServer.config file, you can specify the scheme:

  <set key="Octopus.Proxy.ProxyHost">http://localhost</set>

When I did that, the test worked!

Let me know how you go,

Paul

3

Hi Paul,

We have automated the complete setup and configuration of the Octopus installation.
We use the octopus server command line to configure the proxy from a powershell script. This is convenient. Scripting direct updates in the octopus config file is not so convenient. I’ll try it out, but I would really prefer a fix for the bug, to avoid errors during later updates after it is fixed. Will the url with http://myproxy still be valid in the config file after the bug is fixed?

4

Hi Jan,

Yes, it will still be valid. We’ll fix the bug but not sure how quickly that will happen.

Paul Stovell

5

Hi Paul,

I’ve tried this change by updating the config file. I get another error now. I also test with Fiddler as a proxy, but on another machine. To test this properly I’ve blocked internet access for the test server in the company firewall. Only way it can get internet access for port 80 and 443 is through a proxy on another machine. The proxy still works fine from chocolatey and custom scripts on the Octopus server. But now fails in Octopus with this other error. Please test and see if you can reproduce this.

NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://www.myget.org/F/octopus-dependencies/api/v3/index.json. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 137.116.207.86:443

Regards,
Jan

6

Hi Jan,

Thanks for getting back to us! I’m following this up with another developer from the team. At this point it appears to be either a bug in NuGet 3.4.3 itself or something wrong in the way we’re holding it. The way Proxy support is implemented in NuGet doesn’t give us many (if any) options to tell it which Proxy server to use, but we will get to the bottom of this fairly quickly.

Probably the best place to follow this work from here is on the GitHub Issue: https://github.com/OctopusDeploy/Issues/issues/2926

When we get to the bottom of this you should be configuring your proxy server in Octopus Manager exactly how you did in the beginning without hand-editing the .config file.

Proxy Host = myproxyserver
Proxy Port = 8888
Proxy Login and Password

Hope that helps!
Mike

7

Hi Mike,

I see you have made some progress on this issue. Any ETA on a fix?

Regards
Jan

8

Hi Jan,

I’m glad you’ve been following the issue. The biggest thing left is regression testing at the moment and then the fix should ship in a patch when it is ready. I can’t give you a specific ETA though but I’ll keep posting to that issue as we progress.

Hope that helps!
Mike

9

Hi Mike,

Any news on this? Will it be part of Octopus 3.5.3?
We are currently blocked from installing Octopus on some On-Premise sites due to this issue.

Regards,
Jan

10

Hi Jan,

Thanks for keeping in touch! We’ve had to juggle quite a few priorities this week and we haven’t finished this work in time for 3.5.3.

I’m sorry if this isn’t the news you were hoping for, but rest assured we are burning this issue down and it should ship very soon.

Hope that helps!
Mike

11

Hi Mike,

I see that your pull request to fix this is completed and is for review. A lot of changes. Good work :slight_smile:
When do you think the review will be done?

Regards,
Jan

12

Hi Jan,

It would have been nice if this was a small change! We have needed to juggle a few priorities this week, otherwise it would have already shipped. I know you’re keen to see it shipped, and so am I.

Everything going well I would expect this to ship early next week. I’ll keep in touch if anything surprising comes up.

Hope that helps!
Mike

13

Hi Jan,

That fix shipped in Octopus 3.7.4 - let me know how it works out!

Please note we discovered an issue when testing where the feed credentials are cached on the first attempted connection to an external feed. This looks like it’s buried very deep in NuGet and we need to figure out the best way to move forwards. In the meantime there is a viable workaround: https://github.com/OctopusDeploy/Issues/issues/2959

Hope that helps!
Mike

14

Thanks Mike,

I’ll test this as soon as possible

Regards,
Jan

15

Hi Mike,

I’ve tested and it seem to work fine now

Thanks,
Jan