Unable to connect to teamcity after upgrade to latest version

Hi,

After upgraded from 2019.3.4 to 2021.1.7638., Octopus is not able to retrieve release information from teamcity.

The V2 feed at ‘http://teamcity/guestAuth/app/nuget/v1/FeedService.svc/FindPackagesById()?id=‘xxxxxxxxxxx’&semVerLevel=2.0.0’ returned an unexpected status code ‘503 Service Unavailable’.

.

OS: Windows Server 2016 version 1607 (OS Build 14393.3504).

Any help will be much appreciated.

Regards,

Raymond

Hi @raymond.lau,

Thank you for contacting Octopus Support.

By chance, is the TeamCity service behind a proxy? If so, would it be possible to bypass the proxy temporarily to test? Also, it looks like the URL may be pointing to a V1 endpoint. You could try changing this to V2 to make sure that isn’t the culprit.

If the above does not work, would you mind sharing a HAR file that we can review to help identify the cause: Capture and export a HAR file - Octopus Deploy

I look forward to hearing back from you.

Regards,
Donny

Hi @donny.bell,

Can Octopus use V1 instead ? Our Teamcity server is in 2017.2.1 (build 50732) version ,not sure whether it has NuGet API V2 Feed .

Regards,

Raymond

Hi @raymond.lau,

Thank you for getting back to me.

In order to use FindPackagesById, you’ll need to hit the Nuget API V2 endpoint for TeamCity. It looks like V2 should be available in TeamCity versions 9 and newer.

Let me know if moving to the V2 endpoint resolves the issue.

Regards,
Donny

Hi @donny.bell ,

I’m able to use the exact same URL on the windows server to retrieve nuget package information from Teamcity. It seems only Octopus Server can’t retrieve information from teamcity.
And the situation is that before upgrading to 2021.1.7638, in version 2019.3.4 on the same windows server, we are able to retrieve nuget package information from the same teamcity URL.
are there any particular requirements for version 2021.1.7638 ?

Regards,

Raymond

Hi @raymond.lau,

Thank you for getting back to me.

It looks like this may be a V2 feed despite the V1 endpoint/URL. Assuming that is the case, have you tried replacing the DNS name with the IP address of machine TeamCity is installed on?

Also, can you confirm if there is anything between the Octopus Server and the TeamCity server such as a proxy or load balancer?

Let me know at your earliest convenience.

Regards,
Donny

Hi @donny.bell ,

Look like there is proxy setting on the windows server, we are trying to disable it.
However, it is weird that when running Octopus Server 2019.3.4 on the windows server , it works fine.
Only not working after upgrade to 2021.1.7638 on the exact same windows server.

Hi @donny.bell ,

After we removed proxy settings from windows server, somehow Octopus Server’s teamcity http request still goes to our proxy server. Wondering whether there are anywhere in octopus config files , registry or db saved the proxy setting ?
From Network Monitor, we could see internet explorer it going directly to the teamcity server - first six lines. However, Octopus server is going to our proxy server - the last line in the screenshot.

Regards,

Raymond

Hi Raymond,

Just stepping in for Donny as he’s signed off for the day.

Is your Octopus Server configured to use a custom proxy? I’ve linked our documentation which outlines how to configure a ‘Web Request’ Proxy for Octopus here. This will modify the XML file stored in the Octopus directory where the configuration is stored.

Release 2021.1.7638 should have resolved a known issue with proxies seen here, but it’s possible that this change has outlines another issue. I’ll run some tests on my end to see if I can reproduce this and figure out what’s going on!

Feel free to reach out if you have any questions!

Best Regards,

Hi Raymond,

I forgot to ask previously, could you please confirm which version of the Octopus TeamCity plugin you are using? :slight_smile:

Best Regards,

Hi @finnian.dempsey ,

I beleive Teamcity is not an issue here, the traffic hasn’t reached to teamcity.
What we observed is Octopus Server using proxy to connect to http://teamcity although setting Octopus not to use proxy server.
image

and the windows server is configured not to use proxy.
image

from Network monitor , we could see Internet explorer is accessing the same url directly .
But the http request from Octopus Server is going to proxy server first.

.

Wondering whether that lingering proxy address in LAN Setting causing Octopus still using the proxy setting even it is configured not to.

Here were what we did:

  1. on “A” windows server 2016 , install Octopus Server 2019.3.4, everything went smoothly

  2. try create release, and successfully retrieved nuget package information from teamcity
    here is a screenshot of network activity captured. from Frame# 2019 to 2034 , octopus server made
    queries to the sql server, and retrieve values for feeds-team-city , after that made http request to teamcity directly without using proxy.

  3. on “A” windows server 2016, upgraded Octopus Server from 2019.3.4 to 2021.1.7638, everything went smoothly

  4. try create release, and now Octopus server is not able to communicate with teamcity
    here is a screenshot of network activity captured.
    from Frame#231 to 247 are SQL Query, and at the end retrieved values for feeds-team-city.
    From Frame#249 to 250, it is a request to query proxy server.
    From Frame#254, Octopus Server is using proxy to make http request to http://teamcity

All these were done on the same “A” windows server 2016. Everything works on version 2019.3.4.

I have done complete uninstallation removed all versions of Octopus Server, still come to the same result.

Hi @raymond.lau,

Thank you for getting back to us.

In upgrading from 2019 to 2021, Octopus moves from .NET Framework to .NET Core. It may be worth a quick check to see if the machine that Octopus Server is installed on has any http_proxy environmental variables set that could be getting respected by .NET Core. This could explain why something different is happening versus the proxy settings in Octopus and Internet Explorer.

Let me know at your earliest convenience.

Regards,
Donny

Hi @donny.bell ,

There were http_proxy and https_proxy system environment variables.
was able to create release and retrieve nuget package information from http://teamcity after removed those env variables.

Thank you so much for the help.

Regards,

Raymond

2 Likes

Hi @raymond.lau,

Thank you for the quick reply. I’m really glad to hear you were able to get it figured out.

If there is anything else we can assist with, please let us know.

Regards,
Donny

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.