We are running TeamCity as our build server with Octopus for deployment. Occasionally, TeamCity errors out saying Bad Gateway 502.3. In the build log on TeamCity, it looks like an html page gets printed out. The issue seems to resolve itself if you wait 30 minutes to an hour and attempt to rerun the latest build from TeamCity. The problem is that it happens at the most inopportune times (i.e. when we are trying to push out a final release). Any thought to what might be causing this? I’ve outlined the part of the build log from TeamCity below that looked relevant.
Handshaking with Octopus server: http://octopus.serveraddress
[01:41:18][Step 13/14] Handshake successful. Octopus version: 3.3.10; API version: 3.0.0
[01:41:18][Step 13/14] Authenticated as: Project User <>
[01:41:18][Step 13/14] Finding project: Project
[01:41:19][Step 13/14] Finding channel: Releases
[01:41:19][Step 13/14] Finding deployment process for project: Project
[01:41:19][Step 13/14] Finding release template…
[01:41:20][Step 13/14] Resolving package versions…
[01:41:20][Step 13/14] Finding latest package for step: Deploy Project API
[01:41:51][Step 13/14] Error from Octopus server (HTTP 502):
[01:41:51][Step 13/14]
[01:41:51][Step 13/14]
[01:41:51][Step 13/14] IIS 8.5 Detailed Error - 502.3 - Bad Gateway
Hi Mark,
Thanks for getting in touch! That log shows us that IIS is somehow getting between TeamCity and Octopus and causing the 502.3 Bad Gateway.
IIS 8.5 Detailed Error - 502.3 - Bad Gateway
I’ve seen this happen when using IIS as a reverse proxy using ARR. What really has me confused is that it remedies itself after 30 minutes.
Does this sound about right, that IIS is in the mix here somewhere?
Hope that helps!
Mike
Good call Mike. IIS is in the mix as is ARR. TeamCity, Octopus and a Nuget server are running off that box. So what would be the solution here? Moving Octopus to its own space, or perhaps we don’t have something configured correctly?
Thanks
-Mark
Hi Mark,
Thanks for getting back to me! We, like TeamCity, usually recommend running our services on separate boxes to minimise these and other kinds of pain points. That being said it can be quite convenient to save resources, so I understand the motivation.
If they are on the same box, you could consider configuring TeamCity to talk directly to Octopus using a localhost:port address instead of routing via ARR - and then use ARR to provide external services with nicer URLs.
Otherwise I’m not an ARR expert so I’m not sure why it would start persistently returning an error for a period of time. Maybe it’s caching a bad response from Octopus and then going into a protective mode?
Hope that helps!
Mike
That gives me some direction. Thanks for you help with this.
Hi Mark,
No worries, and best of luck hunting it down!
Mike