Error messages are no longer displayed

Hi @mihai,

Thanks for getting back in touch and sorry to hear you’re still experiencing the issue.

Have you been able to try building and running the diagnostic tool described in the Diagnostics area of the GitHub issue you linked in your original post?

Also please retrieve and build GitHub - droyad/PSProcessTerminateDemo: Demo app to demonstrate a powershell/process execution problem and run the exe on the server and let us know what the output is. (We get you to build it so you can see what is going to be run).

I’m not certain as to what’s behind the NotSpecified error, it may be worth trying to increase the logging level on the target machine to “trace” as this will hopefully show more details.

Kind Regards,
Adam

Hey @adam.hollow,

I’ve build the exe, and I’ve run it on the server. This is the output:

PS E:\PSProcessTerminateDemo\PSProcessTerminateDemo-master\bin> .\Demo.exe
Starting child process
curl: no URL specified!
curl: try ‘curl --help’ for more information
PS E:\PSProcessTerminateDemo\PSProcessTerminateDemo-master\bin> & ‘c:\windows\system32\curl.exe’ -gwefgew
curl: no URL specified!
curl: try ‘curl --help’ for more information
PS E:\PSProcessTerminateDemo\PSProcessTerminateDemo-master\bin>

I’ve also increased logging level to “Trace”.

Thank you.

Best Regards,
Mihai

Hey @mihai,

Thanks for testing that out!

Now that we know what appears when run outside of Octopus, would you be able to try running the exe from within Octopus either using a deployment process to target the exe or the script console?

Hopefully we can see a different/better output from there, apologies for the back and forth on this one.

Kind Regards,
Adam

Hey @adam.hollow,

Don’t worry at all! I hope we will be able to solve this issue soon. :slight_smile:

I’ve uploaded a new file. (test project.log.txt)

Thank you.

Best Regards,
Mihai

Hey @mihai,

Thank you for sending that over, I have sent it to our engineers for review, we have contacted our enginners about this issue as its a bit of a head scratcher so we wanted their input and they did manage to take a look at this last night.

They think the issue stems from this line in your script kubectl apply -f 2>&1 , where you are redirecting errors to stdout.

They wanted to ask what you were trying to achieve with this so that an we may be able to find an alternative solution and see if that fixes this issue.

I look forward to hearing from you, I will let you know if the engineers find anything from that test project file though but it looks like this may be down to how the error handling is being redirected possibly.

Kind Regards,
Clare

Hello @clare.martin,

We used this redirecting “2>&1” only as a test while trying to fix the issues that we recently encountered with Octopus (I read this post: Calamari.exe swallowing errors - #4 by Matt.Richardson and it was saying that this may help fix the problem - but it didn’t in our case). So we are not using that in our nonprod or prod scripts and “NotSpecified” issue is still there.

Thank you.

Best Regards,
Mihai

Hey @mihai,

Thank you for getting back to us so quickly with that information, I have passed that onto the engineers, hopefully they find something useful in the test project build you sent over but I have asked them if there is anything else they would need to diagnose this if the build project log doesn’t show us anything.

I will be in touch when I have some more details from the engineers,
Kind Regards,
Clare

Hello @clare.martin,

If you think it will be useful for you and if you are available maybe we can schedule a Teams meeting(or other software through which we can do a call&screen sharing) so we can show you the exact behaviour and like this you can collect some valuable information regarding our issue.

If you agree feel free to e-mail me so we can setup the meeting details.
Thank you.

Best Regards,
Mihai

Hey @mihai,

Thank you for offering up a screen share, I have asked the engineers if a call would be beneficial. I have also asked what they would like to go through on the call as this is more backend code related so the engineers have a better idea than we (Support) do on what specifically we would need to look at.

They have not managed to take a look at your build test project yet so that might show us something, I have given them a nudge to see if they can see something from that test build project you sent over and if a call would be beneficial for them.

I will get back to you as soon as I hear from them, they are based in Australia so there is a bit of a lag between when I message them and when they reply as they have all gone off shift for the night now.

Hopefully we can get to the bottom of this soon for you!
Kind Regards,
Clare

Hey @mihai,

Our engineers took a look at the build project you sent over but it doesn’t show them much so they would love to hop on a call with you to see if they can troubleshoot this live. Our engineers do work in Australia but can work around customer time zones.

To that affect, what time zone are you based in and what dates / times are you free to hop on the call, we usually aim for around 30 minutes per customer call just to give you an idea of timings and ideally we like to use Zoom for the calls but can accommodate teams and WEBeX if that is your preferred communication method.

Let me know what time zone you are in and what times you have available this week or next and I will liaise with our engineers to see if we can book you in for the call.

Kind Regards,
Clare

Hey @clare.martin,

Thank you for your availability!

Now, to answer your questions, I’ll try to find a free spot on my calendar for next week and I’ll get back to you. I am based in Bucharest (EEST) and as far as I can see Australia and our working hours are not overlaping at all, so we will have to find a solution for this. Unfortunately, except Teams, we ca not use any other communication method. Do you think the engineers can install Teams on their side and create an account?

Thank you.

Best Regards,
Mihai

Hey @mihai,

Thank you for getting back to us, we can certainly install Teams that is not a problem at all, as for the time zones, we do have engineers on call and so they may be able to accommodate EEST, we will get something set up don’t worry.

If you can share your availability for next week I can take the times back to our engineers and we can see if one of the slots suits us.

The engineers are potentially looking at getting you a .net 6.0 tentacle to test but I have enquired about that as this seems like an Octopus Server issue to me (the engineers do know more about the backend code though so I will trust their recommendations on this one).

If anything comes of that in the meantime I will let you know, I look forward to getting some times from you and we can arrange the teams meeting.

Kind Regards,
Clare

Hey @clare.martin,

I will be available for the meeting on Wednesday 21, from 11AM to 4PM. Please let me know if and who will be available so I can create and send the meeting invite.
Thank you.

Best Regards,
Mihai

Hey @mihai,

Thank you for getting back to me with those times, that is 18:00 to 23:00 Australian time (the engineers time zone) and 09:00 to 14:00 UK time (my time zone) so I have asked the engineers if they can make Wednesday and if so what time would they like to hop on the call with you.

I can make Wednesday so can be available for the call but since this is more backend code issues ideally we would like one of the engineers to come on the call with us so they can run through some things with you on the call.

I will get back to you with if they are available on Wednesday and, if so, what time.

Kind Regards,
Clare

Hey @mihai,

The engineers have confirmed they would love to attend a Teams call with you for their 18:00 (which would be your 11:00). I will also attend with the engineer at my 9:00 (don’t you just love time zone changes!).

So to confirm, your 11:00am tomorrow (Wednesday 21st) would be great if you could set that up please.

I will DM you now with the email addresses of who will be attending so you can send us an invite by email, we look forward to trying to get to the bottom of this with you tomorrow.

Kind Regards,
Clare

Hey @clare.martin,

Yes, indeed :).

Plese let me know if you received it.

Thank you.

Best Regards,
Mihai

Hey @mihai,

I can confirm I have received the email invite and have confirmed attendance, I think Shanes would have auto declined it because its out of hours for him but I will forward him my email just to make sure he has the link for it and will let him know to accept the invite if he can.

They have confirmed he will be attending so it looks like its just declined it based on office hours.

I will re-confirm tomorrow around 08:30 my time (your 10:30) we have got everything and both of us are set and we can join the meeting and go through some things with you.

Kind Regards,
Clare

Hey @mihai,

Just to confirm we are all set for the meeting in around 15 mins time, Shane has accepted the invite so we look forward to hopefully getting somewhere with this issue.

Kind Regards,
Clare

Hey @mihai,

It was great meeting with you today, we just had a thought surrounding this that I wondered if you could try for us before we do a big deep dive into troubleshooting our end.

Since you upgraded your tentacle from Windows 2012R2 to 2019 we are wondering if there are some stale Calamari tools hanging around from a previous install.

Would you be able to try completely deleting everything out of the C:\Octopus\Tools folder (this includes all calamari folders in there) on the tentacle 2019 box for me and re-trying a deployment. When you re-try the deployment it will detect there is no calamari and will pull the latest version.

Let me know how you get on there, if that does not work to resolve the NotSpecified issue I will continue with my investigations.

Kind Regards,
Clare

Hey @clare.martin,

Nice to meet you too! :slight_smile:

I just did the test you’ve mentioned but the error is the same. Nothing has changed.
Thank you.

Best Regards,
Mihai