AWS-LOGIN-ERROR-0003 running AWS CLI step

Hi,

I am trying to use the AWS CLI step but am always getting the error:

System.Exception: AWS-LOGIN-ERROR-0003: Failed to access the role information under http://169.254.169.254/latest/meta-data/iam/security-credentials/, or failed to parse the response. This may be because the instance does not have a role assigned to it. For more information visit Deploy an AWS CloudFormation template - Octopus Deploy —> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found).

The script I am trying to run at the moment is just a single call to Get-STSCallerIdentity to make it as simple as possible (I have also tried just having an echo “test” line as well so the issue doesn’t seem to be related to the contents of the script).

I have followed the steps in Connect an AWS Account to Octopus Deploy - Octopus Deploy to add the user info to Octopus. When I test the the account info this indicates success. Then I have created the variable as explained in Run the AWS CLI in Octopus Deploy - Octopus Deploy and I have used the “Preview Variable” to make sure that my account variable is getting substituted correctly.

This is how I setup the account in Octopus:

Then this is how I have my variable defined in the project:

Then this is the result of Preview Variables (if I click on Accounts-1 then it navigates to the “Fargate Deployments” account:

And then this is how I’m selecting the user account in the AWS CLI step:

But I cannot get anything to work…I always get this error:

19:29:30   Verbose  |       Performing variable substitution on 'C:\Octopus\Work\20220110192928-113924-27\Script.ps1'
19:29:30   Verbose  |       Executing 'C:\Octopus\Work\20220110192928-113924-27\Script.ps1'
19:29:30   Error    |       System.Exception: AWS-LOGIN-ERROR-0003: Failed to access the role information under http://169.254.169.254/latest/meta-data/iam/security-credentials/, or failed to parse the response. This may be because the instance does not have a role assigned to it. For more information visit [https://g.octopushq.com/AwsCloudFormationDeploy#aws-login-error-0003](https://g.octopushq.com/AwsCloudFormationDeploy#aws-login-error-0003) ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found).
19:29:30   Error    |       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
19:29:30   Error    |       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
19:29:30   Error    |       at Calamari.CloudAccounts.AwsEnvironmentGeneration.<PopulateKeysFromInstanceRole>d__27.MoveNext()
19:29:30   Error    |       --- End of inner exception stack trace ---
19:29:30   Error    |       at Calamari.CloudAccounts.AwsEnvironmentGeneration.<PopulateKeysFromInstanceRole>d__27.MoveNext()
19:29:30   Error    |       --- End of stack trace from previous location where exception was thrown ---
19:29:30   Error    |       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
19:29:30   Error    |       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
19:29:30   Error    |       at Calamari.CloudAccounts.AwsEnvironmentGeneration.<Initialise>d__15.MoveNext()
19:29:30   Error    |       --- End of stack trace from previous location where exception was thrown ---
19:29:30   Error    |       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
19:29:30   Error    |       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
19:29:30   Error    |       at Calamari.CloudAccounts.AwsEnvironmentGeneration.<Create>d__14.MoveNext()
19:29:30   Error    |       --- End of stack trace from previous location where exception was thrown ---
19:29:30   Error    |       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
19:29:30   Error    |       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
19:29:30   Error    |       at Calamari.Aws.Integration.AwsScriptWrapper.ExecuteScript(Script script, ScriptSyntax scriptSyntax, ICommandLineRunner commandLineRunner, Dictionary`2 environmentVars)
19:29:30   Error    |       at Calamari.Common.Features.FunctionScriptContributions.FunctionAppenderScriptWrapper.ExecuteScript(Script script, ScriptSyntax scriptSyntax, ICommandLineRunner commandLineRunner, Dictionary`2 environmentVars)
19:29:30   Error    |       at Calamari.Deployment.Conventions.ExecuteScriptConvention.Install(RunningDeployment deployment)
19:29:30   Error    |       at Calamari.Deployment.ConventionProcessor.RunInstallConventions()
19:29:30   Error    |       at Calamari.Deployment.ConventionProcessor.RunConventions()

Can you offer any guidance?

Reading other tickets the only possibly related issue I could find was this one: Help around AWS CLI Worker Step Template - #2 by ops but I am using the drop down to select the account so I’m not sure its related.

Hi @andrew3,

Thanks for reaching out and I’m sorry to see you’re running into that error.

Could you let us know which version of Octopus you’re currently running? It’s possible that IMDSv2 is being used where versions of Octopus pre-2021.3 didn’t yet support IMDSv2. The following bug report outlines this with the proposed workaround being to “Re-Enable IMDSv1 or configure an AWS Account variable in the Step to use access keys”.

It does look like we have a bit of documentation for the error AWS-LOGIN-ERROR-0003, you can find some additional troubleshooting steps here: Deploy an AWS CloudFormation template - Octopus Deploy

Essentially, performing a GET request on the URL with your expected Rolename at http://169.254.169.254/latest/meta-data/iam/security-credentials/ROLENAME should let you know if the role is in place.

Could you let me know if that helps you get unstuck?

Regards,
Garrett

Hey @garrett.dass

Thanks for the quick reply.

We are running 2021.3 8275.

I have selected “No” for the “Execute using the AWS service role for an EC2 instance”, so I’m not sure the linked story is relevant?

Basically I have an IAM identity with some permissions that I would like to use to make some calls to AWS. I’m not sure where a role comes into play for this?

HI @andrew3,

I apologize we usually see this behavior with a missing role.

Since it’s failing on variable substitution I’m wondering if you would turn on variable logging so you could see exactly what the variable is evaluating as during the deployment.
If the expected value is not there, it could be variable scoping that is causing the issue even though the variable preview might show otherwise.

Please let me know what you find.

Regards,
Garrett

Hi Andrew,

Just stepping in for Garrett from the Australia based team.

I’ve been trying to reproduce this issue but I haven’t been successful so far, I was wondering if you could please provide any additional info about the configuration so that I can try to match it on my end.

In addition to adding the variable logging as per Garrett’s suggestion, could you please also send through the Deployment Process JSON for this project?

That should provide me enough info to get a reproduction going, feel free to reach out if you have any questions!

Best Regards,

Hi @finnian.dempsey and @garrett.dass

Thanks both for taking the time to help on this. I think in the end it was user error…

I think the original problem was that for the account variable I had defined a “target role” but the AWS CLI step does not allow selecting a target role (normally I’m only using the IIS/Windows Service steps where you define a target role). When I was using the variable preview I was selecting a role which is why it was showing up there.

In my attempts to debug I had renamed the account variable a couple of times and as a result of that the account variable had been unselected in the AWS CLI step settings…

The hints for the variable logging and exporting to JSON were both really helpful - it was the export to JSON that let me see that the “Octopus.Action.AwsAccount.Variable” was actually storing one of the previous variable names…

Hi Andrew,

Great to hear you were able to resolve the issue, thanks for letting us know!

Personally, I like to manage account variables by creating a variable set and attaching that to the project, so that any changes I make to the account in future are automatically updated in any projects that use them.

Feel free to reach out again if you ever have any questions or issues again in the future!

Best Regards,