Web API: 500 Internal Server Error on PUT to variables

resolved
server
usability
support
known
(Helgi Hafþórsson) #1

I am trying to update a variableset. I start by creating a new project by cloning a “template” project. That project already has variables. Then I fetch the variables of the new project, modify them as needed and then I PUT them back. Then I get a 500 Internal Server Error. The body of the request is attached to this message. body.json (5.2 KB)

The diagnostic logs say this:
Unhandled error on request: http://vbdeployer/api/variables/variableset-Projects-68 29d519a643c4476798ca8e1228824c7b by : Object reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object.
at Octopus.Server.Web.Api.Actions.VariableSetUpdateAction.<>c__DisplayClass22_0.b__2(ReferenceDataItem e1) in C:\buildAgent\work\abb2fbfce959a439\source\Octopus.Server\Web\Api\Actions\VariableSetUpdateAction.cs:line 282
at System.Linq.Enumerable.Any[TSource](IEnumerable1 source, Func2 predicate)
at Octopus.Server.Web.Api.Actions.VariableSetUpdateAction.ExecuteRegistered(String id) in C:\buildAgent\work\abb2fbfce959a439\source\Octopus.Server\Web\Api\Actions\VariableSetUpdateAction.cs:line 71
at Octopus.Server.Web.Infrastructure.Api.CustomActionWithIdResponder1.ExecuteRegistered() in C:\buildAgent\work\abb2fbfce959a439\source\Octopus.Server\Web\Infrastructure\Api\CustomActionResponder.cs:line 46 at Octopus.Server.Web.Infrastructure.Api.CustomResponder1.Respond(TDescriptor options, NancyContext context) in C:\buildAgent\work\abb2fbfce959a439\source\Octopus.Server\Web\Infrastructure\Api\CustomResponder.cs:line 296
at Octopus.Server.Web.Infrastructure.OctopusNancyModule.<>c__DisplayClass14_0.<get_Routes>b__1(Object o, CancellationToken x) in C:\buildAgent\work\abb2fbfce959a439\source\Octopus.Server\Web\Infrastructure\OctopusNancyModule.cs:line 125
at Nancy.Routing.Route`1.d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Nancy.Routing.DefaultRouteInvoker.d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Nancy.Routing.DefaultRequestDispatcher.d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Nancy.NancyEngine.d__22.MoveNext()

(Helgi Hafþórsson) #2

Update: If I remove the scope from variable CreateSchedule so it only occurs once in the json, this error does not occur. This however does not fix my problem, since I need to have that variable scoped.

(Jeremy Miller) #4

Hi Helgi,

Thanks for reaching out. Would you be able to share a before JSON of the variables along with the script you’re running to get/modify/put the json? I have a similar script that I’ve written and my JSON looks a bit different before uploading back to the server, but I may be going about it in a different manner so it might be helpful to see your method.

For reference, this is my script I am working on that does something similar. I’m still working on adding features to it and doing testing but it works for modifying a value and adding variables in my testing. If you do use it, please do ample testing in a test environment. Let me know if you can provide those files or if my script helps.

Thanks,
Jeremy

Function Get-OctopusProject

{

    # Define parameters

    param(

        $OctopusServerUrl,

        $ApiKey,

        $ProjectName

    )

    # Call API to get all projects, then filter on name

    $octopusProject = Invoke-RestMethod -Method "get" -Uri "$OctopusServerUrl/api/projects/all" -Headers @{"X-Octopus-ApiKey"="$ApiKey"}

    # return the specific project

    return ($octopusProject | Where-Object {$_.Name -eq $ProjectName})

}

Function Get-OctopusProjectVariables

{

    # Define parameters

    param(

        $OctopusDeployProject,

        $OctopusServerUrl,

        $ApiKey

    )

    # Get reference to the variable list

    return (Invoke-RestMethod -Method "get" -Uri "$OctopusServerUrl/api/variables/$($OctopusDeployProject.VariableSetId)" -Headers @{"X-Octopus-ApiKey"="$ApiKey"})

}

#Code to go find the spaceId

Function Get-SpaceId{

    # Define parameters

    param(

        $Space

    )

    $spaceName = $Space

    $spaceList = Invoke-RestMethod "$OctopusServerUrl/api/spaces?Name=$spaceName" -Headers @{"X-Octopus-ApiKey"=$ApiKey}

    $spaceFilter = @($spaceList.Items | Where {$_.Name -eq $spaceName})

    $spaceId = $spaceFilter[0].Id

    return $spaceId

}

Function Get-EnvironmentId{

    # Define parameters

    param(

        $EnvironmentName,

        $Space

    )

    $environmentName = $EnvironmentName

    $environmentList = Invoke-RestMethod "$OctopusServerUrl/api/$spaceId/environments?skip=0&take=1000&name=$environmentName" -Headers @{"X-Octopus-ApiKey"=$ApiKey}

    $environmentFilter = @($environmentList.Items | Where {$_.Name -eq $environmentName})

    $environmentId = $environmentFilter[0].Id

    return $environmentId

}

Function Modify-Variable{

    # Define parameters

    param(

        $VariableSet,

        $VariableName,

        $VariableValue,

        $VariableEnvScope,

        $Space

    )

    #If given a scope parameter, find the matching variable with scope and modify the value

    if ($VariableEnvScope){

    $spaceId = Get-SpaceId -Space $Space

    write-host "has scope"

    #Code to transform the environment name to environment ID.

    $environmentId = Get-EnvironmentId -EnvironmentName $VariableEnvScope -Space $spaceId

    #loop through all variables and change the value if the name and environment ID match

    ForEach ($variable in $VariableSet.Variables){

    if ($variable.Name -eq $VariableName -and $variable.Scope.Environment -eq $environmentId){

    $variable.Value = $VariableValue

    }

    }

    }

    #When a scope parameter is not given

    else{

    #Find the variable you want to edit by name, then edit the value. Only edit if the variable is unscoped.

    ForEach ($variable in $VariableSet.Variables){

    if ($variable.Name -eq $VariableName -and !$variable.Scope.Environment){

    $variable.Value = $VariableValue

    }

    }

    }

}

Function Add-Variable{

    # Define parameters

    param(

        $VariableSet,

        $VariableName,

        $VariableValue,

        $VariableEnvScope,

        $VariableRoleScope

    )

    #Find the environment ID based on the name given by the parameter.

    $environmentObj = $VariableSet.ScopeValues.Environments | Where { $_.Name -eq $VariableEnvScope } | Select -First 1

        #If there is no Env or Role scope, add variable this way

        if (!$VariableEnvScope -and !$VariableRoleScope){

            $tempVariable = @{

                Name = $VariableName

                Value = $VariableValue

                Scope = @{

                }

            }

        }

        #If there is an Env but no Role scope, add variable this way

        if ($VariableEnvScope -and !$VariableRoleScope){

            $tempVariable = @{

                Name = $VariableName

                Value = $VariableValue

                Scope = @{ 

                    Environment = @(

                        $environmentObj.Id

                    )

                }

            }

        }

        #If there is a Role Scope but no Env scope, add the variable this way

        if ($VariableRoleScope -and !$VariableEnvScope){

            $tempVariable = @{

                Name = $VariableName

                Value = $VariableValue

                Scope = @{ 

                    Role = @(

                        $VariableRoleScope

                    )

                }

            }

        }

        #If both scopes exis, add the variable this way

        if ($VariableEnvScope -and $VariableRoleScope){

            $tempVariable = @{

                Name = $VariableName

                Value = $VariableValue

                Scope = @{

                    Environment = @(

                        $environmentObj.Id

                    )

                    Role = @(

                        $VariableRoleScope

                    )

                }

            }

        }

    #add the variable to the variable set

    $VariableSet.Variables += $tempVariable

}

### INPUT THESE VALUES ####

$OctopusServerUrl = ""  #PUT YOUR SERVER LOCATION HERE. (e.g. http://localhost)

$ApiKey = ""   #PUT YOUR API KEY HERE

$ProjectName = ""   #PUT THE NAME OF THE PROJECT THAT HOUSES THE VARIABLES HERE

### INPUT THESE VALUES ####

try

{

    # Get reference to project

    $octopusProject = Get-OctopusProject -OctopusServerUrl $OctopusServerUrl -ApiKey $ApiKey -ProjectName $ProjectName

    # Get list of existing variables

    $octopusProjectVariables = Get-OctopusProjectVariables -OctopusDeployProject $octopusProject -OctopusServerUrl $OctopusServerUrl -ApiKey $ApiKey

    #Examples

    

    #If you want to modify an Environmentally scoped variable, you must pass the Environment with -VariableEnvScope and the Space with -Space

    #Modify-Variable -VariableSet $octopusProjectVariables -VariableName "Test" -VariableValue "New"

    #Modify-Variable -VariableSet $octopusProjectVariables -VariableName "Test2" -VariableValue "New2" -VariableEnvScope "Development" -Space "Default"

    #Add-Variable -VariableSet $octopusProjectVariables -VariableName "TestNew1" -VariableValue "Nothing to the right"

    #Add-Variable -VariableSet $octopusProjectVariables -VariableName "TestNewEnv" -VariableValue "Env to the right" -VariableEnvScope "Development"

    #Add-Variable -VariableSet $octopusProjectVariables -VariableName "TestNewRole" -VariableValue "Role to the right" -VariableRoleScope "Web"

    #Add-Variable -VariableSet $octopusProjectVariables -VariableName "TestNewEnvRole" -VariableValue "Both to the right" -VariableEnvScope "Development" -VariableRoleScope "Web"

    Add-Variable -VariableSet $octopusProjectVariables -VariableName "CreateSchedule" -VariableValue "False" -VariableEnvScope "Development"

    Add-Variable -VariableSet $octopusProjectVariables -VariableName "CreateSchedule" -VariableValue "True" -VariableEnvScope "Test"

    

    ##### PUT ANY MODIFY AND ADD COMMANDS HERE #####

    

    ##### PUT ANY MODIFY AND ADD COMMANDS HERE #####

    # Convert object to json to upload it

    $jsonBody = $octopusProjectVariables | ConvertTo-Json -Depth 10

    Write-Host $jsonBody

    # Save the variables to the variable set

    Invoke-RestMethod -Method "put" -Uri "$OctopusServerUrl/api/variables/$($octopusProjectVariables.Id)" -Body $jsonBody -Headers @{"X-Octopus-ApiKey"=$ApiKey}

    

}

catch

{

    Write-Error $_.Exception.Message

    throw

}
(Helgi Hafþórsson) #5

Hi Jeremy,

Attached are the json before I modify it and an excerpt of how I get/modify/put the json.variables.json (7.5 KB)
myPowershell.ps1 (3.0 KB)

(Jeremy Miller) #6

Hi Helgi,

I did a quick compare of the 2 jsons and the after-JSON is missing quite a bit of information. When you are putting the JSON back you are only including these 2 sections.

$body = @{

            "Version"   = $variableSet.Version

            "Variables" = $variableSet.Variables

        }

You are using a PUT command here $response = Invoke-RestMethod -Uri $u -Method Put -Headers $headers -Body $json -ContentType 'application/json' to put the modified resource back on your server.

PUT will wipe out the entire resource and replace it with whatever is in the body of the call, so instead of just updating the resources you are intending to update, you are making it so the JSON at that location only has those values and nothing else. This may be the root cause of the issue, but you will have to test and let me know. Can you include everything else in the body that is in the before-JSON and retry and let me know if that allows you to keep your scoping when using your script? For what it’s worth, you shouldn’t need to recreate the body, you should just need to use the body (converted to JSON) that you were manipulating in all the other steps in your PUT call. As always, please do testing before using the scripts we suggest in prod.

So in your code instead of this…

    if ($changed) {

            Write-Host "Updating project variables for $($project.Name)"

            $body = @{

                "Version"   = $variableSet.Version

                "Variables" = $variableSet.Variables

            }

            Invoke-OctoPut -url "variables/$($project.VariableSetId)" -body $body

        }

You should be able to change it to this… (feed it the data you’ve been modifying all along)

if ($changed) {

            Write-Host "Updating project variables for $($project.Name)"

            Invoke-OctoPut -url "variables/$($project.VariableSetId)" -body $variableSet

        }

Thanks,
Jeremy

(Helgi Hafþórsson) #7

This worked! I used the whole body that I got originally from the server as you suggested and now I’m all set. Thank you so much!

1 Like
(Jeremy Miller) #8

Awesome! You’re very welcome, I’m glad we got it figured out. Thanks for reaching back out to update me. Have a great rest of your week.