Aws ecr integration problem

program-management
(Itay Shoshan) #1

i have a CI process which build a docker image and push it to ecr repository in aws.
after that it creates octopus release with the latest image.
Lately i am having some problem with that flow. sometimes after i push an image to ecr and octopus can’t find that image.
This problem makes my docker CI/CD process unstable. sometime it works and sometimes not.
I think maybe to problem is with repos with a lot of images. Maybe octopus can’t find the image if its not in the first page or something…
it is a high severity problem because its stuck our CI/CD process…

(Robert) #3

Hi,
I’m noticing what seems to be an odd behavior with ECR when pushing lots of packages.
I have set up a little script to tag and push a small image into an ECR repository:

#!/bin/bash
for i in {3..1000}
do
   docker tag registry.octopushq.com/octopus-echo:1.1 968802670493.dkr.ecr.ap-southeast-2.amazonaws.com/octopus-echo:1.$i
   docker push 968802670493.dkr.ecr.ap-southeast-2.amazonaws.com/octopus-echo:1.$i --verbose
   docker rmi 968802670493.dkr.ecr.ap-southeast-2.amazonaws.com/octopus-echo:1.$i
done

Looking at the raw list in the portal (sorted by date) it looks as though it sometimes seems to “missplace” some of these images.

Even though those versions looked to push perfectly fine from this looping script

Do you notice if any of the images which Octopus seems to not detect are missing from this list?
Also when you say “Octopus cant find that image” I’m assuming this is during release creation time? If you ignore the warnings and just use that package anyway, does the deployment succeed?

Cheers
Rob

(Itay Shoshan) #4

No, I can find the images on the ECR registry so i don’t think this is the case…
After i got the problem with the repo with a lot of images, i create a new repo with a new image name
and the problem aren’t reproduced… but its not a solution because i am building a lot of images…
i don’t want to change the image name after sometime like i’ve dont…
and about your question, i can’t find the package in octopus, so the answer no

thanks

(Robert) #5

Could you send through some screenshots of the package showing up in ECS and not in Octopus. Also to clarify, if you just force the package to be selected does the deployment still work?
Cheers,
Rob

(Robert) #6

How many images are we talking about here? Ive tested with around 1000 and the AWS portal seemed to be missing the same packages that Octopus was missing.