k8s 기반 GitLab CI 실행 시 간헐적으로 Docker in Docker job 이 실패하는 문제
버그
(선택) 에픽
(필수) 요약
- 현재 내부에서 진행 중인 프로젝트들의 CI/CD 파이프라인을 GitLab CI 를 통해 실행하는데, docker in docker job 에서 간헐적으로
unable to resolve docker endpoint: open /certs/client/ca.pem: no such file or directory
에러가 발생하면서 파이프라인이 실패됨 (여러 번 다시 시도하면 성공함)
(필수) 재현 단계
- Docker In Docker 기능을 사용하는 job 실행 (이미지 빌드 등)
- 랜덤한 확률로
unable to resolve docker endpoint: open /certs/client/ca.pem: no such file or directory
에러가 발생하며 해당 job 이 실패함
(필수) 예상되는 올바른 동작
- job 실행 중 발생하는 문제가 외에는 job 자체가 시작하자마자 실패하면 안 됨
(필수) 버그로 추측되는 현재 동작
- job 실패
(선택) 관련 로그 및 스크린샷
Bell Safe House View - 7623 Job
Running with gitlab-runner 15.8.0 (12335144)
on instance-primary-gitlab-runner-5d57bdc99d-xb5xw UDX5yyuX, system ID: r_LU9Z7wD0015l
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: bsn-gitlab-runner
Using Kubernetes executor with image docker ...
Using attach strategy to execute scripts...
Preparing environment
00:28
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Waiting for pod bsn-gitlab-runner/runner-udx5yyux-project-62-concurrent-09d5px to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper svc-0]"
ContainersNotReady: "containers with unready status: [build helper svc-0]"
Running on runner-udx5yyux-project-62-concurrent-09d5px via instance-primary-gitlab-runner-5d57bdc99d-xb5xw...
Getting source from Git repository
00:03
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/connected-life/bell-safe-house-view/.git/
Created fresh repository.
Checking out 11da55fd as refs/merge-requests/15/head...
Skipping Git submodules setup
Downloading artifacts
00:01
Downloading artifacts for generate_nuxt_app (7618)...
Downloading artifacts from coordinator... ok id=7618 responseStatus=200 OK token=64_vNXkE
Executing "step_script" stage of the job script
00:01
$ docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} ${CI_REGISTRY}
unable to resolve docker endpoint: open /certs/client/ca.pem: no such file or directory
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1
(선택) 환경 정보
Edited by Bell