https://pastein.ru/t/p5M
Загрузка данных
Running with gitlab-runner 15.8.0 (12335144)
on gitlab-runner-gitlab-runner-64775b4cff-qpdc8 9cgwHtm5, system ID: r_FsAm41LqwxCr
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runners
Using Kubernetes executor with image registry.rshbdev.ru/appfarm/infra/images/kube-client-apps:8.21.11 ...
Using attach strategy to execute scripts...
Preparing environment
00:06
Waiting for pod gitlab-runners/runner-9cgwhtm5-project-85437-concurrent-0mb8jt to be running, status is Pending
Waiting for pod gitlab-runners/runner-9cgwhtm5-project-85437-concurrent-0mb8jt to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-9cgwhtm5-project-85437-concurrent-0mb8jt via gitlab-runner-gitlab-runner-64775b4cff-qpdc8...
Getting source from Git repository
00:02
$ git config --global --add url."https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/".insteadOf "https://${CI_SERVER_HOST}" # collapsed multi-line command
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/.git/
Created fresh repository.
Checking out d4dc7083 as stress...
Skipping Git submodules setup
Downloading artifacts
00:00
Downloading artifacts for devsecops_antivirus_scan (23305262)...
Downloading artifacts from coordinator... ok id=23305262 responseStatus=200 OK token=64_P-1ig
Executing "step_script" stage of the job script
11:08
$ ( umask 0077; mkdir -p ~/.kube && echo "$KUBECONFIG_COMBINED" | base64 -d > ~/.kube/config )
$ if [[ ${IS_SENSITIVE_SYSTEM} = true ]]; then export K8S_CLUSTERS=$K8S_CLUSTERS_SENSITIVE; fi
$ echo 'IS_SENSITIVE_SYSTEM:' $IS_SENSITIVE_SYSTEM
IS_SENSITIVE_SYSTEM: true
$ echo 'TARGET_CLUSTERS:' $K8S_CLUSTERS
TARGET_CLUSTERS: rcsdstbl
$ set -x
++ echo '$ for CLUSTER in $K8S_CLUSTERS; do'
$ for CLUSTER in $K8S_CLUSTERS; do
++ for CLUSTER in $K8S_CLUSTERS
++ echo '$ export CLUSTER=$CLUSTER'
$ export CLUSTER=$CLUSTER
++ export CLUSTER=rcsdstbl
++ CLUSTER=rcsdstbl
++ echo '$ if [[ ${CI_ENVIRONMENT_SLUG} = production ]]; then export NAMESPACE=isys-${ISYS_NAME}; else export NAMESPACE=isys-${ISYS_NAME}-${CI_ENVIRONMENT_SLUG}; fi'
$ if [[ ${CI_ENVIRONMENT_SLUG} = production ]]; then export NAMESPACE=isys-${ISYS_NAME}; else export NAMESPACE=isys-${ISYS_NAME}-${CI_ENVIRONMENT_SLUG}; fi
++ [[ stress = production ]]
++ export NAMESPACE=isys-ownfin-stress
++ NAMESPACE=isys-ownfin-stress
++ echo '$ read -r REVISION STATUS < <(helm --kube-context ${CLUSTER:-default} -n $NAMESPACE history $CI_PROJECT_NAME | tail -1 | cut -f1,3) || true'
$ read -r REVISION STATUS < <(helm --kube-context ${CLUSTER:-default} -n $NAMESPACE history $CI_PROJECT_NAME | tail -1 | cut -f1,3) || true
++ read -r REVISION STATUS
+++ helm --kube-context rcsdstbl -n isys-ownfin-stress history pc-server
+++ tail -1
+++ cut -f1,3
++ echo '$ if [[ "$STATUS" =~ "pending" ]]; then helm --kube-context ${CLUSTER:-default} -n $NAMESPACE rollback $CI_PROJECT_NAME $REVISION || true; fi'
$ if [[ "$STATUS" =~ "pending" ]]; then helm --kube-context ${CLUSTER:-default} -n $NAMESPACE rollback $CI_PROJECT_NAME $REVISION || true; fi
++ [[ deployed =~ pending ]]
++ echo '$ helmfile ${HELMFILE_DEFAULT_NAMESPACE:+--namespace $HELMFILE_DEFAULT_NAMESPACE} --environment ${CLUSTER:-default} -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set "additionalWorkloadAnnotations.gitlabPipeline=${CI_PIPELINE_URL}" --set "additionalWorkloadAnnotations.gitlabCommit=${CI_COMMIT_SHA}"'
$ helmfile ${HELMFILE_DEFAULT_NAMESPACE:+--namespace $HELMFILE_DEFAULT_NAMESPACE} --environment ${CLUSTER:-default} -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set "additionalWorkloadAnnotations.gitlabPipeline=${CI_PIPELINE_URL}" --set "additionalWorkloadAnnotations.gitlabCommit=${CI_COMMIT_SHA}"
++ helmfile --environment rcsdstbl -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set additionalWorkloadAnnotations.gitlabPipeline=https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3343980 --set additionalWorkloadAnnotations.gitlabCommit=d4dc7083ff8f58fb21eb683c0d4a82dbf6db2344
== /usr/local/link/helmfile: Initialize base deploy hierarchy in /builds/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
no matches for path: envs/stress/rcsdstbl/helmfile.yaml.gotmpl
Adding repo rshb-charts https://nexus.rshbdev.ru/repository/charts/
"rshb-charts" has been added to your repositories
Comparing release=vault-secrets-pc-server, chart=rshb-charts/raw
Comparing release=psvc-pc-server, chart=rshb-charts/raw
Comparing release=platform-database-pc-server, chart=rshb-charts/raw
Comparing release=links-pc-server, chart=rshb-charts/raw
Comparing release=pc-server-rumsk1, chart=rshb-charts/base
Comparing release=pc-server-rumsk2, chart=rshb-charts/base
********************
Release was not present in Helm. Diff will show entire contents as new.
********************
isys-ownfin-stress, pc-server-rumsk2, Deployment (apps) has been added:
-
+ # Source: base/templates/workload.yaml
+ apiVersion: apps/v1
+ kind: Deployment
+ metadata:
+ name: pc-server-rumsk2
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk2"
+ annotations:
+ gitlabCommit: d4dc7083ff8f58fb21eb683c0d4a82dbf6db2344
+ gitlabPipeline: https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3343980
+ platform.ckpr.integrations.rshbintech.ru/gitlab-commit-sha: d4dc7083
+ platform.ckpr.integrations.rshbintech.ru/gitlab-pipeline-id: "3343980"
+ spec:
+ strategy:
+ type: Recreate
+ replicas: 2
+ selector:
+ matchLabels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ release: pc-server-rumsk2
+ template:
+ metadata:
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk2"
+ annotations:
+ checksum/configMapsEnv: "7d5d794ca3ae5b994e7d8c607783569fcaea43fe786cec3f614ba60ccdd6af3f"
+ checksum/secretEnv: "e0673ccba2b800dc3cf5e6f3a0f19f5d0e19cc221b7e5e0d9b3a391bc713ed8c"
+ ci/commithash: "d4dc7083"
+ inject.istio.io/templates: "sidecar,custom"
+ prometheus.io/path: "/metrics"
+ prometheus.io/port: "8080"
+ prometheus.io/scrape: "true"
+ sidecar.istio.io/userVolume: "[{\"name\": \"wasmfilters-dir\",\"configMap\": {\"name\": \"ownfin-stress-envoy-filters\"}}]"
+ sidecar.istio.io/userVolumeMount: "[{\"mountPath\":\"/var/local/lib/wasm-filters\",\"name\":\"wasmfilters-dir\"}]"
+ spec:
+ automountServiceAccountToken: false
+ serviceAccountName: default
+ nodeSelector:
+ node-role.kubernetes.io/wload: ""
+ workload.topology.app.farm/zone: rumsk2
+ priorityClassName: rumsk2
+ containers:
+ - name: app
+ image: registry.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server:feature-task-140-rshb.stress
+ imagePullPolicy: Always
+ envFrom:
+ - configMapRef:
+ name: pc-server-rumsk2-app-cm-env
+ - secretRef:
+ name: pc-server-rumsk2-app-secret-env
+ resources:
+ limits:
+ cpu: 2000m
+ memory: 1024Mi
+ requests:
+ cpu: 1000m
+ memory: 512Mi
+ ports:
+ livenessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /api/health_check
+ port: 8080
+ initialDelaySeconds: 60
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 5
+ readinessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /api/health_check
+ port: 8080
+ initialDelaySeconds: 60
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 5
+ volumeMounts:
+ - name: app-vault-secrets
+ mountPath: /secrets
+ readOnly: true
+ securityContext:
+ allowPrivilegeEscalation: false
+ capabilities:
+ drop:
+ - ALL
+ privileged: false
+ procMount: Default
+ readOnlyRootFilesystem: false
+ runAsNonRoot: true
+ securityContext:
+ runAsNonRoot: true
+ tolerations:
+ - effect: NoSchedule
+ key: node-role.kubernetes.io/wload
+ operator: Exists
+ - effect: NoSchedule
+ key: workload.topology.app.farm/zone
+ operator: Exists
+ hostNetwork: false
+ volumes:
+ - name: app-vault-secrets
+ csi:
+ driver: secrets-store.csi.k8s.io
+ readOnly: true
+ volumeAttributes:
+ secretProviderClass: pc-server-rumsk2-app-secret-provider
isys-ownfin-stress, pc-server-rumsk2, Service (v1) has been added:
-
+ # Source: base/templates/svc.yaml
+ apiVersion: v1
+ kind: Service
+ metadata:
+ name: pc-server-rumsk2
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ spec:
+ ports:
+ - name: web
+ port: 80
+ protocol: TCP
+ targetPort: 8080
+ selector:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ release: pc-server-rumsk2
+ type: ClusterIP
isys-ownfin-stress, pc-server-rumsk2-app-cm-env, ConfigMap (v1) has been added:
-
+ # Source: base/templates/cm-env.yaml
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: pc-server-rumsk2-app-cm-env
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk2"
+ data:
+ APP_POSTGRESQL_HOST: "10.21.25.43"
+ APP_POSTGRESQL_NAME: "paycontrol6"
+ APP_POSTGRESQL_PORT: "5432"
+ JAVA_OPTS: "-XX:+PrintFlagsFinal -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0 -XX:+UseTransparentHugePages"
+ SERVER_PATHREWRITES_0_FROM: "/metrics"
+ SERVER_PATHREWRITES_0_TO: "/api/actuator/prometheus"
+ SERVER_SERVLET_CONTEXTPATH: "/api"
+ SPRING_DATASOURCE_URL: "jdbc:postgresql://10.21.25.43:5432/paycontrol6"
isys-ownfin-stress, pc-server-rumsk2-app-secret-env, Secret (v1) has been added:
+ Changes suppressed on sensitive content of type Secret
isys-ownfin-stress, pc-server-rumsk2-app-secret-provider, SecretProviderClass (secrets-store.csi.x-k8s.io) has been added:
-
+ # Source: base/templates/secret-provider.yaml
+ apiVersion: secrets-store.csi.x-k8s.io/v1
+ kind: SecretProviderClass
+ metadata:
+ name: pc-server-rumsk2-app-secret-provider
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk2"
+ spec:
+ provider: vault
+ parameters:
+ vaultKubernetesMountPath: rcsdstbl
+ roleName: "csi-secrets-reader-stress"
+ objects: |
+ - objectName: APP_POSTGRESQL_MIGRATION_PASSWORD
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: APP_POSTGRESQL_MIGRATION_PASSWORD
+ - objectName: APP_POSTGRESQL_MIGRATION_USER
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: APP_POSTGRESQL_MIGRATION_USER
+ - objectName: SPRING_DATASOURCE_PASSWORD
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: SPRING_DATASOURCE_PASSWORD
+ - objectName: SPRING_DATASOURCE_USERNAME
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: SPRING_DATASOURCE_USERNAME
+ secretObjects:
+ - secretName: pc-server-rumsk2-app-secret-env
+ type: Opaque
+ labels:
+ app: pc-server
+ fullname: pc-server-rumsk2
+ chart: base-1.15.2
+ release: pc-server-rumsk2
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk2"
+ data:
+ - objectName: APP_POSTGRESQL_MIGRATION_PASSWORD
+ key: APP_POSTGRESQL_MIGRATION_PASSWORD
+ - objectName: APP_POSTGRESQL_MIGRATION_USER
+ key: APP_POSTGRESQL_MIGRATION_USER
+ - objectName: SPRING_DATASOURCE_PASSWORD
+ key: SPRING_DATASOURCE_PASSWORD
+ - objectName: SPRING_DATASOURCE_USERNAME
+ key: SPRING_DATASOURCE_USERNAME
Comparing release=pc-server, chart=rshb-charts/raw
Listing releases matching ^pjob-pc-server$
Listing releases matching ^exsvc-pc-server$
Listing releases matching ^data-pc-server$
Listing releases matching ^assets-config-pc-server$
Listing releases matching ^pc-server-raw$
Listing releases matching ^pc-server-grafana-dashboard$
Listing releases matching ^pc-server-prometheus-rules$
Upgrading release=pc-server-rumsk2, chart=rshb-charts/base
Release "pc-server-rumsk2" does not exist. Installing it now.
FAILED RELEASES:
NAME
pc-server-rumsk2
in deploy/helmfile.yaml: failed processing release pc-server-rumsk2: command "/usr/local/link/helm" exited with non-zero status:
PATH:
/usr/local/link/helm
ARGS:
0: helm (4 bytes)
1: --kube-context (14 bytes)
2: rcsdstbl (8 bytes)
3: upgrade (7 bytes)
4: --install (9 bytes)
5: --reset-values (14 bytes)
6: pc-server-rumsk2 (16 bytes)
7: rshb-charts/base (16 bytes)
8: --version (9 bytes)
9: 1.15.2 (6 bytes)
10: --wait (6 bytes)
11: --timeout (9 bytes)
12: 600s (4 bytes)
13: --atomic (8 bytes)
14: --kube-context (14 bytes)
15: rcsdstbl (8 bytes)
16: --namespace (11 bytes)
17: isys-ownfin-stress (18 bytes)
18: --values (8 bytes)
19: /tmp/helmfile3999734456/isys-ownfin-stress-pc-server-rumsk2-values-865dfcc78 (76 bytes)
20: --values (8 bytes)
21: /tmp/helmfile82454479/isys-ownfin-stress-pc-server-rumsk2-values-647bd47596 (75 bytes)
22: --values (8 bytes)
23: /tmp/helmfile1380027845/isys-ownfin-stress-pc-server-rumsk2-values-6bb8b94458 (77 bytes)
24: --set (5 bytes)
25: additionalWorkloadAnnotations.gitlabPipeline=https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3343980 (148 bytes)
26: --set (5 bytes)
27: additionalWorkloadAnnotations.gitlabCommit=d4dc7083ff8f58fb21eb683c0d4a82dbf6db2344 (83 bytes)
28: --history-max (13 bytes)
29: 10 (2 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: release pc-server-rumsk2 failed, and has been uninstalled due to atomic being set: timed out waiting for the condition
COMBINED OUTPUT:
Release "pc-server-rumsk2" does not exist. Installing it now.
Error: release pc-server-rumsk2 failed, and has been uninstalled due to atomic being set: timed out waiting for the condition
Uploading artifacts for failed job
00:01
Uploading artifacts...
deploy.env: found 1 matching artifact files and directories
Uploading artifacts as "dotenv" to coordinator... 201 Created id=23306300 responseStatus=201 Created token=64_P-1ig
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1