https://pastein.ru/t/VNB
Загрузка данных
Running with gitlab-runner 15.8.0 (12335144)
on gitlab-runner-gitlab-runner-5c5d8dfd84-mrhn4 3Sduy7zd, system ID: r_pxlbVZdKVDlV
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runners
Using Kubernetes executor with image registry.rshbdev.ru/appfarm/infra/images/kube-client-apps:8.21.8 ...
Using attach strategy to execute scripts...
Preparing environment
00:06
Waiting for pod gitlab-runners/runner-3sduy7zd-project-85437-concurrent-0l74xs to be running, status is Pending
Waiting for pod gitlab-runners/runner-3sduy7zd-project-85437-concurrent-0l74xs to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-3sduy7zd-project-85437-concurrent-0l74xs via gitlab-runner-gitlab-runner-5c5d8dfd84-mrhn4...
Getting source from Git repository
00:02
$ git config --global --add url."https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/".insteadOf "https://${CI_SERVER_HOST}" # collapsed multi-line command
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/.git/
Created fresh repository.
Checking out e7166e9a as stress...
Skipping Git submodules setup
Downloading artifacts
00:01
Downloading artifacts for devsecops_antivirus_scan (22773856)...
Downloading artifacts from coordinator... ok id=22773856 responseStatus=200 OK token=64_uzi71
Executing "step_script" stage of the job script
11:21
$ ( umask 0077; mkdir -p ~/.kube && echo "$KUBECONFIG_COMBINED" | base64 -d > ~/.kube/config )
$ if [[ ${IS_SENSITIVE_SYSTEM} = true ]]; then export K8S_CLUSTERS=$K8S_CLUSTERS_SENSITIVE; fi
$ echo 'IS_SENSITIVE_SYSTEM:' $IS_SENSITIVE_SYSTEM
IS_SENSITIVE_SYSTEM: true
$ echo 'TARGET_CLUSTERS:' $K8S_CLUSTERS
TARGET_CLUSTERS: rcsdstbl
$ set -x
++ echo '$ for CLUSTER in $K8S_CLUSTERS; do'
$ for CLUSTER in $K8S_CLUSTERS; do
++ for CLUSTER in $K8S_CLUSTERS
++ echo '$ export CLUSTER=$CLUSTER'
$ export CLUSTER=$CLUSTER
++ export CLUSTER=rcsdstbl
++ CLUSTER=rcsdstbl
++ echo '$ if [[ ${CI_ENVIRONMENT_SLUG} = production ]]; then export NAMESPACE=isys-${ISYS_NAME}; else export NAMESPACE=isys-${ISYS_NAME}-${CI_ENVIRONMENT_SLUG}; fi'
$ if [[ ${CI_ENVIRONMENT_SLUG} = production ]]; then export NAMESPACE=isys-${ISYS_NAME}; else export NAMESPACE=isys-${ISYS_NAME}-${CI_ENVIRONMENT_SLUG}; fi
++ [[ stress = production ]]
++ export NAMESPACE=isys-ownfin-stress
++ NAMESPACE=isys-ownfin-stress
++ echo '$ read -r REVISION STATUS < <(helm --kube-context ${CLUSTER:-default} -n $NAMESPACE history $CI_PROJECT_NAME | tail -1 | cut -f1,3) || true'
$ read -r REVISION STATUS < <(helm --kube-context ${CLUSTER:-default} -n $NAMESPACE history $CI_PROJECT_NAME | tail -1 | cut -f1,3) || true
++ read -r REVISION STATUS
+++ helm --kube-context rcsdstbl -n isys-ownfin-stress history pc-server
+++ tail -1
+++ cut -f1,3
Error: release: not found
++ true
++ echo '$ if [[ "$STATUS" =~ "pending" ]]; then helm --kube-context ${CLUSTER:-default} -n $NAMESPACE rollback $CI_PROJECT_NAME $REVISION || true; fi'
$ if [[ "$STATUS" =~ "pending" ]]; then helm --kube-context ${CLUSTER:-default} -n $NAMESPACE rollback $CI_PROJECT_NAME $REVISION || true; fi
++ [[ '' =~ pending ]]
++ echo '$ helmfile ${HELMFILE_DEFAULT_NAMESPACE:+--namespace $HELMFILE_DEFAULT_NAMESPACE} --environment ${CLUSTER:-default} -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set "additionalWorkloadAnnotations.gitlabPipeline=${CI_PIPELINE_URL}" --set "additionalWorkloadAnnotations.gitlabCommit=${CI_COMMIT_SHA}"'
$ helmfile ${HELMFILE_DEFAULT_NAMESPACE:+--namespace $HELMFILE_DEFAULT_NAMESPACE} --environment ${CLUSTER:-default} -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set "additionalWorkloadAnnotations.gitlabPipeline=${CI_PIPELINE_URL}" --set "additionalWorkloadAnnotations.gitlabCommit=${CI_COMMIT_SHA}"
++ helmfile --environment rcsdstbl -f deploy/helmfile.yaml --log-level info apply --suppress-secrets --set additionalWorkloadAnnotations.gitlabPipeline=https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3267018 --set additionalWorkloadAnnotations.gitlabCommit=e7166e9a19fb964f081fb66543736eb73c0a0123
== /usr/local/link/helmfile: Initialize base deploy hierarchy in /builds/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
no matches for path: envs/stress/rcsdstbl/helmfile.yaml.gotmpl
Adding repo rshb-charts https://nexus.rshbdev.ru/repository/charts/
"rshb-charts" has been added to your repositories
Comparing release=vault-secrets-pc-server, chart=rshb-charts/raw
Comparing release=psvc-pc-server, chart=rshb-charts/raw
Comparing release=platform-database-pc-server, chart=rshb-charts/raw
Comparing release=links-pc-server, chart=rshb-charts/raw
Comparing release=pc-server, chart=rshb-charts/base
********************
Release was not present in Helm. Diff will show entire contents as new.
********************
isys-ownfin-stress, pc-server, Deployment (apps) has been added:
-
+ # Source: base/templates/workload.yaml
+ apiVersion: apps/v1
+ kind: Deployment
+ metadata:
+ name: pc-server
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk1"
+ annotations:
+ gitlabCommit: e7166e9a19fb964f081fb66543736eb73c0a0123
+ gitlabPipeline: https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3267018
+ platform.ckpr.integrations.rshbintech.ru/gitlab-commit-sha: e7166e9a
+ platform.ckpr.integrations.rshbintech.ru/gitlab-pipeline-id: "3267018"
+ spec:
+ strategy:
+ type: Recreate
+ replicas: 1
+ selector:
+ matchLabels:
+ app: pc-server
+ fullname: pc-server
+ release: pc-server
+ template:
+ metadata:
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk1"
+ annotations:
+ checksum/configMapsEnv: "003e8c7f3a28946bd13ab9a355ccc9c982ef96ead2d13e867b6d20ae2dc4d981"
+ checksum/secretEnv: "b1c4927edba0ec257471206a21ca281fedb899856fa47b2a2da442b89b61b8e3"
+ ci/commithash: "e7166e9a"
+ inject.istio.io/templates: "sidecar,custom"
+ prometheus.io/path: "/metrics"
+ prometheus.io/port: "8080"
+ prometheus.io/scrape: "true"
+ sidecar.istio.io/userVolume: "[{\"name\": \"wasmfilters-dir\",\"configMap\": {\"name\": \"ownfin-stress-envoy-filters\"}}]"
+ sidecar.istio.io/userVolumeMount: "[{\"mountPath\":\"/var/local/lib/wasm-filters\",\"name\":\"wasmfilters-dir\"}]"
+ spec:
+ automountServiceAccountToken: false
+ serviceAccountName: default
+ nodeSelector:
+ node-role.kubernetes.io/wload: ""
+ workload.topology.app.farm/zone: rumsk1
+ containers:
+ - name: app
+ image: registry.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server:feature-task-140-rshb.stress
+ imagePullPolicy: Always
+ envFrom:
+ - configMapRef:
+ name: pc-server-app-cm-env
+ - secretRef:
+ name: pc-server-app-secret-env
+ resources:
+ limits:
+ cpu: 300m
+ memory: 512Mi
+ requests:
+ cpu: 100m
+ memory: 512Mi
+ ports:
+ livenessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /api/health_check
+ port: 8080
+ initialDelaySeconds: 60
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 5
+ readinessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /api/health_check
+ port: 8080
+ initialDelaySeconds: 60
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 5
+ volumeMounts:
+ - name: app-vault-secrets
+ mountPath: /secrets
+ readOnly: true
+ securityContext:
+ allowPrivilegeEscalation: false
+ capabilities:
+ drop:
+ - ALL
+ privileged: false
+ procMount: Default
+ readOnlyRootFilesystem: false
+ runAsNonRoot: true
+ securityContext:
+ runAsNonRoot: true
+ tolerations:
+ - effect: NoSchedule
+ key: node-role.kubernetes.io/wload
+ operator: Exists
+ - effect: NoSchedule
+ key: workload.topology.app.farm/zone
+ operator: Exists
+ hostNetwork: false
+ volumes:
+ - name: app-vault-secrets
+ csi:
+ driver: secrets-store.csi.k8s.io
+ readOnly: true
+ volumeAttributes:
+ secretProviderClass: pc-server-app-secret-provider
isys-ownfin-stress, pc-server, Service (v1) has been added:
-
+ # Source: base/templates/svc.yaml
+ apiVersion: v1
+ kind: Service
+ metadata:
+ name: pc-server
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ spec:
+ ports:
+ - name: web
+ port: 80
+ protocol: TCP
+ targetPort: 8080
+ selector:
+ app: pc-server
+ fullname: pc-server
+ release: pc-server
+ type: ClusterIP
isys-ownfin-stress, pc-server-app-cm-env, ConfigMap (v1) has been added:
-
+ # Source: base/templates/cm-env.yaml
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: pc-server-app-cm-env
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk1"
+ data:
+ APP_POSTGRESQL_HOST: "10.21.25.43"
+ APP_POSTGRESQL_NAME: "paycontrol6"
+ APP_POSTGRESQL_PORT: "5432"
+ JAVA_OPTS: "-XX:+PrintFlagsFinal -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0 -XX:+UseTransparentHugePages"
+ SERVER_PATHREWRITES_0_FROM: "/metrics"
+ SERVER_PATHREWRITES_0_TO: "/api/actuator/prometheus"
+ SERVER_SERVLET_CONTEXTPATH: "/api"
+ SPRING_DATASOURCE_URL: "jdbc:postgresql://10.21.25.43:5432/paycontrol6"
isys-ownfin-stress, pc-server-app-secret-env, Secret (v1) has been added:
+ Changes suppressed on sensitive content of type Secret
isys-ownfin-stress, pc-server-app-secret-provider, SecretProviderClass (secrets-store.csi.x-k8s.io) has been added:
-
+ # Source: base/templates/secret-provider.yaml
+ apiVersion: secrets-store.csi.x-k8s.io/v1
+ kind: SecretProviderClass
+ metadata:
+ name: pc-server-app-secret-provider
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk1"
+ spec:
+ provider: vault
+ parameters:
+ vaultKubernetesMountPath: rcsdstbl
+ roleName: "csi-secrets-reader-stress"
+ objects: |
+ - objectName: APP_POSTGRESQL_MIGRATION_PASSWORD
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: APP_POSTGRESQL_MIGRATION_PASSWORD
+ - objectName: APP_POSTGRESQL_MIGRATION_USER
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: APP_POSTGRESQL_MIGRATION_USER
+ - objectName: SPRING_DATASOURCE_PASSWORD
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: SPRING_DATASOURCE_PASSWORD
+ - objectName: SPRING_DATASOURCE_USERNAME
+ secretPath: manual_kv_stress/data/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server
+ secretKey: SPRING_DATASOURCE_USERNAME
+ secretObjects:
+ - secretName: pc-server-app-secret-env
+ type: Opaque
+ labels:
+ app: pc-server
+ fullname: pc-server
+ chart: base-1.15.2
+ release: pc-server
+ heritage: Helm
+ isys: "ownfin-stress"
+ psvc: "pc-server"
+ version: "feature-task-140-rshb.stress"
+ workload.topology.app.farm/zone: "rumsk1"
+ data:
+ - objectName: APP_POSTGRESQL_MIGRATION_PASSWORD
+ key: APP_POSTGRESQL_MIGRATION_PASSWORD
+ - objectName: APP_POSTGRESQL_MIGRATION_USER
+ key: APP_POSTGRESQL_MIGRATION_USER
+ - objectName: SPRING_DATASOURCE_PASSWORD
+ key: SPRING_DATASOURCE_PASSWORD
+ - objectName: SPRING_DATASOURCE_USERNAME
+ key: SPRING_DATASOURCE_USERNAME
Listing releases matching ^pjob-pc-server$
Listing releases matching ^exsvc-pc-server$
Listing releases matching ^data-pc-server$
Listing releases matching ^assets-config-pc-server$
Listing releases matching ^pc-server-raw$
Listing releases matching ^pc-server-grafana-dashboard$
Listing releases matching ^pc-server-rumsk1$
Listing releases matching ^pc-server-rumsk2$
Upgrading release=pc-server, chart=rshb-charts/base
Release "pc-server" does not exist. Installing it now.
FAILED RELEASES:
NAME
pc-server
in deploy/helmfile.yaml: failed processing release pc-server: command "/usr/local/link/helm" exited with non-zero status:
PATH:
/usr/local/link/helm
ARGS:
0: helm (4 bytes)
1: --kube-context (14 bytes)
2: rcsdstbl (8 bytes)
3: upgrade (7 bytes)
4: --install (9 bytes)
5: --reset-values (14 bytes)
6: pc-server (9 bytes)
7: rshb-charts/base (16 bytes)
8: --version (9 bytes)
9: 1.15.2 (6 bytes)
10: --wait (6 bytes)
11: --timeout (9 bytes)
12: 600s (4 bytes)
13: --atomic (8 bytes)
14: --kube-context (14 bytes)
15: rcsdstbl (8 bytes)
16: --namespace (11 bytes)
17: isys-ownfin-stress (18 bytes)
18: --values (8 bytes)
19: /tmp/helmfile2669698693/isys-ownfin-stress-pc-server-values-6d96f8c7fc (70 bytes)
20: --values (8 bytes)
21: /tmp/helmfile463574920/isys-ownfin-stress-pc-server-values-5d5d7df57 (68 bytes)
22: --set (5 bytes)
23: additionalWorkloadAnnotations.gitlabPipeline=https://gitlab.rshbdev.ru/rshbintech/crft/ckof/ownfin/backend/ext-service/pc-server/-/pipelines/3267018 (148 bytes)
24: --set (5 bytes)
25: additionalWorkloadAnnotations.gitlabCommit=e7166e9a19fb964f081fb66543736eb73c0a0123 (83 bytes)
26: --history-max (13 bytes)
27: 10 (2 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: release pc-server failed, and has been uninstalled due to atomic being set: timed out waiting for the condition
COMBINED OUTPUT:
Release "pc-server" does not exist. Installing it now.
Error: release pc-server failed, and has been uninstalled due to atomic being set: timed out waiting for the condition
Uploading artifacts for failed job
00:01
Uploading artifacts...
deploy.env: found 1 matching artifact files and directories
Uploading artifacts as "dotenv" to coordinator... 201 Created id=22776411 responseStatus=201 Created token=64_uzi71
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1