https://pastein.ru/t/x1S
Загрузка данных
dmitriev-aal@VDI-Dmitriev-A:~$ kubectl --context rcsdstbl -n vector-business logs \
> vector-business-1-vector-agent-vlps5 --since=24h | \
> grep -Ei 'error|warn|quotes|897454e2|opensearch|elastic|sink|failed'
Defaulted container "vector" out of: vector, create-serviceentry (init), create-enrichment-tables (init)
2026-10-06T01:43:34.687594Z WARN sink{component_kind="sink" component_id=elasticsearch_sink component_type=elasticsearch component_name=elasticsearch_sink}:request{request_id=10593}: vector::sinks::util::retries: Retrying after response. reason=500 Internal Server Error: {"error":{"root_cause":[{"type":"security_exception","reason":"Unexpected exception indices:data/write/bulk"}],"type":"security_exception","reason":"Unexpected exception indices:data/write/bulk"},"status":500} internal_log_rate_limit=true
2026-10-06T04:37:23.010914Z WARN sink{component_kind="sink" component_id=elasticsearch_sink component_type=elasticsearch component_name=elasticsearch_sink}:request{request_id=16511}: vector::sinks::util::retries: Retrying after response. reason=500 Internal Server Error: {"error":{"root_cause":[{"type":"security_exception","reason":"Unexpected exception indices:data/write/bulk"}],"type":"security_exception","reason":"Unexpected exception indices:data/write/bulk"},"status":500} internal_log_rate_limit=true
dmitriev-aal@VDI-Dmitriev-A:~$ kubectl --context rcsdstbl -n vector-business \
> describe pod vector-business-1-vector-agent-vlps5
Name: vector-business-1-vector-agent-vlps5
Namespace: vector-business
Priority: 0
Service Account: vector-business-1-vector-agent
Node: afs1asd-wl-279/10.21.182.223
Start Time: Thu, 01 Oct 2026 02:28:50 +0300
Labels: app.kubernetes.io/instance=vector-business-1
app.kubernetes.io/name=vector-agent
controller-revision-hash=84b6489949
pod-template-generation=24
vector.dev/exclude=true
Annotations: checksum/config: 9345902ee6aff5e288604b4ff07b3f5b0868e1caf85e8f190d6a73b981814720
checksum/secrets: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
cni.projectcalico.org/containerID: 19742906a05c429f992209ebac1b429a4d48c0e59c8624180253a61a8b72ef62
cni.projectcalico.org/podIP: 100.79.133.242/32
cni.projectcalico.org/podIPs: 100.79.133.242/32
kubectl.kubernetes.io/restartedAt: 2026-10-01T02:26:30+03:00
Status: Running
IP: 100.79.133.242
IPs:
IP: 100.79.133.242
Controlled By: DaemonSet/vector-business-1-vector-agent
Init Containers:
create-serviceentry:
Container ID: containerd://35db10574cb1bda1be973b0edf34a2c4cf46e9918c623d236e4477e351420991
Image: registry.rshbdev.ru/rshbintech/integrations/ckpr/infra/images/kube-bootstrap:1.18.20-rshb.1.1.0
Image ID: registry.rshbdev.ru/rshbintech/integrations/ckpr/infra/images/kube-bootstrap@sha256:19b597f551b7b5d503acbd5fe2c376a815b35b8cab5e112f22aa53089611cbc2
Port: <none>
Host Port: <none>
Command:
sh
-c
echo "
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: ${NODE_NAME}
spec:
hosts:
- ${NODE_NAME}
ports:
- number: 9000
name: http
protocol: HTTP
location: MESH_EXTERNAL
resolution: STATIC
endpoints:
- address: ${NODE_IP}
" | kubectl apply -f -
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 04 Oct 2026 13:07:59 +0300
Finished: Sun, 04 Oct 2026 13:08:01 +0300
Ready: True
Restart Count: 1
Environment:
NODE_IP: (v1:status.hostIP)
NODE_NAME: (v1:spec.nodeName)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cvxnn (ro)
create-enrichment-tables:
Container ID: containerd://21d26114240dc88421ae0b4bb20301ef065c0530613edff3bb31613e0393e2cb
Image: registry.rshbdev.ru/rshbintech/integrations/ckpr/infra/images/kube-client-apps:8.3.7
Image ID: registry.rshbdev.ru/rshbintech/integrations/ckpr/infra/images/kube-client-apps@sha256:de6be3cf42e8dd2148ffef7afd0b63b8dd332719afb5b3704b218d0b6f198c9e
Port: <none>
Host Port: <none>
Command:
sh
Args:
-c
export ZONE=$(kubectl get nodes $NODE_NAME -o jsonpath='{.metadata.labels.topology\.app\.farm/zone}')
echo "index,zone
1,$ZONE" > table.csv
cp table.csv /output/table.csv
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 04 Oct 2026 13:08:02 +0300
Finished: Sun, 04 Oct 2026 13:08:02 +0300
Ready: True
Restart Count: 0
Environment:
NODE_NAME: (v1:spec.nodeName)
Mounts:
/output from vector-csv (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cvxnn (ro)
Containers:
vector:
Container ID: containerd://cd088d1746567bd7ad27e76ea79b05220d6d82d67f2fad73178b093471426216
Image: registry-proxy.rshbdev.ru/timberio/vector:0.27.1-alpine
Image ID: registry-proxy.rshbdev.ru/timberio/vector@sha256:faaac8315ccad3a05e255ed0f8d02dcbc960ef5c8fdb8a7452fb70f9af08c65e
Ports: 9011/TCP (tcp), 9001/TCP (http), 8686/TCP (api), 9598/TCP (metrics)
Host Ports: 9011/TCP (tcp), 9001/TCP (http), 0/TCP (api), 0/TCP (metrics)
Args:
--config-dir
/etc/vector/
State: Running
Started: Sun, 04 Oct 2026 13:08:04 +0300
Last State: Terminated
Reason: Unknown
Exit Code: 255
Started: Thu, 01 Oct 2026 02:29:15 +0300
Finished: Sun, 04 Oct 2026 13:06:58 +0300
Ready: True
Restart Count: 1
Limits:
cpu: 3
memory: 4Gi
Requests:
cpu: 100m
memory: 128Mi
Liveness: http-get http://:api/health delay=0s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:api/health delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
VECTOR_SELF_NODE_NAME: (v1:spec.nodeName)
VECTOR_SELF_POD_NAME: vector-business-1-vector-agent-vlps5 (v1:metadata.name)
VECTOR_SELF_POD_NAMESPACE: vector-business (v1:metadata.namespace)
PROCFS_ROOT: /host/proc
SYSFS_ROOT: /host/sys
VECTOR_LOG: INFO
VECTOR_WATCH_CONFIG: true
Mounts:
/etc/vector from config-dir (ro)
/opt/table from vector-csv (rw)
/usr/share/cacert from cacert (rw)
/var/lib/kubelet from var-lib-kubelet (ro)
/var/lib/vector from data-dir (rw)
/var/log/ from var-log (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cvxnn (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
var-log:
Type: HostPath (bare host directory volume)
Path: /var/log/
HostPathType:
var-lib-kubelet:
Type: HostPath (bare host directory volume)
Path: /var/lib/kubelet
HostPathType:
data-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/buffer-bus/
HostPathType:
config-dir:
Type: Projected (a volume that contains injected data from multiple sources)
ConfigMapName: vector-business-1-vector-agent
Optional: false
cacert:
Type: Secret (a volume populated by a Secret)
SecretName: rshb-ca-bundle
Optional: false
vector-csv:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-cvxnn:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
Optional: false
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node-role.kubernetes.io/wload:NoSchedule op=Exists
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
nodes-binding.app.farm/system-id:NoSchedule op=Exists
workload.topology.app.farm/zone:NoSchedule op=Exists
Events: <none>
dmitriev-aal@VDI-Dmitriev-A:~$ kubectl --context rcsdstbl -n vector-business exec \
> vector-business-1-vector-agent-vlps5 -- \
> ls -la /var/log/pods | grep quotes
Defaulted container "vector" out of: vector, create-serviceentry (init), create-enrichment-tables (init)
drwxr-xr-x 5 root root 4096 Oct 5 12:30 isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae
dmitriev-aal@VDI-Dmitriev-A:~$ kubectl --context rcsdstbl -n vector-business exec \
> vector-business-1-vector-agent-vlps5 -- \
> sh -c 'find /var/log/pods -path "*quotes*s5msn*" -o -path "*897454e2*"'
Defaulted container "vector" out of: vector, create-serviceentry (init), create-enrichment-tables (init)
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/istio-init
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/istio-init/0.log
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app/0.log.20261005-133420.gz
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app/0.log.20261005-190444.gz
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app/0.log.20261005-132148.gz
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app/0.log
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/app/0.log.20261006-041716
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/istio-proxy
/var/log/pods/isys-sinv-int_quotes-5b5d6675f4-s5msn_897454e2-90d9-454a-bbe0-6281f34892ae/istio-proxy/0.log
dmitriev-aal@VDI-Dmitriev-A:~$ 