Загрузка данных


PRISM v5.23.0 [690b962] [sched=eevdf+pandemonium]
captured: 20261004-140722

SCHEDULER STABILITY
  no ejection -- scheduler ran clean

CLEAN-ROOM
  state        NOISY -- load 0.4/16cpu, uptime 2.5h

SYSTEM
  cpu        AMD Ryzen 9 5950X 16-Core Processor (16c/16t)
  memory     31,2 GiB
  gpu        amdgpu (1002:747E)
  kernel     7.2.9-1-cachyos
  scheduler  pandemonium

STALL SUSCEPTIBILITY
  kernel     7.2.9-1-cachyos
  tick       HZ=1000  NO_HZ_FULL  preempt=PREEMPT_DYNAMIC
  cpus       16  governor=performance  deepest-idle=C2 (3 states)
  cmdline    nohz_full=-  isolcpus=-
  verdict    score 4 -> MODERATE
  factors    HZ=1000 (tight ticks -- narrows window); idle CPUs go tickless (NO_HZ_IDLE behavior; NO_HZ_FULL built but no nohz_full= mask, so unused); 16 CPUs (many idle at once)

COMPARISON  (p99 wake2run, lower is better)
  fork-thread        EEVDF n/a -> BPF 16723us, ADAPTIVE 23688us
  ipc                EEVDF n/a -> BPF 667us, ADAPTIVE 670us
  pcpu-burst         EEVDF n/a -> PANDEMONIUM 23998us
  sojourn-pressure   EEVDF n/a -> PANDEMONIUM 938us
  cachyos            EEVDF n/a -> BPF 998us, ADAPTIVE 999us

POPULATION  (montauk cross-run statistics over 9 run(s), per workload)
  cachyos-primes     eevdf vs pandemonium-adaptive: N=2 cliff +1.00 perm p=0.018 power n=3
  cachyos-primes     eevdf vs pandemonium-bpf: N=2 cliff +1.00 perm p=0.018 power n=3
  cachyos-primes     pandemonium-adaptive vs pandemonium-bpf: N=9 cliff +0.09 perm p=0.873 power n>20
  cachyos-stress-ng-cpu-cache-mem eevdf vs pandemonium-adaptive: N=2 cliff +0.89 perm p=0.036 power n=4
  cachyos-stress-ng-cpu-cache-mem eevdf vs pandemonium-bpf: N=2 cliff +1.00 perm p=0.018 power n=4
  cachyos-stress-ng-cpu-cache-mem pandemonium-adaptive vs pandemonium-bpf: N=9 cliff +0.33 perm p=0.326 power n>20
  cachyos-xz-compression eevdf vs pandemonium-adaptive: N=2 cliff +1.00 perm p=0.018 power n=4
  cachyos-xz-compression eevdf vs pandemonium-bpf: N=2 cliff +1.00 perm p=0.018 power n=3
  cachyos-xz-compression pandemonium-adaptive vs pandemonium-bpf: N=9 cliff +0.11 perm p=0.718 power n>20
  cold-wake-16c      eevdf vs pandemonium: N=2 cliff -0.86 perm p=0.278 power n=9
  cold-wake-starve-16c eevdf vs pandemonium: N=2 cliff +1.00 perm p=0.028 power n=3
  fork-thread        eevdf vs pandemonium-adaptive: N=2 cliff +1.00 perm p=0.018 power n=3
  fork-thread        eevdf vs pandemonium-bpf: N=2 cliff +1.00 perm p=0.018 power n=3
  fork-thread        pandemonium-adaptive vs pandemonium-bpf: N=9 cliff -0.04 perm p=0.842 power n>20
  ipc-16c            eevdf vs pandemonium-adaptive: N=2 cliff +0.14 perm p=0.833 power n=19
  ipc-16c            eevdf vs pandemonium-bpf: N=2 cliff -0.14 perm p=0.444 power n=14
  ipc-16c            pandemonium-adaptive vs pandemonium-bpf: N=7 cliff -0.06 perm p=0.797 power n>20
  ipc-32c            pandemonium-adaptive vs pandemonium-bpf: N=2 cliff +1.00 perm p=0.333 power n=3
  pcpu-burst-16c     eevdf vs pandemonium: N=2 cliff -0.57 perm p=0.222 power n=8
  sojourn-pressure-16c eevdf vs pandemonium: N=2 cliff +0.86 perm p=0.056 power n=5
  cachyos-primes     MCB best (lower p99): pandemonium-adaptive (tied), pandemonium-bpf (tied)
  cachyos-stress-ng-cpu-cache-mem MCB best (lower p99): pandemonium-adaptive (tied), pandemonium-bpf (tied)
  cachyos-xz-compression MCB best (lower p99): pandemonium-adaptive (tied), pandemonium-bpf (tied)
  cold-wake-16c      MCB best (lower p99): eevdf
  cold-wake-starve-16c MCB best (lower p99): pandemonium
  fork-thread        MCB best (lower p99): pandemonium-adaptive (tied), pandemonium-bpf (tied)
  ipc-16c            MCB best (lower p99): eevdf (tied), pandemonium-adaptive (tied)
  ipc-32c            MCB best (lower p99): pandemonium-bpf
  pcpu-burst-16c     MCB best (lower p99): eevdf
  sojourn-pressure-16c MCB best (lower p99): pandemonium
  N<3 per arm -- montauk reports the effect size but the inference is underpowered; repeat runs accumulate here

STORM  (cpu_release flood, powersave -- lower storm% is better)
  clean -- no storm intervals; storm 0/72 intervals (0.0%); reenq/s p50=13428 peak=13636; kick/s=219708 (preempt 118621) reenq/s=12607


WORKLOAD cachyos-PANDEMONIUM-ADAPTIVE-primes
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
hot-cpu        15                 l2_miss_share           51.8416  HIGH
long-range-dependent dispatch-rate      hurst                  0.586287  MED
discord        682ms              profile_vs_mean         1.58664  MED
long-range-dependent migration-rate     hurst                  0.586071  MED

CAPTURE  complete -- 11119 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 6.1k wake2run; p50 4us p99 1001us p999 1003us worst 2403us; 76.4% fast(<100us) / 18.1% mid / 5.5% tick-floor(>=900us); 2.9% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~4020 distinct values; inversion ratio 0.58

WORKLOAD cachyos-PANDEMONIUM-ADAPTIVE-stress-ng-cpu-cache-mem
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
long-range-dependent migration-rate     hurst                  0.882687  MED
migration-avalanche migration-rate     runs                         67  MED
discord        8390ms             profile_vs_mean         2.11169  MED
long-range-dependent dispatch-rate      hurst                  0.876169  MED
hot-cpu        15                 l2_miss_share           6.84308  LOW

CAPTURE  complete -- 71066 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 61.2k wake2run; p50 4us p99 1000us p999 1002us worst 1677us; 81.8% fast(<100us) / 16.0% mid / 2.3% tick-floor(>=900us); 4.6% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~50719 distinct values; inversion ratio 0.53

WORKLOAD cachyos-PANDEMONIUM-ADAPTIVE-xz-compression
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
long-range-dependent dispatch-rate      hurst                   0.68998  MED
migration-avalanche migration-rate     runs                         15  MED
discord        2372ms             profile_vs_mean         1.57868  MED
long-range-dependent migration-rate     hurst                  0.584428  MED
ioctl-slow     469490/xz          duration_ms             0.00047  LOW
hot-cpu        10                 l2_miss_share           7.47497  LOW
ioctl-slow     469490/xz          duration_ms             0.00035  LOW

CAPTURE  complete -- 74316 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 12.1k wake2run; p50 4us p99 999us p999 1004us worst 1998us; 81.4% fast(<100us) / 16.3% mid / 2.3% tick-floor(>=900us); 2.5% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~9642 distinct values; inversion ratio 0.48

WORKLOAD cachyos-PANDEMONIUM-BPF-primes
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
hot-cpu        15                 l2_miss_share           55.6374  HIGH
long-range-dependent migration-rate     hurst                  0.605498  MED
migration-avalanche migration-rate     runs                         19  MED
discord        617ms              profile_vs_mean         1.65303  MED
long-range-dependent dispatch-rate      hurst                   0.59497  MED

CAPTURE  complete -- 11443 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 6.5k wake2run; p50 4us p99 1000us p999 1002us worst 1003us; 72.9% fast(<100us) / 22.3% mid / 4.8% tick-floor(>=900us); 3.5% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~4542 distinct values; inversion ratio 0.52

WORKLOAD cachyos-PANDEMONIUM-BPF-stress-ng-cpu-cache-mem
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
long-range-dependent dispatch-rate      hurst                  0.883447  MED
migration-avalanche migration-rate     runs                        100  MED
discord        9759ms             profile_vs_mean         1.87373  MED
long-range-dependent migration-rate     hurst                  0.835662  MED
hot-cpu        15                 l2_miss_share            7.0621  LOW

CAPTURE  complete -- 70853 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 59.3k wake2run; p50 4us p99 823us p999 1000us worst 1004us; 79.6% fast(<100us) / 19.8% mid / 0.6% tick-floor(>=900us); 3.8% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~52805 distinct values; inversion ratio 0.47
LOCATED: 1 structured region(s) NEARLY_SORTED[71%..72%]

WORKLOAD cachyos-PANDEMONIUM-BPF-xz-compression
POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
long-range-dependent dispatch-rate      hurst                  0.667594  MED
migration-avalanche migration-rate     runs                         18  MED
discord        827ms              profile_vs_mean         1.54144  MED
long-range-dependent migration-rate     hurst                  0.613145  MED
ioctl-slow     469258/xz          duration_ms             0.00062  LOW
hot-cpu        11                 l2_miss_share           7.65949  LOW
ioctl-slow     469258/xz          duration_ms              0.0005  LOW

CAPTURE  complete -- 74920 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 12.4k wake2run; p50 4us p99 998us p999 1003us worst 1048us; 81.7% fast(<100us) / 16.4% mid / 1.9% tick-floor(>=900us); 4.0% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~10072 distinct values; inversion ratio 0.48

WORKLOAD cold-wake-16c-PANDEMONIUM
THERMAL/POWER
  cpu temp   peak 58.2 C  avg 51.5 C
  power      avg 71.8 W  peak 104.0 W
  idle       C2 avg 96.1% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
hot-cpu        4                  l2_miss_share           70.0112  HIGH
kthread-strand comm#288f4905      max_strand_ms           21.9971  MED
long-range-dependent dispatch-rate      hurst                  0.695716  MED
kthread-strand comm#1beae9e3      max_strand_ms           6.00038  MED

CAPTURE  complete -- 1000 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 168 wake2run; p50 2us p99 678us p999 1559us worst 1559us; 97.0% fast(<100us) / 2.4% mid / 0.6% tick-floor(>=900us); 0.0% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~157 distinct values; inversion ratio 0.48

REPORT kstrand
VERDICT: 3 strands across 2 per-CPU kthreads; worst HELD strand 22.0ms (I/O-completion freeze signature)
kthread            cpu   strands max_ms    p99_ms    held   dark   held_by
comm#288f4905      4     2       22.0      22.0      2      0      tid=0 0%
comm#1beae9e3      4     1       6.0       6.0       1      0      tid=0 0%
  COLD-WAKE: cold-cache worst 3.3x at 32KB (0.02ms cold); CPU freq ramp worst +43%

WORKLOAD cold-wake-starve-16c-PANDEMONIUM
THERMAL/POWER
  cpu temp   peak 61.5 C  avg 53.7 C
  power      avg 84.7 W  peak 108.0 W
  idle       C2 avg 94.7% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
discord        42470ms            profile_vs_mean         3.30513  HIGH
hot-cpu        15                 l2_miss_share           66.0071  HIGH
mid-trace-death comm#e3e3b2d0/489914 before_end_s             35.023  MED
long-range-dependent dispatch-rate      hurst                  0.917944  MED
mid-trace-death comm#e3e3b2d0/489943 before_end_s            28.0028  MED
mid-trace-death comm#e3e3b2d0/490032 before_end_s            7.00089  MED

CAPTURE  complete -- 216034 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 98.4k wake2run; p50 2us p99 5us p999 11us worst 777us; 100.0% fast(<100us) / 0.0% mid / 0.0% tick-floor(>=900us); 0.0% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~70721 distinct values; inversion ratio 0.70
LOCATED: 3 structured region(s) FEW_UNIQUE[27%..27%], FEW_UNIQUE[28%..30%], FEW_UNIQUE[35%..35%]

WORKLOAD fork-thread-PANDEMONIUM-ADAPTIVE
THERMAL/POWER
  cpu temp   peak 62.0 C  avg 55.1 C
  power      avg 96.9 W  peak 164.6 W
  idle       C2 avg 77.0% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
order-starved  tid=467398         passover_picks             8745  HIGH
held-cpu       tid=473134         held_ms                 9.47399  HIGH
self-preempt   tid=467553         stint_wake_ratio           8739  HIGH
self-preempt   tid=467398         stint_wake_ratio           8738  HIGH
self-preempt   tid=467554         stint_wake_ratio           8720  HIGH
self-preempt   tid=467440         stint_wake_ratio           8697  HIGH
order-starved  tid=467554         passover_picks             8243  HIGH
held-cpu       tid=473331         held_ms                 7.62668  HIGH
self-preempt   tid=473084         stint_wake_ratio           4811  HIGH
self-preempt   tid=473086         stint_wake_ratio           2877  HIGH
self-preempt   tid=473083         stint_wake_ratio           1866  HIGH
order-starved  tid=467440         passover_picks             8124  HIGH
held-cpu       tid=473209         held_ms                 4.87002  HIGH
self-preempt   tid=473085         stint_wake_ratio           1443  HIGH
long-range-dependent dispatch-rate      hurst                  0.834541  MED
migration-avalanche migration-rate     runs                        362  MED
discord        144ms              profile_vs_mean         1.76619  MED
long-range-dependent migration-rate     hurst                  0.809113  MED
hot-cpu        5                  l2_miss_share           6.65894  LOW

CAPTURE  complete -- 548798 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 64.2k wake2run; p50 2176us p99 23688us p999 37148us worst 39599us; 22.4% fast(<100us) / 14.3% mid / 63.3% tick-floor(>=900us); 3.6% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~64229 distinct values; inversion ratio 0.61
LOCATED: 2 structured region(s) NEARLY_SORTED[0%..1%], NEARLY_SORTED[79%..80%]

REPORT dispatch-stall
VERDICT: 40.7k saturated wakes over the 900us floor; PREEMPT-STARVED 0% (0 intervening picks) / ORDER-STARVED 100% (CPU served others first); avg 47.6 pass-overs, p99 230 pass-overs

WORKLOAD fork-thread-PANDEMONIUM-BPF
THERMAL/POWER
  cpu temp   peak 62.0 C  avg 55.4 C
  power      avg 95.9 W  peak 165.1 W
  idle       C2 avg 74.7% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
order-starved  tid=467440         passover_picks            33411  HIGH
held-cpu       tid=472031         held_ms                 11.6609  HIGH
self-preempt   tid=467553         stint_wake_ratio           8624  HIGH
self-preempt   tid=467554         stint_wake_ratio           8605  HIGH
self-preempt   tid=467398         stint_wake_ratio           8573  HIGH
self-preempt   tid=467440         stint_wake_ratio           8550  HIGH
order-starved  tid=467553         passover_picks            29140  HIGH
held-cpu       tid=472030         held_ms                 6.17598  HIGH
self-preempt   tid=471973         stint_wake_ratio           3421  HIGH
self-preempt   tid=471975         stint_wake_ratio           2625  HIGH
self-preempt   tid=471972         stint_wake_ratio           1863  HIGH
order-starved  tid=467554         passover_picks            16362  HIGH
held-cpu       tid=472268         held_ms                 5.99774  HIGH
self-preempt   tid=471974         stint_wake_ratio           1401  HIGH
long-range-dependent dispatch-rate      hurst                   0.80872  MED
migration-avalanche migration-rate     runs                        360  MED
discord        577ms              profile_vs_mean         1.87049  MED
long-range-dependent migration-rate     hurst                  0.808628  MED
hot-cpu        8                  l2_miss_share           6.50245  LOW

CAPTURE  complete -- 593756 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 66.9k wake2run; p50 2237us p99 16723us p999 27875us worst 30411us; 24.3% fast(<100us) / 13.4% mid / 62.3% tick-floor(>=900us); 3.4% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~66902 distinct values; inversion ratio 0.58
LOCATED: 1 structured region(s) NEARLY_SORTED[0%..1%]

REPORT dispatch-stall
VERDICT: 41.7k saturated wakes over the 900us floor; PREEMPT-STARVED 0% (0 intervening picks) / ORDER-STARVED 100% (CPU served others first); avg 49.5 pass-overs, p99 228 pass-overs

WORKLOAD ipc-PANDEMONIUM-ADAPTIVE-16c
THERMAL/POWER
  cpu temp   peak 57.5 C  avg 53.0 C
  power      avg 116.0 W  peak 141.3 W
  idle       C2 avg 40.3% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
order-starved  comm#8a5f93b2      passover_picks               94  HIGH
held-cpu       comm#8a5f93b2      held_ms                 22.1587  HIGH
self-preempt   tid=486248         stint_wake_ratio          29158  HIGH
hot-cpu        0                  l2_miss_share           19.6138  HIGH
self-preempt   tid=486256         stint_wake_ratio          27789  HIGH
self-preempt   tid=486249         stint_wake_ratio          25498  HIGH
self-preempt   tid=486266         stint_wake_ratio          20814  HIGH
order-starved  comm#8a5f93b2      passover_picks               81  HIGH
held-cpu       comm#8a5f93b2      held_ms                 18.8434  HIGH
self-preempt   tid=467440         stint_wake_ratio          11165  HIGH
self-preempt   tid=467553         stint_wake_ratio          11147  HIGH
self-preempt   tid=467398         stint_wake_ratio          11103  HIGH
order-starved  comm#8a5f93b2      passover_picks               77  HIGH
held-cpu       comm#8a5f93b2      held_ms                 16.3602  HIGH
self-preempt   tid=467554         stint_wake_ratio          11098  HIGH
hot-waker      486319             wakes_issued              79619  MED
long-range-dependent dispatch-rate      hurst                  0.884915  MED
migration-avalanche migration-rate     runs                        701  MED
long-range-dependent migration-rate     hurst                  0.573166  MED
recvfrom-slow  486296/pand-ipc    duration_ms             1.67516  LOW
recvfrom-slow  486294/pand-ipc    duration_ms             1.22195  LOW
recvfrom-slow  486294/pand-ipc    duration_ms             1.04714  LOW
recvfrom-slow  486294/pand-ipc    duration_ms            0.798966  LOW
recvfrom-slow  486294/pand-ipc    duration_ms            0.765166  LOW
ioctl-slow     486312/python3     duration_ms             0.00137  LOW
ioctl-slow     486312/python3     duration_ms             0.00054  LOW
ioctl-slow     486312/python3     duration_ms              0.0005  LOW
ioctl-slow     486312/python3     duration_ms             0.00047  LOW
ioctl-slow     486312/python3     duration_ms             0.00045  LOW

CAPTURE  complete -- 1695678 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 259k wake2run; p50 3us p99 670us p999 1112us worst 1967us; 95.5% fast(<100us) / 4.0% mid / 0.4% tick-floor(>=900us); 4.5% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~178358 distinct values; inversion ratio 0.51
LOCATED: 33 structured region(s) FEW_UNIQUE[0%..0%], FEW_UNIQUE[0%..1%], FEW_UNIQUE[1%..1%], FEW_UNIQUE[2%..3%], FEW_UNIQUE[3%..4%] +28 more

REPORT dispatch-stall
VERDICT: 1.1k saturated wakes over the 900us floor; PREEMPT-STARVED 17% (0 intervening picks) / ORDER-STARVED 83% (CPU served others first); avg 3.4 pass-overs, p99 12 pass-overs

WORKLOAD ipc-PANDEMONIUM-BPF-16c
THERMAL/POWER
  cpu temp   peak 57.8 C  avg 54.1 C
  power      avg 119.2 W  peak 141.6 W
  idle       C2 avg 30.6% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
order-starved  comm#8a5f93b2      passover_picks               73  HIGH
held-cpu       comm#8a5f93b2      held_ms                 17.1721  HIGH
self-preempt   tid=485892         stint_wake_ratio          29710  HIGH
self-preempt   tid=485895         stint_wake_ratio          28557  HIGH
self-preempt   tid=485897         stint_wake_ratio          20552  HIGH
self-preempt   tid=485899         stint_wake_ratio          19875  HIGH
order-starved  comm#328ab0c6      passover_picks               62  HIGH
held-cpu       comm#8a5f93b2      held_ms                 15.3571  HIGH
self-preempt   tid=485880         stint_wake_ratio          12709  HIGH
self-preempt   tid=467440         stint_wake_ratio          12222  HIGH
self-preempt   tid=467398         stint_wake_ratio          12205  HIGH
order-starved  comm#8a5f93b2      passover_picks               60  HIGH
held-cpu       comm#8a5f93b2      held_ms                 14.2286  HIGH
self-preempt   tid=467553         stint_wake_ratio          12157  HIGH
hot-waker      485952             wakes_issued              79661  MED
long-range-dependent dispatch-rate      hurst                   0.91042  MED
migration-avalanche migration-rate     runs                        420  MED
discord        5216ms             profile_vs_mean          1.6239  MED
hot-cpu        1                  l2_miss_share           15.6103  MED
long-range-dependent migration-rate     hurst                  0.575401  MED
recvfrom-slow  485929/pand-ipc    duration_ms             3.02187  LOW
recvfrom-slow  485931/pand-ipc    duration_ms             3.02169  LOW
recvfrom-slow  485929/pand-ipc    duration_ms             2.08727  LOW
recvfrom-slow  485931/pand-ipc    duration_ms             2.08336  LOW
recvfrom-slow  485929/pand-ipc    duration_ms             2.02452  LOW
ioctl-slow     485947/python3     duration_ms             0.00099  LOW
ioctl-slow     485947/python3     duration_ms             0.00055  LOW
ioctl-slow     485947/python3     duration_ms             0.00049  LOW
ioctl-slow     485947/python3     duration_ms             0.00047  LOW
ioctl-slow     485947/python3     duration_ms             0.00038  LOW

CAPTURE  complete -- 1712262 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 257k wake2run; p50 3us p99 667us p999 994us worst 2338us; 95.7% fast(<100us) / 4.0% mid / 0.3% tick-floor(>=900us); 2.7% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~192957 distinct values; inversion ratio 0.50
LOCATED: 39 structured region(s) FEW_UNIQUE[1%..1%], FEW_UNIQUE[1%..1%], FEW_UNIQUE[2%..2%], FEW_UNIQUE[3%..4%], FEW_UNIQUE[4%..4%] +34 more

REPORT dispatch-stall
VERDICT: 860 saturated wakes over the 900us floor; PREEMPT-STARVED 16% (0 intervening picks) / ORDER-STARVED 84% (CPU served others first); avg 2.2 pass-overs, p99 11 pass-overs

WORKLOAD pcpu-burst-16c-PANDEMONIUM
THERMAL/POWER
  cpu temp   peak 63.5 C  avg 59.9 C
  power      avg 137.2 W  peak 161.9 W
  idle       C2 avg 30.5% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
order-starved  comm#328ab0c6      passover_picks            11755  HIGH
doublefree-logic 486700/0x0000558bdef517f0 bytes                        24  HIGH
self-preempt   tid=467553         stint_wake_ratio          17558  HIGH
doublefree-logic 486699/0x0000564318bf47f0 bytes                        24  HIGH
doublefree-logic 486698/0x000055f60257d7f0 bytes                        24  HIGH
self-preempt   tid=467554         stint_wake_ratio          17466  HIGH
doublefree-logic 486697/0x0000561c9d5cf7f0 bytes                        24  HIGH
doublefree-logic 486696/0x000055aab896f7f0 bytes                        24  HIGH
self-preempt   tid=467440         stint_wake_ratio          17437  HIGH
doublefree-logic 486695/0x000055b0435cd7f0 bytes                        24  HIGH
doublefree-logic 486694/0x000055b61c55c7f0 bytes                        24  HIGH
self-preempt   tid=467398         stint_wake_ratio          17212  HIGH
doublefree-logic 486693/0x0000558aa17777f0 bytes                        24  HIGH
order-starved  comm#328ab0c6      passover_picks            10403  HIGH
doublefree-logic 486692/0x00005588174457f0 bytes                        24  HIGH
self-preempt   tid=16             stint_wake_ratio           4524  HIGH
doublefree-logic 486691/0x0000558c2e8cc7f0 bytes                        24  HIGH
doublefree-logic 486690/0x0000556352f2f7f0 bytes                        24  HIGH
self-preempt   tid=486662         stint_wake_ratio           4232  HIGH
doublefree-logic 486688/0x0000561f08a857f0 bytes                        24  HIGH
doublefree-logic 486684/0x000056540efbd7f0 bytes                        24  HIGH
self-preempt   tid=486641         stint_wake_ratio           3013  HIGH
doublefree-logic 486683/0x0000560a2f5747f0 bytes                        24  HIGH
doublefree-logic 486680/0x000055c4075627f0 bytes                        24  HIGH
order-starved  comm#328ab0c6      passover_picks             6412  HIGH
doublefree-logic 486679/0x0000555f02b3e7f0 bytes                        24  HIGH
self-preempt   tid=981            stint_wake_ratio           2330  HIGH
kthread-strand comm#12f62140      max_strand_ms           9.91583  MED
long-range-dependent dispatch-rate      hurst                  0.855332  MED
discord        11430ms            profile_vs_mean         2.41306  MED
poll-slow      486669/pandemonium duration_ms              0.0015  LOW
hot-cpu        0                  l2_miss_share           7.55532  LOW
poll-slow      486670/pandemonium duration_ms             0.00128  LOW
poll-slow      486671/pandemonium duration_ms             0.00127  LOW
poll-slow      486685/pandemonium duration_ms             0.00122  LOW
poll-slow      486673/pandemonium duration_ms             0.00121  LOW

CAPTURE  complete -- 465548 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 27.9k wake2run; p50 4358us p99 23998us p999 27106us worst 31997us; 24.2% fast(<100us) / 3.5% mid / 72.3% tick-floor(>=900us); 0.0% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~27451 distinct values; inversion ratio 0.35
LOCATED: 8 structured region(s) NEARLY_SORTED[20%..22%], NEARLY_SORTED[29%..31%], NEARLY_SORTED[39%..42%], NEARLY_SORTED[50%..51%], NEARLY_SORTED[59%..61%] +3 more

REPORT dispatch-stall
VERDICT: 20.2k saturated wakes over the 900us floor; PREEMPT-STARVED 0% (0 intervening picks) / ORDER-STARVED 100% (CPU served others first); avg 13.6 pass-overs, p99 61 pass-overs

REPORT kstrand
VERDICT: 1 strands across 1 per-CPU kthreads; worst HELD strand 9.9ms (I/O-completion freeze signature)
kthread            cpu   strands max_ms    p99_ms    held   dark   held_by
comm#12f62140      6     1       9.9       9.9       1      0      comm#8a5f93b2 21%

WORKLOAD sojourn-pressure-16c-PANDEMONIUM
THERMAL/POWER
  cpu temp   peak 61.0 C  avg 57.3 C
  power      avg 128.9 W  peak 156.2 W
  idle       C2 avg 37.9% (dominant)

POORLY-BEHAVING ITEMS (ranked)
kind           id                 metric                    value  sev
held-cpu       comm#8a5f93b2      held_ms                 3.24178  HIGH
self-preempt   tid=467440         stint_wake_ratio          13030  HIGH
self-preempt   tid=467398         stint_wake_ratio          13003  HIGH
self-preempt   tid=467554         stint_wake_ratio          12991  HIGH
self-preempt   tid=467553         stint_wake_ratio          12978  HIGH
held-cpu       comm#8a5f93b2      held_ms                 2.89858  HIGH
self-preempt   tid=489335         stint_wake_ratio           4690  HIGH
self-preempt   tid=489420         stint_wake_ratio           2536  HIGH
self-preempt   tid=489355         stint_wake_ratio           2534  HIGH
held-cpu       comm#8a5f93b2      held_ms                 2.20147  HIGH
self-preempt   tid=489398         stint_wake_ratio           2532  HIGH
order-starved  comm#328ab0c6      passover_picks                4  MED
mean-reverting dispatch-rate      hurst                  0.318688  MED
discord        1864ms             profile_vs_mean         1.61897  MED
order-starved  comm#328ab0c6      passover_picks                2  MED
order-starved  comm#8a5f93b2      passover_picks                2  MED
hot-cpu        15                 l2_miss_share            9.1186  LOW

CAPTURE  complete -- 287138 events, none dropped

KEY METRICS
REPORT sched
VERDICT: 4.4k wake2run; p50 4us p99 938us p999 1166us worst 1804us; 96.0% fast(<100us) / 2.9% mid / 1.1% tick-floor(>=900us); 0.0% cross-domain
STRUCTURE: latency-over-trace RANDOM; ~2962 distinct values; inversion ratio 0.58

REPORT dispatch-stall
VERDICT: 48 saturated wakes over the 900us floor; PREEMPT-STARVED 71% (0 intervening picks) / ORDER-STARVED 29% (CPU served others first); avg 1.4 pass-overs, p99 3 pass-overs