Benchmark run #13

Started2026-05-13 16:15:15.0
Stopped2026-05-13 19:44:02.0
NoteRun 13: Quarkus 3.35.2 NO tree-shaking (baseline for tree-shake delta), Temurin 25, SerialGC + aggressive heap return, -Xms128m -Xmx256m, 7 iter — quarkus-jvm, quarkus-leyden, quarkus-virtual, quarkus-virtual-leyden
Tests runmeasure-time-to-first-request,measure-rss,run-load-test
Iterations7
HostHetzner vServer · AMD EPYC-Genoa Processor (16 cpus) · 30Gi
OSUbuntu 24.04.3 LTS · 6.8.0-111-generic
Javatemurin-25.0.2
GraalVMgraalvm-community-25.0.2
Mandrelmandrel-25.0.2.0-Final+java25
dotnet10.0.100
JVM heap-Xms128m -Xmx256m
JVM args-XX:+UseSerialGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20
CPU pinning (app/db/load/otel)0-3 · 4-6 · 10-12 · 7-9
Deploy commitfc104f4 · claude/vigorous-mestorf-af0b95 · (dirty)
Deploy run_id0835a071-4f55-43fc-9fda-6c0e83a9ce20
Deployed at2026-05-12 14:55:36.0

Workload simulation

Closed-form models that translate per-runtime metrics into deployment-archetype outcomes. Inputs come from the measured columns in the per-runtime summary; the math is shown alongside the score so readers can sanity-check it.

Serverless function

Simulated workload: 1,000,000 invocations, cold-start probability 20%. Per-invocation cost = avg latency × RSS-after-first-request (warm) + pcold × TTFR × RSS (cold tax). Result is total billable GB·seconds; lower is better.

RankRuntimeWarm cost
MiB·ms / req
Cold tax
MiB·ms / req
Total
GB·seconds
Ratio vs winner
1quarkus-leyden1,95538,16739,1821.00×
2quarkus-virtual-leyden1,58440,09840,7041.04×
3quarkus-jvm2,071131,869130,8013.34×
4quarkus-virtual1,721137,627136,0823.47×

Long-running service

Simulated workload: 10,000 rps for 24 hours, instances pinned to 80% utilization, Service Level Objective (SLO) target latency 100 ms — i.e. the maximum request latency the service commits to. The simulator uses the measured max latency as a proxy: full credit at the SLO, zero credit at 2× the SLO, linear in between. Score = SLO-compliant requests served per GB·hour of cluster memory; higher is better.

RankRuntimeSLO complianceEffective
rps
Instances
required
Cluster RSS
MiB
Score
SLO-rps / GB·hour
Ratio vs winner
1quarkus-virtual-leyden100%14,5531379141,399,5291.00×
2quarkus-virtual100%14,8321423129,391,7210.92×
3quarkus-leyden100%11,391283250,472,5470.36×
4quarkus-jvm100%12,023290848,799,3430.35×

Per-runtime summary

Cells show mean ± sample standard deviation across iterations.

n = 7 iterations per cell
RuntimeThroughput
rps
TTFR
ms
rps / MiB
rps/MiB
RSS @ startup
MiB
RSS @ 1st req
MiB
RSS under load
MiB
Conn err
count
Timeouts
count
quarkus-jvm12,023 ± 442 (CV 3.7%)2,635.8 ± 40.7 (CV 1.5%)26.48 ± 1.00 (CV 3.8%)234.1 ± 3.6 (CV 1.5%)250.2 ± 3.6 (CV 1.4%)454.1 ± 8.9 (CV 1.9%)0 ± 00 ± 0
quarkus-leyden11,391 ± 584 (CV 5.1%)853.4 ± 36.7 (CV 4.3%)27.39 ± 1.25 (CV 4.6%)217.9 ± 2.4 (CV 1.1%)223.6 ± 1.9 (CV 0.85%)416.0 ± 11.3 (CV 2.7%)0 ± 00 ± 0
quarkus-virtual14,832 ± 290 (CV 2.0%)2,682.8 ± 85.3 (CV 3.2%)35.10 ± 0.65 (CV 1.9%)233.7 ± 4.1 (CV 1.8%)256.5 ± 6.3 (CV 2.4%)422.6 ± 7.7 (CV 1.8%)0 ± 00 ± 0
quarkus-virtual-leyden14,553 ± 306 (CV 2.1%)865.9 ± 28.5 (CV 3.3%)38.38 ± 1.24 (CV 3.2%)222.0 ± 4.0 (CV 1.8%)231.5 ± 4.7 (CV 2.0%)379.4 ± 11.4 (CV 3.0%)0 ± 00 ± 0

Charts

Bar = mean across iterations. Error bar = sample standard deviation. Runtimes that didn't produce a metric are omitted from that chart.

Time to first request
Load test throughput
RSS at startup
RSS during load test
Per-iteration raw values
RuntimeMetriciter 0iter 1iter 2iter 3iter 4iter 5iter 6
quarkus-jvmThroughput11,79912,75411,76911,79511,80411,66912,569
quarkus-jvmTTFR2,619.32,645.02,634.82,555.62,649.22,664.12,682.5
quarkus-jvmrps / MiB26.3428.1526.7126.0625.2125.6127.28
quarkus-jvmRSS @ startup230.8234.9233.2241.4233.0234.6230.7
quarkus-jvmRSS @ 1st req245.6251.0249.8256.3247.8252.9247.8
quarkus-jvmRSS under load447.9453.1440.6452.6468.2455.6460.7
quarkus-jvmConn err0000000
quarkus-jvmTimeouts0000000
quarkus-leydenThroughput10,72410,96412,55411,23011,37611,31611,575
quarkus-leydenTTFR886.7839.6873.6792.0824.7895.0861.9
quarkus-leydenrps / MiB26.5025.4029.2926.9128.1327.8327.63
quarkus-leydenRSS @ startup219.6219.4221.2216.5215.5218.3214.8
quarkus-leydenRSS @ 1st req225.7223.1226.1222.1224.9221.7221.9
quarkus-leydenRSS under load404.6431.6428.6417.3404.4406.6418.9
quarkus-leydenConn err0000000
quarkus-leydenTimeouts0000000
quarkus-virtualThroughput14,63914,64815,01515,24215,14014,55414,587
quarkus-virtualTTFR2,691.52,536.72,782.72,612.72,764.32,708.22,683.6
quarkus-virtualrps / MiB35.5235.2434.7935.2736.1734.4534.29
quarkus-virtualRSS @ startup232.1232.2238.8232.4231.5228.7239.9
quarkus-virtualRSS @ 1st req250.9259.3256.3250.3254.5255.3268.9
quarkus-virtualRSS under load412.1415.7431.5432.2418.5422.4425.4
quarkus-virtualConn err0000000
quarkus-virtualTimeouts0000000
quarkus-virtual-leydenThroughput13,95414,46814,72214,72014,54914,92714,534
quarkus-virtual-leydenTTFR862.8858.7856.7925.2846.5838.3873.4
quarkus-virtual-leydenrps / MiB36.7038.8338.7438.5737.2738.0340.54
quarkus-virtual-leydenRSS @ startup220.7224.2221.4220.9220.9216.5229.6
quarkus-virtual-leydenRSS @ 1st req229.2231.8235.2230.6230.6224.0239.1
quarkus-virtual-leydenRSS under load380.3372.6380.0381.6390.4392.5358.5
quarkus-virtual-leydenConn err0000000
quarkus-virtual-leydenTimeouts0000000