Benchmark run #12

Started2026-05-13 09:29:38.0
Stopped2026-05-13 13:02:49.0
NoteRun 12: Temurin 25 + Quarkus Leyden AOT + Quarkus 3.35.2 tree-shaking, -Xms128m -Xmx256m + SerialGC + aggressive heap return (MinHeapFreeRatio=10 MaxHeapFreeRatio=20), 7 iter — quarkus-jvm, quarkus-leyden, quarkus-virtual, quarkus-virtual-leyden
Tests runmeasure-time-to-first-request,measure-rss,run-load-test
Iterations7
HostHetzner vServer · AMD EPYC-Genoa Processor (16 cpus) · 30Gi
OSUbuntu 24.04.3 LTS · 6.8.0-111-generic
Javatemurin-25.0.2
GraalVMgraalvm-community-25.0.2
Mandrelmandrel-25.0.2.0-Final+java25
dotnet10.0.100
JVM heap-Xms128m -Xmx256m
JVM args-XX:+UseSerialGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20
Quarkus build args-Dquarkus.package.jar.tree-shake.mode=classes
CPU pinning (app/db/load/otel)0-3 · 4-6 · 10-12 · 7-9
Deploy commitfc104f4 · claude/vigorous-mestorf-af0b95 · (dirty)
Deploy run_id0835a071-4f55-43fc-9fda-6c0e83a9ce20
Deployed at2026-05-12 14:55:36.0

Workload simulation

Closed-form models that translate per-runtime metrics into deployment-archetype outcomes. Inputs come from the measured columns in the per-runtime summary; the math is shown alongside the score so readers can sanity-check it.

Serverless function

Simulated workload: 1,000,000 invocations, cold-start probability 20%. Per-invocation cost = avg latency × RSS-after-first-request (warm) + pcold × TTFR × RSS (cold tax). Result is total billable GB·seconds; lower is better.

RankRuntimeWarm cost
MiB·ms / req
Cold tax
MiB·ms / req
Total
GB·seconds
Ratio vs winner
1quarkus-leyden1,98638,71839,7501.00×
2quarkus-virtual-leyden1,65640,89241,5511.05×
3quarkus-jvm1,959123,627122,6423.09×
4quarkus-virtual1,695136,556135,0123.40×

Long-running service

Simulated workload: 10,000 rps for 24 hours, instances pinned to 80% utilization, Service Level Objective (SLO) target latency 100 ms — i.e. the maximum request latency the service commits to. The simulator uses the measured max latency as a proxy: full credit at the SLO, zero credit at 2× the SLO, linear in between. Score = SLO-compliant requests served per GB·hour of cluster memory; higher is better.

RankRuntimeSLO complianceEffective
rps
Instances
required
Cluster RSS
MiB
Score
SLO-rps / GB·hour
Ratio vs winner
1quarkus-virtual-leyden100%14,1601376138,666,8801.00×
2quarkus-virtual100%15,0511414134,116,6750.97×
3quarkus-jvm100%12,369289850,782,6450.37×
4quarkus-leyden100%11,383283550,276,1690.36×

Per-runtime summary

Cells show mean ± sample standard deviation across iterations.

n = 7 iterations per cell
RuntimeThroughput
rps
TTFR
ms
rps / MiB
rps/MiB
RSS @ startup
MiB
RSS @ 1st req
MiB
RSS under load
MiB
Conn err
count
Timeouts
count
quarkus-jvm12,369 ± 565 (CV 4.6%)2,538.1 ± 77.9 (CV 3.1%)27.57 ± 1.61 (CV 5.9%)227.9 ± 6.0 (CV 2.6%)243.5 ± 5.9 (CV 2.4%)448.9 ± 10.5 (CV 2.3%)0 ± 00 ± 0
quarkus-leyden11,383 ± 537 (CV 4.7%)852.9 ± 27.1 (CV 3.2%)27.31 ± 1.63 (CV 6.0%)218.9 ± 2.3 (CV 1.1%)227.0 ± 3.2 (CV 1.4%)417.3 ± 15.9 (CV 3.8%)0 ± 00 ± 0
quarkus-virtual15,051 ± 397 (CV 2.6%)2,663.6 ± 50.6 (CV 1.9%)36.40 ± 1.42 (CV 3.9%)231.4 ± 3.9 (CV 1.7%)256.3 ± 2.7 (CV 1.0%)413.7 ± 9.6 (CV 2.3%)0 ± 00 ± 0
quarkus-virtual-leyden14,160 ± 184 (CV 1.3%)867.9 ± 62.5 (CV 7.2%)37.69 ± 1.95 (CV 5.2%)226.6 ± 4.2 (CV 1.8%)235.6 ± 5.0 (CV 2.1%)376.5 ± 16.3 (CV 4.3%)0 ± 00 ± 0

Charts

Bar = mean across iterations. Error bar = sample standard deviation. Runtimes that didn't produce a metric are omitted from that chart.

Time to first request
Load test throughput
RSS at startup
RSS during load test
Per-iteration raw values
RuntimeMetriciter 0iter 1iter 2iter 3iter 4iter 5iter 6
quarkus-jvmThroughput13,02611,83211,73413,22912,19412,22712,339
quarkus-jvmTTFR2,595.62,508.12,608.42,450.42,647.62,491.82,464.6
quarkus-jvmrps / MiB29.5526.4525.3029.3826.9226.8228.60
quarkus-jvmRSS @ startup239.9222.9229.4228.7222.9223.7227.5
quarkus-jvmRSS @ 1st req255.0238.1244.4245.4237.6240.4243.9
quarkus-jvmRSS under load440.8447.4463.8450.3453.0455.8431.5
quarkus-jvmConn err0000000
quarkus-jvmTimeouts0000000
quarkus-leydenThroughput10,88112,11110,99711,10611,20212,19111,195
quarkus-leydenTTFR837.9841.9848.3824.1852.2909.5856.8
quarkus-leydenrps / MiB26.3029.5427.0724.7526.6328.8728.01
quarkus-leydenRSS @ startup217.0218.0217.9217.6219.0224.0218.8
quarkus-leydenRSS @ 1st req229.3221.5224.5226.2231.4228.2227.7
quarkus-leydenRSS under load413.7410.0406.2448.8420.6422.3399.8
quarkus-leydenConn err0000000
quarkus-leydenTimeouts0000000
quarkus-virtualThroughput15,02315,52715,04615,42915,24514,64314,442
quarkus-virtualTTFR2,589.32,648.12,663.72,673.52,743.72,624.32,702.8
quarkus-virtualrps / MiB34.7037.7936.0038.1037.6035.7234.91
quarkus-virtualRSS @ startup228.5228.5231.3236.9227.0231.5236.2
quarkus-virtualRSS @ 1st req256.8252.6254.8260.3255.6255.1259.2
quarkus-virtualRSS under load433.0410.9418.0404.9405.5409.9413.7
quarkus-virtualConn err0000000
quarkus-virtualTimeouts0000000
quarkus-virtual-leydenThroughput14,24613,88814,34314,28414,01514,01314,335
quarkus-virtual-leydenTTFR778.5974.7866.9825.0904.0883.8842.6
quarkus-virtual-leydenrps / MiB41.7435.5237.5236.9837.2236.7638.05
quarkus-virtual-leydenRSS @ startup220.1224.1227.9225.7228.5233.6226.0
quarkus-virtual-leydenRSS @ 1st req228.7231.3241.8239.4235.3240.1232.5
quarkus-virtual-leydenRSS under load341.3390.9382.2386.3376.5381.1376.8
quarkus-virtual-leydenConn err0000000
quarkus-virtual-leydenTimeouts0000000