Benchmark run #14

Started2026-05-14 10:51:06.0
Stopped2026-05-14 14:25:35.0
Tests runmeasure-time-to-first-request,measure-rss,run-load-test
Iterations7
HostHetzner vServer · AMD EPYC-Genoa Processor (16 cpus) · 30Gi
OSUbuntu 24.04.3 LTS · 6.8.0-111-generic
Javasemeru-openj9-25.0.2+10_openj9-0.57.0
GraalVMgraalvm-community-25.0.2
Mandrelmandrel-25.0.2.0-Final+java25
dotnet10.0.100
JVM heap-Xms128m -Xmx256m
JVM args-Xgcpolicy:optthruput -Xgcthreads1 -Xminf0.1 -Xmaxf0.2
Quarkus build args-Dquarkus.package.jar.tree-shake.mode=classes
CPU pinning (app/db/load/otel)0-3 · 4-6 · 10-12 · 7-9
Deploy commitfc104f4 · claude/vigorous-mestorf-af0b95 · (dirty)
Deploy run_idc138de3a-8cab-4446-98d3-68f3cc072ded
Deployed at2026-05-14 10:49:27.0

Workload simulation

Closed-form models that translate per-runtime metrics into deployment-archetype outcomes. Inputs come from the measured columns in the per-runtime summary; the math is shown alongside the score so readers can sanity-check it.

Serverless function

Simulated workload: 1,000,000 invocations, cold-start probability 20%. Per-invocation cost = avg latency × RSS-after-first-request (warm) + pcold × TTFR × RSS (cold tax). Result is total billable GB·seconds; lower is better.

RankRuntimeWarm cost
MiB·ms / req
Cold tax
MiB·ms / req
Total
GB·seconds
Ratio vs winner
1quarkus-semeru2,810172,461171,1621.00×
2quarkus-virtual-semeru2,727174,668173,2371.01×
3quarkus-jvm2,749221,428218,9231.28×
4quarkus-virtual2,717221,630219,0881.28×

Long-running service

Simulated workload: 10,000 rps for 24 hours, instances pinned to 80% utilization, Service Level Objective (SLO) target latency 100 ms — i.e. the maximum request latency the service commits to. The simulator uses the measured max latency as a proxy: full credit at the SLO, zero credit at 2× the SLO, linear in between. Score = SLO-compliant requests served per GB·hour of cluster memory; higher is better.

RankRuntimeSLO complianceEffective
rps
Instances
required
Cluster RSS
MiB
Score
SLO-rps / GB·hour
Ratio vs winner
1quarkus-virtual100%10,279262061,137,6321.00×
2quarkus-virtual-semeru76%7,415258646,668,4550.76×
3quarkus-semeru62%5,830263034,090,3300.56×
4quarkus-jvm63%6,388269733,807,4430.55×

Per-runtime summary

Cells show mean ± sample standard deviation across iterations.

n = 7 iterations per cell
RuntimeThroughput
rps
TTFR
ms
rps / MiB
rps/MiB
RSS @ startup
MiB
RSS @ 1st req
MiB
RSS under load
MiB
Conn err
count
Timeouts
count
quarkus-jvm10,127 ± 261 (CV 2.6%)3,958.8 ± 105.9 (CV 2.7%)29.08 ± 0.90 (CV 3.1%)265.8 ± 0.1 (CV 0.05%)279.7 ± 0.3 (CV 0.10%)348.3 ± 6.7 (CV 1.9%)0 ± 00 ± 0
quarkus-semeru9,398 ± 165 (CV 1.8%)3,250.5 ± 93.9 (CV 2.9%)29.83 ± 0.92 (CV 3.1%)255.0 ± 0.4 (CV 0.16%)265.3 ± 0.7 (CV 0.26%)315.2 ± 8.1 (CV 2.6%)0 ± 00 ± 0
quarkus-virtual10,279 ± 180 (CV 1.7%)3,948.8 ± 106.5 (CV 2.7%)33.18 ± 0.86 (CV 2.6%)265.7 ± 0.1 (CV 0.03%)280.6 ± 0.4 (CV 0.15%)309.9 ± 5.7 (CV 1.8%)0 ± 00 ± 0
quarkus-virtual-semeru9,693 ± 113 (CV 1.2%)3,288.9 ± 95.2 (CV 2.9%)33.10 ± 0.38 (CV 1.1%)255.0 ± 0.3 (CV 0.12%)265.5 ± 1.1 (CV 0.41%)292.8 ± 0.8 (CV 0.28%)0 ± 00 ± 0

Charts

Bar = mean across iterations. Error bar = sample standard deviation. Runtimes that didn't produce a metric are omitted from that chart.

Time to first request
Load test throughput
RSS at startup
RSS during load test
Per-iteration raw values
RuntimeMetriciter 0iter 1iter 2iter 3iter 4iter 5iter 6
quarkus-jvmThroughput10,22010,16210,40110,0199,8709,75610,458
quarkus-jvmTTFR4,057.23,912.23,804.73,857.04,013.44,093.63,973.4
quarkus-jvmrps / MiB29.7529.4228.9628.9827.6628.4130.40
quarkus-jvmRSS @ startup265.7265.7265.7265.9265.6266.0265.8
quarkus-jvmRSS @ 1st req279.4279.5279.6279.6279.5280.2279.8
quarkus-jvmRSS under load343.5345.4359.2345.8356.8343.4344.0
quarkus-jvmConn err0000000
quarkus-jvmTimeouts0000000
quarkus-semeruThroughput9,4019,3069,1519,3689,5259,3609,672
quarkus-semeruTTFR3,218.03,378.23,233.03,131.53,173.53,245.73,373.6
quarkus-semerurps / MiB30.0130.2328.2830.4128.9229.9930.97
quarkus-semeruRSS @ startup254.4255.1254.4255.3255.1255.4255.1
quarkus-semeruRSS @ 1st req265.0265.7264.9265.9265.7264.0265.8
quarkus-semeruRSS under load313.3307.8323.6308.0329.3312.1312.3
quarkus-semeruConn err0000000
quarkus-semeruTimeouts0000000
quarkus-virtualThroughput10,34510,19010,28910,02810,19310,29610,612
quarkus-virtualTTFR4,009.53,949.13,985.43,823.03,924.63,824.44,125.3
quarkus-virtualrps / MiB33.4131.6233.2932.7833.3333.3134.51
quarkus-virtualRSS @ startup265.7265.7265.8265.7265.7265.8265.7
quarkus-virtualRSS @ 1st req281.2280.6280.6280.2280.2281.1280.4
quarkus-virtualRSS under load309.6322.3309.0305.9305.8309.1307.5
quarkus-virtualConn err0000000
quarkus-virtualTimeouts0000000
quarkus-virtual-semeruThroughput9,5909,5859,6419,5999,8129,8039,824
quarkus-virtual-semeruTTFR3,287.93,260.63,148.93,408.63,417.13,258.13,241.2
quarkus-virtual-semerurps / MiB32.8232.8532.8832.6833.4233.4033.65
quarkus-virtual-semeruRSS @ startup255.2255.1255.1254.7254.5255.3255.2
quarkus-virtual-semeruRSS @ 1st req264.7264.5266.5264.0265.9266.6266.6
quarkus-virtual-semeruRSS under load292.2291.8293.2293.7293.6293.4292.0
quarkus-virtual-semeruConn err0000000
quarkus-virtual-semeruTimeouts0000000