Benchmark run #11

Started2026-05-12 15:40:01.0
Stopped2026-05-12 19:13:41.0
NoteRun 11: OpenJ9/Semeru SCC + Quarkus 3.35.2 tree-shaking, -Xms128m -Xmx256m + OpenJ9 GC (optthruput, -Xgcthreads1, aggressive heap return), 7 iter — quarkus-jvm, quarkus-semeru, quarkus-virtual, quarkus-virtual-semeru
Tests runmeasure-time-to-first-request,measure-rss,run-load-test
Iterations7
HostHetzner vServer · AMD EPYC-Genoa Processor (16 cpus) · 30Gi
OSUbuntu 24.04.3 LTS · 6.8.0-111-generic
Javasemeru-openj9-25.0.2+10_openj9-0.57.0
GraalVMgraalvm-community-25.0.2
Mandrelmandrel-25.0.2.0-Final+java25
dotnet10.0.100
JVM heap-Xms128m -Xmx256m
JVM args-Xgcpolicy:optthruput -Xgcthreads1 -Xminf0.1 -Xmaxf0.2
Quarkus build args-Dquarkus.package.jar.tree-shake.mode=classes
CPU pinning (app/db/load/otel)0-3 · 4-6 · 10-12 · 7-9
Deploy commitfc104f4 · claude/vigorous-mestorf-af0b95 · (dirty)
Deploy run_id0835a071-4f55-43fc-9fda-6c0e83a9ce20
Deployed at2026-05-12 14:55:36.0

Workload simulation

Closed-form models that translate per-runtime metrics into deployment-archetype outcomes. Inputs come from the measured columns in the per-runtime summary; the math is shown alongside the score so readers can sanity-check it.

Serverless function

Simulated workload: 1,000,000 invocations, cold-start probability 20%. Per-invocation cost = avg latency × RSS-after-first-request (warm) + pcold × TTFR × RSS (cold tax). Result is total billable GB·seconds; lower is better.

RankRuntimeWarm cost
MiB·ms / req
Cold tax
MiB·ms / req
Total
GB·seconds
Ratio vs winner

Long-running service

Simulated workload: 10,000 rps for 24 hours, instances pinned to 80% utilization, Service Level Objective (SLO) target latency 100 ms — i.e. the maximum request latency the service commits to. The simulator uses the measured max latency as a proxy: full credit at the SLO, zero credit at 2× the SLO, linear in between. Score = SLO-compliant requests served per GB·hour of cluster memory; higher is better.

RankRuntimeSLO complianceEffective
rps
Instances
required
Cluster RSS
MiB
Score
SLO-rps / GB·hour
Ratio vs winner
1quarkus-virtual-semeru100%9,958259461,779,5771.00×
2quarkus-virtual100%10,134261261,053,4400.99×
3quarkus-semeru100%9,713266853,606,8020.87×
4quarkus-jvm100%9,871270251,849,1020.84×

Per-runtime summary

Cells show mean ± sample standard deviation across iterations.

n = 7 iterations per cell
RuntimeThroughput
rps
TTFR
ms
rps / MiB
rps/MiB
RSS @ startup
MiB
RSS @ 1st req
MiB
RSS under load
MiB
Conn err
count
Timeouts
count
quarkus-jvm9,871 ± 275 (CV 2.8%)4,001.6 ± 134.5 (CV 3.4%)28.14 ± 0.85 (CV 3.0%)264.8 ± 0.2 (CV 0.08%)279.5 ± 1.0 (CV 0.34%)350.9 ± 7.5 (CV 2.1%)0 ± 00 ± 0
quarkus-semeru9,713 ± 120 (CV 1.2%)3,247.6 ± 77.0 (CV 2.4%)29.10 ± 0.76 (CV 2.6%)254.4 ± 0.3 (CV 0.10%)263.9 ± 1.1 (CV 0.42%)334.0 ± 6.8 (CV 2.0%)0 ± 00 ± 0
quarkus-virtual10,134 ± 274 (CV 2.7%)3,986.1 ± 81.4 (CV 2.0%)33.13 ± 0.96 (CV 2.9%)264.9 ± 0.3 (CV 0.10%)280.0 ± 1.0 (CV 0.36%)305.9 ± 1.4 (CV 0.46%)0 ± 00 ± 0
quarkus-virtual-semeru9,958 ± 204 (CV 2.0%)3,337.1 ± 115.6 (CV 3.5%)33.52 ± 0.72 (CV 2.2%)254.2 ± 0.2 (CV 0.08%)264.1 ± 1.1 (CV 0.43%)297.1 ± 2.1 (CV 0.69%)0 ± 00 ± 0

Charts

Bar = mean across iterations. Error bar = sample standard deviation. Runtimes that didn't produce a metric are omitted from that chart.

Time to first request
Load test throughput
RSS at startup
RSS during load test
Per-iteration raw values
RuntimeMetriciter 0iter 1iter 2iter 3iter 4iter 5iter 6
quarkus-jvmThroughput9,83810,29910,0859,9469,4539,6709,804
quarkus-jvmTTFR4,046.93,885.74,079.13,776.53,999.44,189.44,034.2
quarkus-jvmrps / MiB28.4829.6028.3127.3027.3527.3828.54
quarkus-jvmRSS @ startup264.5265.1264.9264.9264.6264.9264.6
quarkus-jvmRSS @ 1st req278.5281.5279.3279.8279.1279.3279.0
quarkus-jvmRSS under load345.5348.0356.3364.3345.6353.2343.5
quarkus-jvmConn err0000000
quarkus-jvmTimeouts0000000
quarkus-semeruThroughput9,9109,7329,7249,6179,5429,6699,795
quarkus-semeruTTFR3,230.93,239.63,160.23,215.93,406.03,214.23,266.6
quarkus-semerurps / MiB29.3229.0728.6528.4728.5128.9930.66
quarkus-semeruRSS @ startup254.1254.2254.4254.6254.7254.3254.7
quarkus-semeruRSS @ 1st req265.1262.7263.0265.4263.4264.8263.3
quarkus-semeruRSS under load338.0334.8339.4337.8334.7333.5319.4
quarkus-semeruConn err0000000
quarkus-semeruTimeouts0000000
quarkus-virtualThroughput10,37610,38910,15410,1709,76310,3349,750
quarkus-virtualTTFR3,879.43,953.33,928.54,053.14,047.93,938.34,102.0
quarkus-virtualrps / MiB33.9033.9333.2633.2631.6233.9331.98
quarkus-virtualRSS @ startup264.6264.9264.7264.7265.4265.2264.7
quarkus-virtualRSS @ 1st req279.2279.6279.3280.0280.2282.1279.4
quarkus-virtualRSS under load306.1306.2305.3305.7308.8304.6304.8
quarkus-virtualConn err0000000
quarkus-virtualTimeouts0000000
quarkus-virtual-semeruThroughput10,1919,94510,0709,7589,62010,00110,119
quarkus-virtual-semeruTTFR3,518.93,222.93,420.23,386.73,183.63,309.83,317.3
quarkus-virtual-semerurps / MiB34.4633.6334.0932.7932.4833.2033.98
quarkus-virtual-semeruRSS @ startup254.1254.0254.3254.4254.2254.0254.5
quarkus-virtual-semeruRSS @ 1st req265.7263.1263.5263.7263.8263.1265.7
quarkus-virtual-semeruRSS under load295.7295.7295.4297.6296.1301.2297.8
quarkus-virtual-semeruConn err0000000
quarkus-virtual-semeruTimeouts0000000