Benchmark run #10

Started2026-05-03 10:37:27.0
Stopped2026-05-03 19:09:31.0
NoteRun 10: Quarkus container-image preset (Parallel GC + GCTimeRatio=4 + AdaptiveSizePolicyWeight=90 + aggressive heap return), -Xms512m -Xmx512m, JDK 25, 7 iter
Tests runmeasure-time-to-first-request,measure-rss,run-load-test
Iterations7
HostHetzner vServer · AMD EPYC-Genoa Processor (16 cpus) · 30Gi
OSUbuntu 24.04.3 LTS · 6.8.0-110-generic
Javatemurin-25.0.2
GraalVMgraalvm-community-25.0.2
Mandrelmandrel-25.0.2.0-Final+java25
dotnet10.0.100
JVM heap-Xms512m -Xmx512m
JVM args-XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError
CPU pinning (app/db/load/otel)0-3 · 4-6 · 10-12 · 7-9
Deploy commit090f8e2 · main
Deploy run_id9b6f2b9d-16de-4445-83eb-9a2f6cc6100d
Deployed at2026-05-02 22:08:11.0

dotnet vs Quarkus comparison

Per metric, the ratio in parentheses is quarkus / dotnet: values above 1× mean Quarkus produced a higher number, below 1× means dotnet did. Green marks the per-row winner within each category. Red marks RSS values that exceed the runtime's configured memory budget. Two importance-weighted scores are reported per category: the Overall winner uses the table-row order (Throughput → TTFR → rps/MiB → memory rows → build time → errors), while the Efficiency-weighted winner reorders to rps/MiB → TTFR → throughput → RSS-under-load. In both, the leading metric outweighs every later metric combined, so ranking is effectively lexicographic in the chosen order.

dotnet-aspnet-ef vs JVM-mode Quarkus variants

Overall winner: Quarkus (JVM-mode) (metric wins: dotnet-aspnet-ef=5, Quarkus (JVM-mode)=2, ties=2; tiebreaker: Throughput)

Efficiency-weighted winner (rps/MiB → TTFR → throughput → RSS load): dotnet-aspnet-ef (metric wins: dotnet-aspnet-ef=2, Quarkus (JVM-mode)=2; tiebreaker: rps / MiB)

Metricdotnet-aspnet-efquarkus-jvmquarkus-leydenquarkus-virtualquarkus-virtual-leyden
valueratiovalueratiovalueratiovalueratio
Throughput (rps)6,60118,6572.83×17,0742.59×17,6872.68×16,3582.48×
TTFR (ms)1,426.82,535.71.78×985.00.690×2,556.51.79×859.90.603×
rps / MiB (rps/MiB)29.0622.760.783×22.980.791×21.210.730×20.630.710×
RSS @ startup (MiB)74.8246.63.30×240.73.22×250.63.35×223.62.99×
RSS @ 1st req (MiB)133.1264.11.98×253.21.90×272.52.05×239.61.80×
RSS under load (MiB)227.2820.13.61×743.73.27×834.03.67×793.33.49×
Build time (s)1.588.105.12×52.6233.3×8.065.09×52.8433.4×
Conn err (count)00—0—0—0—
Timeouts (count)00—0—0—0—

dotnet-aspnet-ef vs Native Quarkus variants

Overall winner: Quarkus (Native) (metric wins: dotnet-aspnet-ef=3, Quarkus (Native)=4, ties=2; tiebreaker: Throughput)

Efficiency-weighted winner (rps/MiB → TTFR → throughput → RSS load): Quarkus (Native) (metric wins: dotnet-aspnet-ef=1, Quarkus (Native)=3; tiebreaker: rps / MiB)

Metricdotnet-aspnet-efquarkus-nativequarkus-native-mandrel
valueratiovalueratio
Throughput (rps)6,6018,5141.29×8,3541.27×
TTFR (ms)1,426.8103.60.073×99.80.070×
rps / MiB (rps/MiB)29.0629.611.02×28.950.996×
RSS @ startup (MiB)74.889.01.19×89.41.20×
RSS @ 1st req (MiB)133.195.10.714×96.20.723×
RSS under load (MiB)227.2287.61.27×288.61.27×
Build time (s)1.58155.7498.5×152.3496.3×
Conn err (count)00—0—
Timeouts (count)00—0—

Statistical comparisons (Java/Quarkus vs dotnet)

Per-metric pairwise comparisons. Cohen's d sign reflects (Quarkus − dotnet): positive = Quarkus higher, negative = dotnet higher. Welch's two-tailed t-test does not assume equal variances. p < 0.001 · p < 0.05 · not significant.

Throughput (rps) — higher is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm6,60118,65763.12 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden6,60117,07434.04 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native6,6018,51412.96 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native-mandrel6,6018,35410.45 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual6,60117,68750.79 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden6,60116,35817.97 large<0.0001 ✓

TTFR (ms) — lower is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm1,426.82,535.725.48 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden1,426.8985.0-12.70 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native1,426.8103.6-44.62 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native-mandrel1,426.899.8-43.65 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual1,426.82,556.526.31 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden1,426.8859.9-15.94 large<0.0001 ✓

rps / MiB (rps/MiB) — higher is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm29.0622.76-10.80 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden29.0622.98-6.92 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native29.0629.610.79 medium0.1635 ✗
dotnet-aspnet-ef vs quarkus-native-mandrel29.0628.95-0.16 negligible0.7636 ✗
dotnet-aspnet-ef vs quarkus-virtual29.0621.21-14.01 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden29.0620.63-9.03 large<0.0001 ✓

RSS @ startup (MiB) — lower is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm74.8246.669.05 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden74.8240.761.28 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native74.889.0158.10 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native-mandrel74.889.4163.24 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual74.8250.670.41 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden74.8223.630.39 large<0.0001 ✓

RSS @ 1st req (MiB) — lower is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm133.1264.145.25 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden133.1253.232.91 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native133.195.1-254.04 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native-mandrel133.196.2-21.05 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual133.1272.552.06 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden133.1239.618.89 large<0.0001 ✓

RSS under load (MiB) — lower is better

Comparisondotnet meanQuarkus meanCohen's dWelch p
dotnet-aspnet-ef vs quarkus-jvm227.2820.139.69 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-leyden227.2743.734.38 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native227.2287.68.83 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-native-mandrel227.2288.610.00 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual227.2834.049.84 large<0.0001 ✓
dotnet-aspnet-ef vs quarkus-virtual-leyden227.2793.367.20 large<0.0001 ✓

Workload simulation

Closed-form models that translate per-runtime metrics into deployment-archetype outcomes. Inputs come from the measured columns in the per-runtime summary; the math is shown alongside the score so readers can sanity-check it.

Serverless function

Simulated workload: 1,000,000 invocations, cold-start probability 20%. Per-invocation cost = avg latency × RSS-after-first-request (warm) + pcold × TTFR × RSS (cold tax). Result is total billable GB·seconds; lower is better.

RankRuntimeWarm cost
MiB·ms / req		Cold tax
MiB·ms / req		Total
GB·seconds		Ratio vs winner
1quarkus-native-mandrel1,1481,9212,9981.00×
2quarkus-native1,1121,9703,0101.00×
3dotnet-aspnet-ef2,00337,98239,04813.03×
4quarkus-virtual-leyden1,46041,20941,66813.90×
5quarkus-leyden1,47249,87250,14116.73×
6quarkus-jvm1,403133,946132,17744.09×
7quarkus-virtual1,533139,356137,58745.90×

Long-running service

Simulated workload: 10,000 rps for 24 hours, instances pinned to 80% utilization, Service Level Objective (SLO) target latency 100 ms — i.e. the maximum request latency the service commits to. The simulator uses the measured max latency as a proxy: full credit at the SLO, zero credit at 2× the SLO, linear in between. Score = SLO-compliant requests served per GB·hour of cluster memory; higher is better.

RankRuntimeSLO complianceEffective
rps		Instances
required		Cluster RSS
MiB		Score
SLO-rps / GB·hour		Ratio vs winner
1quarkus-leyden100%17,074174484,633,5801.00×
2quarkus-jvm100%18,657182083,868,0190.99×
3quarkus-virtual100%17,687183478,181,4140.92×
4quarkus-virtual-leyden100%16,358179376,011,9780.90×
5quarkus-native100%8,514257554,566,5640.64×
6dotnet-aspnet-ef100%6,601245453,542,7460.63×
7quarkus-native-mandrel100%8,354257753,355,7860.63×

Native closed-world analysis

Output of GraalVM/Mandrel native-image's build-time reachability analysis. Under the closed-world assumption only types/fields/methods reached at build time end up in the binary; anything dynamic (reflection, JNI, resources) must be explicitly registered. The reflection columns show registered count and as a percentage of total reachable.

RuntimeBinary size
MiB		Build RSS
GiB		Classes reachableFields reachableMethods reachableClasses registered
for reflection		Fields registered
for reflection		Methods registered
for reflection
quarkus-native121.84.9827,58737,248135,4819,051 (32.8%)1,816 (4.9%)17,176 (12.7%)
quarkus-native-mandrel122.45.1027,58737,248135,4819,051 (32.8%)1,816 (4.9%)17,176 (12.7%)

Per-runtime summary

Cells show mean ± sample standard deviation across iterations.

RuntimeThroughput
rps		TTFR
ms		rps / MiB
rps/MiB		RSS @ startup
MiB		RSS @ 1st req
MiB		RSS under load
MiB		Build time
s		Conn err
count		Timeouts
count
dotnet-aspnet-ef6,601 ± 175 (CV 2.7%) (n=7)1,426.8 ± 40.1 (CV 2.8%) (n=7)29.06 ± 0.73 (CV 2.5%) (n=7)74.8 ± 0.1 (CV 0.17%) (n=7)133.1 ± 0.2 (CV 0.16%) (n=7)227.2 ± 7.8 (CV 3.4%) (n=7)1.58 ± 0.03 (CV 1.8%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-jvm18,657 ± 206 (CV 1.1%) (n=7)2,535.7 ± 46.7 (CV 1.8%) (n=7)22.76 ± 0.39 (CV 1.7%) (n=7)246.6 ± 3.5 (CV 1.4%) (n=7)264.1 ± 4.1 (CV 1.5%) (n=7)820.1 ± 19.6 (CV 2.4%) (n=7)8.10 ± 0.18 (CV 2.3%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-leyden17,074 ± 398 (CV 2.3%) (n=7)985.0 ± 28.5 (CV 2.9%) (n=7)22.98 ± 1.01 (CV 4.4%) (n=7)240.7 ± 3.8 (CV 1.6%) (n=7)253.2 ± 5.2 (CV 2.0%) (n=7)743.7 ± 19.8 (CV 2.7%) (n=7)52.62 ± 0.33 (CV 0.62%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-native8,514 ± 113 (CV 1.3%) (n=7)103.6 ± 12.4 (CV 12%) (n=7)29.61 ± 0.66 (CV 2.2%) (n=7)89.0 ± 0.0 (CV 0.03%) (n=7)95.1 ± 0.0 (CV 0.03%) (n=7)287.6 ± 5.7 (CV 2.0%) (n=7)155.74 ± 1.95 (CV 1.3%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-native-mandrel8,354 ± 160 (CV 1.9%) (n=7)99.8 ± 15.6 (CV 16%) (n=7)28.95 ± 0.62 (CV 2.2%) (n=7)89.4 ± 0.0 (CV 0.02%) (n=7)96.2 ± 2.5 (CV 2.6%) (n=7)288.6 ± 3.8 (CV 1.3%) (n=7)152.34 ± 1.66 (CV 1.1%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-virtual17,687 ± 254 (CV 1.4%) (n=7)2,556.5 ± 45.6 (CV 1.8%) (n=7)21.21 ± 0.31 (CV 1.5%) (n=7)250.6 ± 3.5 (CV 1.4%) (n=7)272.5 ± 3.8 (CV 1.4%) (n=7)834.0 ± 15.3 (CV 1.8%) (n=7)8.06 ± 0.13 (CV 1.6%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)
quarkus-virtual-leyden16,358 ± 748 (CV 4.6%) (n=7)859.9 ± 30.4 (CV 3.5%) (n=7)20.63 ± 1.10 (CV 5.3%) (n=7)223.6 ± 6.9 (CV 3.1%) (n=7)239.6 ± 8.0 (CV 3.3%) (n=7)793.3 ± 9.0 (CV 1.1%) (n=7)52.84 ± 0.65 (CV 1.2%) (n=7)0 ± 0 (n=7)0 ± 0 (n=7)

Charts

Bar = mean across iterations. Error bar = sample standard deviation. Runtimes that didn't produce a metric are omitted from that chart.

Time to first request
Load test throughput
RSS at startup
RSS during load test
Build time
Per-iteration raw values
RuntimeMetriciter 0iter 1iter 2iter 3iter 4iter 5iter 6
dotnet-aspnet-efThroughput6,4546,8896,4776,4906,5526,8136,534
dotnet-aspnet-efTTFR1,378.81,386.41,440.31,400.01,433.11,464.91,484.3
dotnet-aspnet-efrps / MiB29.6128.5427.9229.4028.7030.0929.18
dotnet-aspnet-efRSS @ startup74.974.774.874.674.974.974.7
dotnet-aspnet-efRSS @ 1st req133.2133.0132.9132.9133.4133.3132.9
dotnet-aspnet-efRSS under load217.9241.4232.0220.8228.3226.4223.9
dotnet-aspnet-efBuild time1.561.621.591.581.531.601.59
dotnet-aspnet-efConn err0000000
dotnet-aspnet-efTimeouts0000000
quarkus-jvmThroughput18,96818,42018,76318,42318,60218,59818,825
quarkus-jvmTTFR2,572.22,507.62,592.02,551.62,563.52,502.62,460.3
quarkus-jvmrps / MiB22.7122.6022.6623.0122.8923.3322.09
quarkus-jvmRSS @ startup248.8247.9246.3252.6244.3242.3243.8
quarkus-jvmRSS @ 1st req263.5262.7265.9269.7259.6268.3259.1
quarkus-jvmRSS under load835.3814.9827.9800.6812.6797.1852.1
quarkus-jvmBuild time8.128.148.348.277.817.938.12
quarkus-jvmConn err0000000
quarkus-jvmTimeouts0000000
quarkus-leydenThroughput16,73116,65817,85316,89917,03517,12917,213
quarkus-leydenTTFR1,009.21,036.7952.0967.3977.9975.9975.8
quarkus-leydenrps / MiB22.2322.3624.5821.6923.3122.8223.85
quarkus-leydenRSS @ startup242.9239.4242.1244.8244.4235.7235.8
quarkus-leydenRSS @ 1st req254.0247.7263.1253.9254.3250.7248.4
quarkus-leydenRSS under load752.6745.1726.2779.1730.7750.6721.6
quarkus-leydenBuild time53.0852.0752.3852.5852.7152.6352.86
quarkus-leydenConn err0000000
quarkus-leydenTimeouts0000000
quarkus-nativeThroughput8,6038,4718,7148,4668,3598,4968,490
quarkus-nativeTTFR92.0116.5121.6105.890.193.5106.0
quarkus-nativerps / MiB30.0329.0430.4928.7130.1129.1529.76
quarkus-nativeRSS @ startup89.089.089.089.089.089.089.0
quarkus-nativeRSS @ 1st req95.095.095.095.195.095.095.1
quarkus-nativeRSS under load286.5291.7285.8294.9277.6291.4285.2
quarkus-nativeBuild time155.12154.43156.91154.68156.58159.17153.32
quarkus-nativeConn err0000000
quarkus-nativeTimeouts0000000
quarkus-native-mandrelThroughput8,4008,0768,4188,5598,3308,4638,231
quarkus-native-mandrelTTFR105.9107.678.7120.879.6107.898.5
quarkus-native-mandrelrps / MiB29.3527.7229.4428.9328.8429.5828.79
quarkus-native-mandrelRSS @ startup89.489.489.489.489.489.489.4
quarkus-native-mandrelRSS @ 1st req101.895.395.395.395.395.395.3
quarkus-native-mandrelRSS under load286.2291.3285.9295.9288.9286.2285.9
quarkus-native-mandrelBuild time152.13151.20149.96152.14152.22155.14153.59
quarkus-native-mandrelConn err0000000
quarkus-native-mandrelTimeouts0000000
quarkus-virtualThroughput17,69317,53417,99317,86917,31217,48417,923
quarkus-virtualTTFR2,561.52,568.52,605.32,527.72,596.12,470.32,566.4
quarkus-virtualrps / MiB20.6321.2221.3521.5621.0621.4721.19
quarkus-virtualRSS @ startup254.8249.4251.5247.1255.2250.2245.9
quarkus-virtualRSS @ 1st req279.5273.9270.9268.3274.8270.3270.2
quarkus-virtualRSS under load857.8826.2842.9828.8821.9814.2845.9
quarkus-virtualBuild time8.048.058.068.148.287.977.86
quarkus-virtualConn err0000000
quarkus-virtualTimeouts0000000
quarkus-virtual-leydenThroughput16,05217,40515,15716,24817,15416,08416,406
quarkus-virtual-leydenTTFR858.3832.4890.3862.7838.1828.8908.9
quarkus-virtual-leydenrps / MiB20.4422.1718.8620.1221.7820.2820.74
quarkus-virtual-leydenRSS @ startup232.8227.7227.6215.1227.4215.7219.0
quarkus-virtual-leydenRSS @ 1st req249.3243.0245.5232.1244.3227.4235.6
quarkus-virtual-leydenRSS under load785.2785.0803.6807.7787.6793.2790.9
quarkus-virtual-leydenBuild time51.6552.7652.9453.6652.8852.5653.44
quarkus-virtual-leydenConn err0000000
quarkus-virtual-leydenTimeouts0000000