December 2019
Anomalies
One interesting anomaly I found is a large drop in performance of the tests under JDK 13 and 14 compared to the same tests under JDK 11 and 12.
ARM Cortex-A9
Ubuntu 20.04
copyTo16New/Old
PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| UbuntuJDK11New | large | thrpt | 25 | 13.686 | ± | 0.011 | ops/s |
| UbuntuJDK11Old | large | thrpt | 25 | 10.084 | ± | 0.085 | ops/s |
| AdoptJDK11New | large | thrpt | 25 | 13.659 | ± | 0.013 | ops/s |
| AdoptJDK11Old | large | thrpt | 25 | 10.122 | ± | 0.089 | ops/s |
| AdoptJDK12New | large | thrpt | 25 | 13.673 | ± | 0.012 | ops/s |
| AdoptJDK12Old | large | thrpt | 25 | 10.010 | ± | 0.026 | ops/s |
| UbuntuJDK13New | large | thrpt | 25 | 4.034 | ± | 0.005 | ops/s |
| UbuntuJDK13Old | large | thrpt | 25 | 3.386 | ± | 0.023 | ops/s |
| AdoptJDK13New | large | thrpt | 25 | 4.034 | ± | 0.004 | ops/s |
| AdoptJDK13Old | large | thrpt | 25 | 3.368 | ± | 0.031 | ops/s |
| UbuntuJDK14New | large | thrpt | 25 | 4.033 | ± | 0.004 | ops/s |
| UbuntuJDK14Old | large | thrpt | 25 | 3.596 | ± | 0.226 | ops/s |
QEMU ARM VM
Ubuntu 20.04
copyTo16New/Old
PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| UbuntuJDK11New | large | thrpt | 25 | 18.020 | ± | 0.244 | ops/s |
| UbuntuJDK11Old | large | thrpt | 25 | 12.939 | ± | 2.982 | ops/s |
| AdoptJDK11New | large | thrpt | 25 | 14.737 | ± | 3.608 | ops/s |
| AdoptJDK11Old | large | thrpt | 25 | 11.844 | ± | 2.490 | ops/s |
| AdoptJDK12New | large | thrpt | 25 | 17.904 | ± | 0.845 | ops/s |
| AdoptJDK12Old | large | thrpt | 25 | 11.286 | ± | 2.892 | ops/s |
| UbuntuJDK13New | large | thrpt | 25 | 4.226 | ± | 1.075 | ops/s |
| UbuntuJDK13Old | large | thrpt | 25 | 3.632 | ± | 0.857 | ops/s |
| AdoptJDK13New | large | thrpt | 25 | 5.393 | ± | 0.902 | ops/s |
| AdoptJDK13Old | large | thrpt | 25 | 4.152 | ± | 0.718 | ops/s |
| UbuntuJDK14New | large | thrpt | 25 | 5.991 | ± | 0.168 | ops/s |
| UbuntuJDK14Old | large | thrpt | 25 | 4.190 | ± | 0.787 | ops/s |
Benchmarks
The results of the benchmark tests are grouped by the processor, operating system, and build of the Java Development Kit (JDK) on which they ran.
I ran the following command on the ARM Cortex-A8 processor:
$ java -jar target/benchmarks.jar -p type=small
I ran the following command on the ARM Cortex-A9 processor and QEMU ARM virtual machine:
$ java -jar target/benchmarks.jar -p type=large
ARM Cortex-A8
Ubuntu 14.04
AdoptOpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | small | thrpt | 25 | 2.662 | ± | 0.083 | ops/s |
| copyTo16New | small | thrpt | 25 | 4.365 | ± | 0.006 | ops/s |
| copyTo16Old | small | thrpt | 25 | 3.768 | ± | 0.006 | ops/s |
| copyTo32New | small | thrpt | 25 | 38.170 | ± | 0.463 | ops/s |
| copyTo32Old | small | thrpt | 25 | 38.058 | ± | 0.345 | ops/s |
| writeTo08New | small | thrpt | 25 | 2.557 | ± | 0.215 | ops/s |
| writeTo16New | small | thrpt | 25 | 6.896 | ± | 0.011 | ops/s |
| writeTo16Old | small | thrpt | 25 | 3.869 | ± | 0.004 | ops/s |
| writeTo32New | small | thrpt | 25 | 45.831 | ± | 0.171 | ops/s |
| writeTo32Old | small | thrpt | 25 | 45.685 | ± | 0.227 | ops/s |
ARM Cortex-A9
Ubuntu 18.04
Ubuntu OpenJDK 11
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 9.940 | ± | 0.011 | ops/s |
| copyTo16New | large | thrpt | 25 | 13.683 | ± | 0.013 | ops/s |
| copyTo16Old | large | thrpt | 25 | 10.129 | ± | 0.086 | ops/s |
| copyTo32New | large | thrpt | 25 | 107.174 | ± | 10.068 | ops/s |
| copyTo32Old | large | thrpt | 25 | 105.480 | ± | 8.603 | ops/s |
| writeTo08New | large | thrpt | 25 | 8.947 | ± | 0.355 | ops/s |
| writeTo16New | large | thrpt | 25 | 13.983 | ± | 0.021 | ops/s |
| writeTo16Old | large | thrpt | 25 | 9.950 | ± | 0.093 | ops/s |
| writeTo32New | large | thrpt | 25 | 95.590 | ± | 0.164 | ops/s |
| writeTo32Old | large | thrpt | 25 | 94.621 | ± | 0.667 | ops/s |
AdoptOpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 3.127 | ± | 0.136 | ops/s |
| copyTo16New | large | thrpt | 25 | 4.033 | ± | 0.004 | ops/s |
| copyTo16Old | large | thrpt | 25 | 3.379 | ± | 0.029 | ops/s |
| copyTo32New | large | thrpt | 25 | 34.385 | ± | 1.633 | ops/s |
| copyTo32Old | large | thrpt | 25 | 32.060 | ± | 0.936 | ops/s |
| writeTo08New | large | thrpt | 25 | 2.942 | ± | 0.108 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.268 | ± | 0.005 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.226 | ± | 0.095 | ops/s |
| writeTo32New | large | thrpt | 25 | 29.513 | ± | 0.125 | ops/s |
| writeTo32Old | large | thrpt | 25 | 29.522 | ± | 0.118 | ops/s |
Ubuntu 20.04
Ubuntu OpenJDK 11
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 9.942 | ± | 0.008 | ops/s |
| copyTo16New | large | thrpt | 25 | 13.686 | ± | 0.011 | ops/s |
| copyTo16Old | large | thrpt | 25 | 10.084 | ± | 0.085 | ops/s |
| copyTo32New | large | thrpt | 25 | 100.353 | ± | 3.295 | ops/s |
| copyTo32Old | large | thrpt | 25 | 101.164 | ± | 2.290 | ops/s |
| writeTo08New | large | thrpt | 25 | 8.725 | ± | 0.006 | ops/s |
| writeTo16New | large | thrpt | 25 | 13.968 | ± | 0.016 | ops/s |
| writeTo16Old | large | thrpt | 25 | 9.834 | ± | 0.064 | ops/s |
| writeTo32New | large | thrpt | 25 | 95.030 | ± | 0.296 | ops/s |
| writeTo32Old | large | thrpt | 25 | 95.355 | ± | 0.643 | ops/s |
AdoptOpenJDK 11
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 9.927 | ± | 0.008 | ops/s |
| copyTo16New | large | thrpt | 25 | 13.659 | ± | 0.013 | ops/s |
| copyTo16Old | large | thrpt | 25 | 10.122 | ± | 0.089 | ops/s |
| copyTo32New | large | thrpt | 25 | 100.754 | ± | 3.756 | ops/s |
| copyTo32Old | large | thrpt | 25 | 108.632 | ± | 10.148 | ops/s |
| writeTo08New | large | thrpt | 25 | 8.712 | ± | 0.008 | ops/s |
| writeTo16New | large | thrpt | 25 | 13.973 | ± | 0.022 | ops/s |
| writeTo16Old | large | thrpt | 25 | 9.798 | ± | 0.011 | ops/s |
| writeTo32New | large | thrpt | 25 | 95.209 | ± | 0.598 | ops/s |
| writeTo32Old | large | thrpt | 25 | 95.742 | ± | 0.658 | ops/s |
AdoptOpenJDK 12
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 9.571 | ± | 0.558 | ops/s |
| copyTo16New | large | thrpt | 25 | 13.673 | ± | 0.012 | ops/s |
| copyTo16Old | large | thrpt | 25 | 10.010 | ± | 0.026 | ops/s |
| copyTo32New | large | thrpt | 25 | 100.413 | ± | 7.340 | ops/s |
| copyTo32Old | large | thrpt | 25 | 103.118 | ± | 8.428 | ops/s |
| writeTo08New | large | thrpt | 25 | 8.624 | ± | 0.008 | ops/s |
| writeTo16New | large | thrpt | 25 | 14.259 | ± | 0.017 | ops/s |
| writeTo16Old | large | thrpt | 25 | 9.944 | ± | 0.063 | ops/s |
| writeTo32New | large | thrpt | 25 | 88.766 | ± | 0.405 | ops/s |
| writeTo32Old | large | thrpt | 25 | 89.101 | ± | 0.582 | ops/s |
Ubuntu OpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 3.127 | ± | 0.137 | ops/s |
| copyTo16New | large | thrpt | 25 | 4.034 | ± | 0.005 | ops/s |
| copyTo16Old | large | thrpt | 25 | 3.386 | ± | 0.023 | ops/s |
| copyTo32New | large | thrpt | 25 | 32.735 | ± | 1.214 | ops/s |
| copyTo32Old | large | thrpt | 25 | 34.853 | ± | 2.217 | ops/s |
| writeTo08New | large | thrpt | 25 | 2.878 | ± | 0.003 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.277 | ± | 0.003 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.320 | ± | 0.021 | ops/s |
| writeTo32New | large | thrpt | 25 | 29.675 | ± | 0.136 | ops/s |
| writeTo32Old | large | thrpt | 25 | 29.808 | ± | 0.259 | ops/s |
AdoptOpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 3.214 | ± | 0.004 | ops/s |
| copyTo16New | large | thrpt | 25 | 4.034 | ± | 0.004 | ops/s |
| copyTo16Old | large | thrpt | 25 | 3.368 | ± | 0.031 | ops/s |
| copyTo32New | large | thrpt | 25 | 33.297 | ± | 2.081 | ops/s |
| copyTo32Old | large | thrpt | 25 | 29.669 | ± | 0.739 | ops/s |
| writeTo08New | large | thrpt | 25 | 3.019 | ± | 0.133 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.265 | ± | 0.005 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.167 | ± | 0.083 | ops/s |
| writeTo32New | large | thrpt | 25 | 29.577 | ± | 0.105 | ops/s |
| writeTo32Old | large | thrpt | 25 | 29.555 | ± | 0.179 | ops/s |
Ubuntu OpenJDK 14
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 2.951 | ± | 0.170 | ops/s |
| copyTo16New | large | thrpt | 25 | 4.033 | ± | 0.004 | ops/s |
| copyTo16Old | large | thrpt | 25 | 3.596 | ± | 0.226 | ops/s |
| copyTo32New | large | thrpt | 25 | 46.822 | ± | 2.818 | ops/s |
| copyTo32Old | large | thrpt | 25 | 48.062 | ± | 0.541 | ops/s |
| writeTo08New | large | thrpt | 25 | 3.093 | ± | 0.134 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.279 | ± | 0.004 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.384 | ± | 0.169 | ops/s |
| writeTo32New | large | thrpt | 25 | 29.713 | ± | 0.170 | ops/s |
| writeTo32Old | large | thrpt | 25 | 29.722 | ± | 0.174 | ops/s |
QEMU ARM VM
Ubuntu 20.04
Ubuntu OpenJDK 11
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 5.127 | ± | 0.564 | ops/s |
| copyTo16New | large | thrpt | 25 | 18.020 | ± | 0.244 | ops/s |
| copyTo16Old | large | thrpt | 25 | 12.939 | ± | 2.982 | ops/s |
| copyTo32New | large | thrpt | 25 | 137.360 | ± | 3.631 | ops/s |
| copyTo32Old | large | thrpt | 25 | 138.173 | ± | 3.022 | ops/s |
| writeTo08New | large | thrpt | 25 | 5.973 | ± | 0.020 | ops/s |
| writeTo16New | large | thrpt | 25 | 14.448 | ± | 2.651 | ops/s |
| writeTo16Old | large | thrpt | 25 | 13.042 | ± | 2.160 | ops/s |
| writeTo32New | large | thrpt | 25 | 79.773 | ± | 1.038 | ops/s |
| writeTo32Old | large | thrpt | 25 | 85.746 | ± | 1.941 | ops/s |
AdoptOpenJDK 11
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 5.602 | ± | 0.492 | ops/s |
| copyTo16New | large | thrpt | 25 | 14.737 | ± | 3.608 | ops/s |
| copyTo16Old | large | thrpt | 25 | 11.844 | ± | 2.490 | ops/s |
| copyTo32New | large | thrpt | 25 | 140.446 | ± | 3.476 | ops/s |
| copyTo32Old | large | thrpt | 25 | 138.769 | ± | 2.951 | ops/s |
| writeTo08New | large | thrpt | 25 | 5.992 | ± | 0.031 | ops/s |
| writeTo16New | large | thrpt | 25 | 13.959 | ± | 2.540 | ops/s |
| writeTo16Old | large | thrpt | 25 | 8.846 | ± | 1.553 | ops/s |
| writeTo32New | large | thrpt | 25 | 83.199 | ± | 1.738 | ops/s |
| writeTo32Old | large | thrpt | 25 | 81.432 | ± | 1.383 | ops/s |
AdoptOpenJDK 12
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 6.062 | ± | 0.029 | ops/s |
| copyTo16New | large | thrpt | 25 | 17.904 | ± | 0.845 | ops/s |
| copyTo16Old | large | thrpt | 25 | 11.286 | ± | 2.892 | ops/s |
| copyTo32New | large | thrpt | 25 | 139.635 | ± | 3.768 | ops/s |
| copyTo32Old | large | thrpt | 25 | 138.231 | ± | 2.540 | ops/s |
| writeTo08New | large | thrpt | 25 | 5.845 | ± | 0.069 | ops/s |
| writeTo16New | large | thrpt | 25 | 15.880 | ± | 0.200 | ops/s |
| writeTo16Old | large | thrpt | 25 | 9.377 | ± | 3.078 | ops/s |
| writeTo32New | large | thrpt | 25 | 68.567 | ± | 9.545 | ops/s |
| writeTo32Old | large | thrpt | 25 | 69.847 | ± | 3.513 | ops/s |
Ubuntu OpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 1.899 | ± | 0.154 | ops/s |
| copyTo16New | large | thrpt | 25 | 4.226 | ± | 1.075 | ops/s |
| copyTo16Old | large | thrpt | 25 | 3.632 | ± | 0.857 | ops/s |
| copyTo32New | large | thrpt | 25 | 42.608 | ± | 2.769 | ops/s |
| copyTo32Old | large | thrpt | 25 | 44.853 | ± | 0.635 | ops/s |
| writeTo08New | large | thrpt | 25 | 1.701 | ± | 0.232 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.307 | ± | 0.906 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.343 | ± | 0.865 | ops/s |
| writeTo32New | large | thrpt | 25 | 22.757 | ± | 0.439 | ops/s |
| writeTo32Old | large | thrpt | 25 | 23.865 | ± | 0.536 | ops/s |
AdoptOpenJDK 13
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 1.774 | ± | 0.187 | ops/s |
| copyTo16New | large | thrpt | 25 | 5.393 | ± | 0.902 | ops/s |
| copyTo16Old | large | thrpt | 25 | 4.152 | ± | 0.718 | ops/s |
| copyTo32New | large | thrpt | 25 | 45.505 | ± | 0.554 | ops/s |
| copyTo32Old | large | thrpt | 25 | 45.239 | ± | 0.676 | ops/s |
| writeTo08New | large | thrpt | 25 | 1.923 | ± | 0.008 | ops/s |
| writeTo16New | large | thrpt | 25 | 4.307 | ± | 0.913 | ops/s |
| writeTo16Old | large | thrpt | 25 | 3.855 | ± | 0.732 | ops/s |
| writeTo32New | large | thrpt | 25 | 22.746 | ± | 0.887 | ops/s |
| writeTo32Old | large | thrpt | 25 | 23.544 | ± | 0.956 | ops/s |
Ubuntu OpenJDK 14
Log File • PNG Chart • SVG Chart • PDF Chart
| Benchmark | (type) | Mode | Cnt | Score | ± | Error | Units |
|---|---|---|---|---|---|---|---|
| copyTo08New | large | thrpt | 25 | 1.974 | ± | 0.015 | ops/s |
| copyTo16New | large | thrpt | 25 | 5.991 | ± | 0.168 | ops/s |
| copyTo16Old | large | thrpt | 25 | 4.190 | ± | 0.787 | ops/s |
| copyTo32New | large | thrpt | 25 | 45.315 | ± | 0.935 | ops/s |
| copyTo32Old | large | thrpt | 25 | 45.415 | ± | 0.587 | ops/s |
| writeTo08New | large | thrpt | 25 | 1.846 | ± | 0.153 | ops/s |
| writeTo16New | large | thrpt | 25 | 5.174 | ± | 0.108 | ops/s |
| writeTo16Old | large | thrpt | 25 | 2.474 | ± | 0.361 | ops/s |
| writeTo32New | large | thrpt | 25 | 23.307 | ± | 0.307 | ops/s |
| writeTo32Old | large | thrpt | 25 | 23.507 | ± | 0.421 | ops/s |