Choosing a processor for a build farm - Revision history

Willy: /* New test process */

2024-06-08T14:46:53Z

‎New test process

Willy: /* ROCK 5 ITX - 2024-06-08 */

2024-06-08T13:40:52Z

‎ROCK 5 ITX - 2024-06-08

Willy: update with 2736 Mhz rock5

2024-06-08T13:36:09Z

update with 2736 Mhz rock5

Willy: /* ROCK 5 ITX - 2024-06-08 */

2024-06-08T08:33:20Z

‎ROCK 5 ITX - 2024-06-08

Willy: /* Devices */

2024-06-08T08:13:53Z

‎Devices

Willy: add rock 5 itx

2024-06-08T08:09:26Z

add rock 5 itx

Willy: Update for ROCK 5 ITX

2024-06-08T07:34:07Z

Update for ROCK 5 ITX

Willy: /* Odroid-H3 - 2022-10-30 */

2022-11-21T20:27:53Z

‎Odroid-H3 - 2022-10-30

Willy: /* Odroid-H3 - 2022-10-30 */

2022-11-21T20:27:31Z

‎Odroid-H3 - 2022-10-30

Willy: /* Odroid-H3 - 2022-10-30 */

2022-11-21T20:27:13Z

‎Odroid-H3 - 2022-10-30

← Older revision		Revision as of 14:46, 8 June 2024
Line 774:		Line 774:

	After managing to boot using version 1.11 of the DRAM training code setting it at 2736 MHz, the build is indeed faster, but still does not catch up with ROCK 5B's LPDDR4X.		After managing to boot using version 1.11 of the DRAM training code setting it at 2736 MHz, the build is indeed faster, but still does not catch up with ROCK 5B's LPDDR4X.
		+
		+	== Scalability consideration of the test process ==
		+	On many-core systems there starts to be more and more noise in the measurements, requiring to issue multiple tests and try to get the best value. In addition it's becoming almost impossible to have any reproducible measurement on heterogeneous platforms because a single file sent to a slow code will completely offset the measurements.
		+
		+	A new attempt was made with haproxy-3.0, which is made of significantly more files and seems to yield much more reproducible measurements. In addition, this will better reflect the build performance of modern projects (the version used till now is now 10 years old). A first approach of a procedure would be the following:
		+
		+	cd /dev/shm
		+	curl -L https://github.com/haproxy/haproxy/archive/refs/tags/v3.0.0.tar.gz \| tar zxf -
		+	cd haproxy-3.0.0
		+	make -j$(nproc) clean
		+	make -j$(nproc) haproxy TARGET=linux-glibc EXTRA= LD='#' CC=/path/to/bin/i586-gcc47_glibc218-linux-gnu-gcc CFLAGS=-E
		+	find src -name '*.o' \| xargs cat \| grep -v '^#' \| grep -v '^$' \| wc -l
		+	# 2167430
		+	make -j$(nproc) clean
		+	time taskset -c 4-7 make -j4 TARGET=linux-glibc haproxy EXTRA= LD='#' CC=/path/to/bin/i586-gcc47_glibc218-linux-gnu-gcc
		+	# real 0m24.185s
		+	# user 1m32.539s
		+	# sys 0m3.759s
		+	make -j$(nproc) clean
		+	time make -j8 TARGET=linux-glibc haproxy EXTRA= LD='#' CC=/path/to/bin/i586-gcc47_glibc218-linux-gnu-gcc
		+	# real 0m19.112s
		+	# user 2m24.839s
		+	# sys 0m5.855s
		+
		+	This was run on a ROCK 5 ITX (RK3588 with 4xA76 & 4xA55). Issuing the same command over and over shows roughly 0.15s of variations.

← Older revision		Revision as of 13:40, 8 June 2024
Line 772:		Line 772:
	=== ROCK 5 ITX - 2024-06-08 ===		=== ROCK 5 ITX - 2024-06-08 ===
	The board is designed with excellent I/O distribution (2 PCIe Gen3 lanes for M.2, 2 for the 4 SATA3 ports), and comes equipped with 8 GB of LPDDR5. Apparently the initial goal was to have it run at 2736 MHz which was the case till rkbin-1.11 but images started to ship with an updated version that reduces this to the significantly lower 2400 MHz. It looks like this frequency is too low to cope with the LPDDR4X in the ROCK 5B running at only 2112 MHz. This is visible on ramlat tests showing roughly 7% higher DRAM access times, and here on the build times that are systematically around 11% higher than on the ROCK 5B. Some commit messages seem to mention stability as the cause for lowering the DRAM frequency. It's not clear why intermediary frequencies were not tried; a rough calculation shows that anything around 2600 MHz should recover the lost performance. More investigation is needed on this front. Regardless, the board is capable of reading from disks at 10 Gbps and feeding that over TCP on a 10GbE adapter connected to M.2, and given its rich connectivity that makes it look more like a server, it's possible that the focus has been set on maximal stability before performance.		The board is designed with excellent I/O distribution (2 PCIe Gen3 lanes for M.2, 2 for the 4 SATA3 ports), and comes equipped with 8 GB of LPDDR5. Apparently the initial goal was to have it run at 2736 MHz which was the case till rkbin-1.11 but images started to ship with an updated version that reduces this to the significantly lower 2400 MHz. It looks like this frequency is too low to cope with the LPDDR4X in the ROCK 5B running at only 2112 MHz. This is visible on ramlat tests showing roughly 7% higher DRAM access times, and here on the build times that are systematically around 11% higher than on the ROCK 5B. Some commit messages seem to mention stability as the cause for lowering the DRAM frequency. It's not clear why intermediary frequencies were not tried; a rough calculation shows that anything around 2600 MHz should recover the lost performance. More investigation is needed on this front. Regardless, the board is capable of reading from disks at 10 Gbps and feeding that over TCP on a 10GbE adapter connected to M.2, and given its rich connectivity that makes it look more like a server, it's possible that the focus has been set on maximal stability before performance.
		+
		+	After managing to boot using version 1.11 of the DRAM training code setting it at 2736 MHz, the build is indeed faster, but still does not catch up with ROCK 5B's LPDDR4X.

@@ Line 433: / Line 433: @@
 |-
 | 2024/06/08 || ROCK 5 ITX || 4 || 6.70 || 68488 || 7134 || 4*A76@2.4 GHz - aarch64
 |}

← Older revision		Revision as of 08:33, 8 June 2024
Line 765:		Line 765:

	In any case, creating build farms from such a device is way more interesting than using old generations of ARM or x86 cores, as the build time was cut in half compared to the then performant mcbin and nanopi-m4, and it's 3 times faster than an RPi-4B.		In any case, creating build farms from such a device is way more interesting than using old generations of ARM or x86 cores, as the build time was cut in half compared to the then performant mcbin and nanopi-m4, and it's 3 times faster than an RPi-4B.
		+
		+	=== ROCK 5 ITX - 2024-06-08 ===
		+	The board is designed with excellent I/O distribution (2 PCIe Gen3 lanes for M.2, 2 for the 4 SATA3 ports), and comes equipped with 8 GB of LPDDR5. Apparently the initial goal was to have it run at 2736 MHz which was the case till rkbin-1.11 but images started to ship with an updated version that reduces this to the significantly lower 2400 MHz. It looks like this frequency is too low to cope with the LPDDR4X in the ROCK 5B running at only 2112 MHz. This is visible on ramlat tests showing roughly 7% higher DRAM access times, and here on the build times that are systematically around 11% higher than on the ROCK 5B. Some commit messages seem to mention stability as the cause for lowering the DRAM frequency. It's not clear why intermediary frequencies were not tried; a rough calculation shows that anything around 2600 MHz should recover the lost performance. More investigation is needed on this front. Regardless, the board is capable of reading from disks at 10 Gbps and feeding that over TCP on a 10GbE adapter connected to M.2, and given its rich connectivity that makes it look more like a server, it's possible that the focus has been set on maximal stability before performance.

@@ Line 113: / Line 113: @@
 | 2018/10/27 || Raspberry-Pi 3B+ || armv8/cortex A53 || BCM2837B0 || 1.37 GHz || 4 || 4 || 1 GB LPDDR2 || 32 || 800
 |-
-| 2018/10/27 || ROC-RK3328-CC || armv8/cortex A53 || RK3328 || 1.4 GHz || 4 || 4 || 2 GB LPDDR4 || 32 || 2133
+| 2018/10/27 || ROC-RK3328-CC || armv8/cortex A53 || RK3328 || 1.4 GHz || 4 || 4 || 2 GB LPDDR4 || 32 || 800
 |-
-| 2018/11/04 || Odroid-H2 || x86_64 || Celeron-J4105 || 2.4-2.5 GHz || 4 || 4 || 8 GB LPDDR4 || 64 || 2400
+| 2018/11/04 || Odroid-H2 || x86_64 || Celeron-J4105 || 2.4-2.5 GHz || 4 || 4 || 8 GB LPDDR4 || 64 || 1200
 |-
-| 2019/02/22 || Odroid-N2 || armv8/cortex A73+A53 || Amlogic S922X || A73:1.800 GHz A53:1.896 GHz || 4*A73+2*A53 || 6 || 4 GB LPDDR4 || 32 || 2666
+| 2019/02/22 || Odroid-N2 || armv8/cortex A73+A53 || Amlogic S922X || A73:1.800 GHz A53:1.896 GHz || 4*A73+2*A53 || 6 || 4 GB LPDDR4 || 32 || 1333
 |-
 | 2019/09/17 || Raspberry-Pi 4B || armv8/cortex A72 || BCM2711 || 1.5/2.0 GHz || 4 || 4 || 1 GB LPDDR4 || 32 || 1200

@@ Line 107: / Line 107: @@
 | 2018/06/07 || OrangePI One Plus || armv8/cortex A53 || Allwinner H6 || 1.8/1.8 GHz || 4 || 4 || 1 GB DDR3 || 32 || ?
 |-
-| 2018/06/09 || Marvell MACCHIATObin || armv8/cortex A72 || Armada 8040 || 2.0/2.0 GHz || 4 || 4 || 4 GB DDR4 || 64 || 2400
+| 2018/06/09 || Marvell MACCHIATObin || armv8/cortex A72 || Armada 8040 || 2.0/2.0 GHz || 4 || 4 || 4 GB DDR4 || 64 || 1200
 |-
-| 2018/09/08 || NanoPI-M4 || armv8/cortex A72+A53 || RK3399 || A72:1.8 GHz A53:1.416 GHz || 2*A72+4*A53 || 6 || 2 GB DDR3 || 64 || 1600
+| 2018/09/08 || NanoPI-M4 || armv8/cortex A72+A53 || RK3399 || A72:1.8 GHz A53:1.416 GHz || 2*A72+4*A53 || 6 || 2 GB DDR3 || 64 || 800
 |-
 | 2018/10/27 || Raspberry-Pi 3B+ || armv8/cortex A53 || BCM2837B0 || 1.37 GHz || 4 || 4 || 1 GB LPDDR2 || 32 || 800
@@ Line 123: / Line 123: @@
 | 2021/04/04 || Khadas VIM3L || armv8.2/cortex A55 || Amlogic S905D3 || 1.9/2.2 GHz || 4 || 4 || 2 GB LPDDR4 || 32 || 1608
 |-
-| 2022/06/06 || Odroid-N2+ || armv8/cortex A73+A53 || Amlogic S922X || A73:2.400 GHz A53:2.016 GHz || 4*A73+2*A53 || 6 || 4 GB LPDDR4 || 32 || 2666
+| 2022/06/06 || Odroid-N2+ || armv8/cortex A73+A53 || Amlogic S922X || A73:2.400 GHz A53:2.016 GHz || 4*A73+2*A53 || 6 || 4 GB LPDDR4 || 32 || 1333
 |-
-| 2022/07/04 || ROCK 5B || armv8/cortex A76+A55 || RK3588 || A76:2.304 GHz A55:1.800 GHz || 4*A76+4*A55 || 8 || 4 GB LPDDR4X || 64 || 3200
+| 2022/07/04 || ROCK 5B || armv8/cortex A76+A55 || RK3588 || A76:2.304 GHz A55:1.800 GHz || 4*A76+4*A55 || 8 || 4 GB LPDDR4X || 64 || 2112
 |-
-| 2022/10/30 || Odroid-H3 || x86_64 || Celeron-N5105 || 2.0-2.9 GHz || 4 || 4 || 4 GB DDR4 || 64 || 2933
+| 2022/10/30 || Odroid-H3 || x86_64 || Celeron-N5105 || 2.0-2.9 GHz || 4 || 4 || 4 GB DDR4 || 64 || 1466
 |}

@@ Line 423: / Line 423: @@
 |-
 | 2022/11/21 || Odroid-H3 || 4 || 6.39 || 71810 || 6411 || 4 cores at 2.8 GHz (PL4=0), 2 DRAM sticks
 |}

@@ Line 750: / Line 750: @@
 === Odroid-H3 - 2022-10-30 ===
-This is the low-end of this new fanless board from Hardkernel. It uses a Celeron N5105 limited to 2.9 GHz (2.4 with all cores on). Its big sister uses a Pentium N6005 at 3.1 GHz. Also the board involved in this test only had one memory stick while the CPU has two DDR4 memory channels, and we know how much memory performance is important for build processes. Despite this, the board is both faster and more efficient per core than the good old PhenomX2 that was melting the PSU's wires. It's even faster and more efficient than a 3rd gen core i5, but it doesn't beat the Cortex-A76 in the Rock-5B. It looks like we've reached a new generation of very efficient chips that are capable of delivering quite decent performance without heating too much. In the past this was limited to very low performance Atoms, but nowadays the chips deliver efficiencies that are closer to what Skylakes can do. The combination of moderate frequencies (2.4 GHz) and 10nm process node definitely helps. And of course, the memory speed significantly helps. The last test with a second memory stick shows a ~3% performance improve thanks to the use of the second memory channel.
+This is the low-end of this new fanless board from Hardkernel. It uses a Celeron N5105 limited to 2.9 GHz (2.4 with all cores on). Its big sister uses a Pentium N6005 at 3.1 GHz. Also the board involved in this test only had one memory stick while the CPU has two DDR4 memory channels, and we know how much memory performance is important for build processes. Despite this, the board is both faster and more efficient per core than the good old PhenomX2 that was melting the PSU's wires. It's even faster and more efficient than a 3rd gen core i5, but it doesn't beat the Cortex-A76 in the Rock-5B. It looks like we've reached a new generation of very efficient chips that are capable of delivering quite decent performance without heating too much. In the past this was limited to very low performance Atoms, but nowadays the chips deliver efficiencies that are closer to what Skylakes can do. The combination of moderate frequencies (2.4 GHz) and 10nm process node definitely helps. And of course, the memory speed significantly helps. The last test with a second memory stick shows ~3% performance improvement thanks to the use of the second memory channel.
 By setting the PL4 time limit to 0 in the BIOS as documented on the HardKernel wiki, the 4 cores manage to stay at 2.8 GHz during the whole build, resulting in even shorter build times, with the machine getting closer to the Radxa Rock 5B.
 In any case, creating build farms from such a device is way more interesting than using old generations of ARM or x86 cores, as the build time was cut in half compared to the then  performant mcbin and nanopi-m4, and it's 3 times faster than an RPi-4B.