sviko.com

Review of ASRock Rack EPYCD8-2T - server motherboard for AMD EPYC 7000

AMD EPYC 7000 platform is good because a large number of PCI Express channels are available in 1-socket format. To build a server on several GPUs, this is a serious advantage, because 1-processor layout allows you to assemble compact machines in the ATX form factor.


This motherboard from the company ASRock Rack is designed for this application, and unlike analogues, this model already has a pair of 10-Gigabit network ports on the bot, that is, you do not need to spend money on interconnects.

Memory and expansion slots

The platform has 8 DIMM slots so you can enjoy the performance of an 8-channel CPU memory controller, but that’s not the main plus of the EPYCD8-2T. Here you have 4 PCI Express 3.0 16x slots, plus 3 PCI Express 3.0 8x slots and 2 NVME M. 2 slots. The standard ATX layout allows you to use any number of expansion slots without fear of the graphics card or any expansion card bumping into memory slots or cables.


Of course, expansion slots are not quite standard: the top slot is closer to the memory slot than is usually allowed, so in some cases you may be uncomfortable to open the locking latch, if the nearest DIMM is occupied. Moreover, this is not the only difficulty of the assembly.

For example, in front of the fifth PCI Express slot there are three slots for connecting system fans, and a long video card will easily block them.

Cooling system

There is no, of course on this motherboard there is no shortage of connectors for fans: just their 7 (1 CPU / 4 Front / 2 Rear), of which 4 are located near VRM processor, next to OCULink ports. And SATA ports are deployed parallel to the Board, so as not to interfere with your graphics cards.

Each fan connector has a breakage led indication to make it easier for you to see which connector to disconnect if the sensor has fixed the impeller stop. For fans, automatic speed adjustment is available depending on the temperature of the processor. In total, the motherboard monitors three temperature indicators: CPU, Motherboard, MB Card Side and TR1 remote sensor (not included). Of course, the monitoring system can read temperature readings from the controllers of RDIMM modules.

Storage subsystem

In terms of storage system in ASRock Rack EPYCD8-2T - full abundance. First, it is provided with 8 SATA ports via 2 x Mini-SAS for the storage of cold data and data-type data-at-rest. There is also a ninth SATA port for SATA-DOM flash drive, which can be installed OSes. Note that RAID functions are not declared, so you will have to combine disks into arrays by software.

Secondly, for PCIE SSD you are given two OCULink ports, which are likely to remain out of business, because there are two M. 2 ports with support for disks up to 22110. Please note that the use of PCI Express SSD does not affect the bus width of PCI Express expansion slots: there are 128 PCI Express 3.0 lines in EPYC 7000 processors.

Network and peripherals

We consider the top version of this motherboard, which has 2 ports 10G Base-T based network controller Intel X550 (read our comparison of Aquantia AQC107 vs Intel X550-T2). Yes, ASRock Rack EPYCD8-2T - this is the case when meeting on the same Board, Intel and AMD do not conflict, but complement each other.


The network controller is cooled by some excessively large radiator, clearly designed for the fact that in normal life it will be closed from above by several hot video cards. Having a fast network interface is a huge plus for this motherboard, because you just do not spend money and PCI slot for an additional controller.

However, the EPYCD8-2T is a server board, with no hint of use in workstations, so there are only two USB 3.0 ports (plus another pad for connecting two front USB 3.0 ports), no audio path and no frills like bluetooth. Oh, what a pity, because the company does not have a similar board for graphic stations.

The power supply circuit

The EPYCD8-2T motherboard has an 8-phase processor power supply circuit with filters based on solid-state capacitors. Here, power filtering is installed even on PCI Express slots, although not every vendor shows such excessive care about power. But in general, only 2 complex chips are soldered on the motherboard: a network controller and a remote control processor. In the absence of the south bridge is less loaded in terms of temperature and power, and simply put, there is nothing to break.

Let’s calculate how PCI Express 3.0 channels are distributed in this Board:

  • PCI Express x16 slots: 64 lanes
  • PCI Express x8 slots: 24 lanes
  • M.2 PCIe 3.0 x4 slots: 8 lanes
  • Intel X550 network card: 8 lanes
  • 8 SATA ports: 8 lanes
  • 2 x4 Oculink ports: 2 lanes
  • Aspeed AST2500 chip: 1 lane
    • Total: 115 Lanes


As you can see, the stock on the PCI Express lines is still there, because of the 128 possible lines, we counted only 115, so that’s why people choose AMD EPYC: you can use all PCI Express devices at full speed.

Remote Management and BIOS

The ASPeed AST2500 chip with a dedicated 1-Gigabit network port is used For remote management, which is used only for BMC functions and nothing more. I would like to say that AST2500 is the best solution today, which has a modern HTML-5 interface optimized for tablets and smartphones. Fast and convenient console management, connection of a remote disk for installation of an operating system both from the computer, and through NAS is available to you. There is a very convenient temperature monitoring and the ability to record POST-tests.

Compared to Dell iDRAC, HPE iLO and Lenovo XClarity Control systems, here you do not pay for remote control, but get almost the same features, not to mention the fact that you can simply forget about IP-KVM as a nightmare.

The motherboard has an AMI BIOS with a standard set of settings. Among the sensors should be noted the temperature of the network card Intel X550, and among the settings of the periphery - the bandwidth limit PCI Express slots.

Testing

First of all, let’s check the speed with which Virtual Media image is mounted, because this is a very important point when deploying a software environment in the data center.
En_test_virtual_media
The performance here is on par with other motherboards with the chip AST2500 on the Intel platform. Let’s look at the speed of the network connection:
En_iscsi_latency En_iscsi_throughput
In general, Intel X550 is a very interesting network card, about which you need to say the following. First, unlike Intel X540 and Intel X557 (used in Xeon-D), this chip has no compatibility problems with the old 1G equipment, especially firms Mikrotik and Ubiquiti, that is, with the above two chips, the old switches may not work, and with Intel X550 - always work. Of course, you can say that it makes no sense to connect a 10-Gigabit card to 1-Gigabit ports, but I will not agree with you, because everything happens in the work. Secondly, the Intel X550 inherently supports intermediate speeds of 2.5 GB / s and 5 GB / s when using a standard category 5E cable, but the implementation of these standards depends already on the device in which this chip is used. For the server market, where switching is carried out between the racks, intermediate speeds are of no value, because inside the rack 10G is quite a cable 5E, and between the racks it is easy to stretch CAT6/7. In our test for sport I tried to get intermediate speeds, but either 1 Gbit/s or 10 Gbit/s came out. Well, as they say, no one else promised: the main thing is that 10 Gbit/c work as well as on a discrete Intel X550-T2.

How to buy this board via a tender

To the tender you put it ASRock Rack EPYCD8-2T, specify a 10-Gigabit network with ports 10GBase-T, the presence of ports Oculink and M. 2 in a single-platform under Epyc 7000. She is such one, and no substitutes will not bring you.

Conclusions

If you want to build a server with 4 GPU in ATX format, then here’s a great solution. You do not have to buy any high-speed network interface or controller for SATA drives - ASRock Rack EPYCD8-2T already has on Board everything you need, plus a smart remote control system with iKVM, which you get for free. Comparing with analogs, you wonder what the engineers ASRock Rack not to limit your imagination and put on charge even two Oculink port and two more claimed - M.2.

As such, the motherboard has no shortcomings, it is a great harvester, which lacks only the sound part for use in workstations, but if your goal is to build a server for calculations on the GPU, the EPYCD8-2T is a platform that can be safely put in the project, the rare case when we have a board without flaws.