sviko.com

QSAN XN8012R review: virtualization + ZFS onboard

Today, many technical experts believe that ZFS is something that works only under FreeBSD, in clusters, with configuration from the command line, and mere mortals there is better not to meddle, because it is the lot of the elect. In fact, it is not, and even the ZFS on Linux port has already reached the level of stability when it is used in scalable production-grade hardware solutions. Many system administrators in small organizations, quite clearly understand the advantages of ZFS and you can justify to management the savings that software compression and deduplication, LZ4 when buying a new product, only inexpensive decisions under this file system was not. But the Holy place is never empty, and QSAN decided to combine in one device all that we love in Linux and all that we value in ZFS.

Things we love about ZFS? For the snapshots, deduplication and absolute reliability given to us by Oracle. What’s not to love? For FreeBSD, which is able to do one thing, but stuck in its development somewhere in the first decade of the 21st century, and therefore is not suitable for NAS.

WHY?

Because a modern NAS is not only a file saver, but also an application server that runs containers, virtual machines, databases, and native programs. Few NAS manufacturers are ready to develop software for FreeBSD, including because of licenses, and what about Linux - you are wellcome: here you have a well-working and regularly updated kernel.

Test bench configuration:

Software:

  • VMware ESXi 6.5
  • Windows Server 2016
  • IOmeter patterns of real problems

QSAN XN8012R:

  • Firmware version 3.1.6
  • HDD: 3 x Seagate IronWolf 14 Tb, RAID 5, Gzip=On
  • SSD: Kingston 3 x 1.92 Tb, RAID5 3x, Gzip=On
  • Intel X520-DA2

Perhaps that’s what the developers of the company QSAN, the oldest manufacturer of entry-level storage. They decided that modern processors are ready to handle all the delights of ZFS: memory becomes cheaper, SSD drives-too, and therefore you can make a NAS, which has no analogue on the market. Previously, we have not tested QSAN products, and the more interesting it will be to get acquainted with the idea of its authors: everything is different - a different ideology, a different design… well, let’s go for it.

It is single-controller NAS based on 4-core Intel Xeon CPU with 3.3 GHz, the device has:

  • 12 bays for 3.5-inch SATA drives,
  • 4 separate bays for SATA SSD
  • 2 bays for hot plug SSD 2.5,
  • 2 bays for expansion cards,

and it is only what regards to hardware. As for the software, the manufacturer promises multi-level storage (Tiering), support for SSD caching, built-in virtualization, backup to the cloud and all the delights of ZFS, including compression, deduplication and renowned reliability.

Design features

To fit in the same case 12 high capacity and 6 high speed drives, the developers placed SSD bays on the back side. This is a very brave decision, allowing maximum use of space in the 2U case, but it has some features. The fact is that the drives installed at the outlet of the air flow tend to overheat, and to avoid this, QSAN installed a separate fan on their basket.

Moreover, if for 2.5-inch 7-mm SATA SSD cooling may not be necessary, for 15-mm NVMe SSD, of which there may be two pieces, this is a prerequisite for operation.

Two types of SSD need to fine-tune Tiering: conventional HDD represent a lower level of performance, SATA SSD - medium and SSD NVMe - supreme. Each layer can use its own RAID-array, and it is clear to all RAID 5/6, not the mysterious RAID Z1/Z2/Z3. The use of standard terminology can only be happy, but more on that later.

I don’t know who came up with such a serious NAS to deliver with 1-Gigabit network interfaces. And look - under each of the 4 RJ45 ports installed USB 3.0 port. To say that it’s uncomfortable - all the same that to tell nothing, fortunately, storage has two low-profile expansion slots: PCI-E Gen3 x8 and Gen3 x4, which we will supply two 10-Gigabit network card, Intel X550-T2 for copper and Intel X520-DA2 for DAC. Although QSAN recommends the use of their adapters (judging by the photos premakeready Intel) with native Intel we have no problems, but please (!) NEVER install the card with RJ45 ports in the left slot! Due to the fact that the expansion compartments are recessed into the body, your fingers will not fit to the snaps of the RJ45 chips, and to remove the network cables without tweezers will be VERY difficult.

Fortunately, this is the only design flaw found. Otherwise, everything is standard: fault-tolerant power supply manufactured by Delta, cooling system on the principle of N+1, convenient trays for disks and … even HDMI port. For Apple users, the company offers Thunderbolt 3 controller, and for connecting disk expansion bays - SAS-12Gb / s controller. By default, XN8012R comes with 8 GB of RAM, but the memory can be expanded up to 64 GB. The Linux-based operating system is not memory-intensive, so you should only add RAM if you use deduplication and the built-in hypervisor. Well, since we’re talking about software, let’s run and configure the NAS.

Software and virtualization

To put it mildly, QSAN was inspired by Synology products when developing the NAS operating system. In the world of Linux software there is nothing wrong with this: companies use OpenSource libraries and borrow ideas from each other, the only question is who will do something new or something better than others, and QSAN turned out… cool: HTML5, control panel, widgets, taskbar, monitoring the current load… Work, and be happy, just use Google Chrome or Firefox everything displays as it should.

If your company for the ZFS features used FreeNAS, XigmaNAS or just one of the clones FreeBSD, then you probably don’t have built-in file Manager: it is helpful to copy any amount of data inside NAS and not using the network interface. The developers of FreeNAS takes hostility to the requests of the community on the implementation of the file manager and community administrator FreeNAS did write that web filemanager is the “stupidest idea”? Actually, QSAN QSM is an operating system with support for ZFS and a normal web-based file manager with preview images, with basic functions (copy/delete), but this is not the only thing that distinguishes a commercial Linux product from a free FreeBSD.

Not so long ago under FreeNAS screwed hypervisor Bhyde, but it has compatibility issues with any guest operating systems, including most FreeBSD and Linux. Here you have QEMU, industrial de-facto standard with the used VNC libraries and drivers, equally well support both Linux and Windows as guest systems. Of course, under Linux there is a favorite hypervisor Proxmox by many system administrators, but there is no management of storage and virtualization from a single window, as in NAS-es.

Of course, the presence of built-in virtualization makes it pointless to integrate many other packages that are not directly related to the NAS-activity of the device, so QSAN does not bother with this direction. And although there is a SQL server (MariaDB, and VPN-server), from my point of view, deserves attention, except that the antivirus that scans the contents of the NAS-a schedule…

As for backup, there are very interesting features. First, as mentioned above, you can use snapshots as an alternative to local backups. In the ZFS file system, snapshots are part of the total amount of data that is deduplicated, so in terms of the cost of storing copies, ZFS snapshots are the best choice for today. For a large infrastructure, it makes sense to configure mirroring of the target folder on the NAS to a remote NAS (Qsan Xmirror Technology) or backup to the cloud. If you want to back up the contents of the Windows computers of your employees, Qsan offers to use the program Xreplicator, which was developed on the basis of Acronis True Image your with free license for 100 machines and Linux hosts use backup over Rsync.

However, there are disadvantages: for example, there is no mechanism for reserving virtual machines under VMware ESX, as a cloud backup storage, which we strongly recommend you to do, only three API providers are supported: Amazon S3, Hicloud S3 and Alibaba Cloud SS. There are claims and to work with browsers not on Chrome-core (in particular, under Firefox in the interface some buttons are not pressed), but basically, everything works without complaints.

Drive pools

As mentioned in the beginning of the article, XN8012R QSAN ZFS does not use ZFS terminology of type Zvol/Zdev, which confused even experienced system administrators. Here, everything is traditional: RAID, storage pools, shares. For example, if you have an array for a large number of disks, you will need an array RAIDZ3, allowing simultaneous failure of 3 hard drives (read or test report how Huawei Oceanstor survived failure of 5 HDDS in RAID5). It is now considered good practice to constantly test hard drives for early diagnosis of breakdowns, and QSM has a scheduled S. M. A. R. T. launch. tests, scanning the entire array (Scrub), as well as support for Seagate Ironwolf Health Management.

When creating a disk array, you have to choose whether you will have a streaming or transactional load, or a mixed load? It is also necessary to select the level of the array in a multilayer Tier-structure. For hard drives, this is the lowest level, and for different SSD speeds-the average or the highest, as well as the mechanism of data migration: on a schedule or manually. Treat this stage responsibly, as in the future changes will not be possible.

Tiering

The Tiering function works as follows: the file system constantly records the frequency of access to blocks, and then transfers data by layers, what is rarely used - stores on HDD, and the most popular data - on SSD or NVMe SSD. In this case, the SSD layer is the same array as your disk RAID, on which you can create folders and iSCSI volumes. Since you can use it as a regular array, its size can be safely added to HDD storage, and this is a key difference from SSD cache. Using Tiering, you specify a retention policy for each disk array when you create it. For example, higher level can be moved (with SATA SSD NVMe SSD), and below - impossible. The transfer of data from HDD to SSD is carried out on a schedule, for example, at night or on weekends.

When the “Auto Tiering” mode is enabled, the operating system independently determines the storage policy for the disk array, as well as the time of data transfer between levels. But keep in mind that data usage statistics can accumulate for a very long time, and what you would like to see on an SSD can be a dead weight on the HDD, so do not rely only on Tiering, and actively use the SSD pool for data that is important to low latency. Generally, in the Enterprise environment for the function of Tiering made to pay, and we are particularly pleased that the multilayer QSAN storage included storage.

SSD Cache

ZFS has native SSD caching (L2ARC) support, and even a single drive you can use for read/write caching. The cache warms up quickly enough, but the peak performance of the SSD does not reach, so high-intensity operations are performed with data on SSD arrays. For SSD cache, you can use any number of pooled drives. If you use only the read cache, keep in mind that it will be reset when you restart the machine. In General, the SSD cache is filled with blocks, and you can evaluate the efficiency in the diagrams below. Please note: the access speed does not exceed 2500 IOPS, so for high-intensity loads, the SSD cache on top of the HDD will not replace the normal SSD pool even in read mode.

En_SSD_Cache_hit_rate_1

En_SSD_Cache_hit_rate_2

The presence of SSD read cache is a prerequisite for the use of block deduplication.

Block Level Deduplication

This excellent ZFS capability works on the fly, allowing you to save tens of terabytes of data. When writing data to disk, the storage processor finds duplicate extents and replaces them with links pointing to the address of the block, due to which the duplicate data does not occupy space on the disk, and in General is not re-written. As a result, you not only gain in data volume, but also increase the speed of random reading and writing. If your storage maintains a secondary institution operating as a vault backup, virtuallock of Argov documents, here’s how much you can save:

  • Backups without software compression (usually - Rsync): from 85% to 90%
  • Virtual machine disks: 50% to 90%%
  • Database dumps: 45% to 70%
  • User home directories: 20% to 50%%
  • Mail archives: 20% to 40%

The more data in your organization, the greater the benefit. You can enable shared deduplication for folders and iSCSI volumes that use the file as storage. In general, deduplication should greatly reduce the speed of writing to disk, and on the chart we see that the speed of simple synthetics is significantly reduced.

En_Dedup_ssd_random

En_dedup_random

But in the patterns of real-world tasks deduplication on the contrary, gives a significant increase in speed.

En_dedup_pattern

En_Dedup_ssd_pattern

When testing access speed on 64-kilobyte blocks, we had a surprise: the performance was higher than that of 4-kilobyte blocks. I had to recheck the tests, but the results were repeated, and then I asked for clarification from the manufacturer of storage. Engineers reported that these storage systems are written in 64-kilobyte blocks to improve performance in most tasks. In General, this speed-just a gift for those who purchase a storage for use under Microsoft SQL Server, because it is 64-kilobyte access recommends to configure the storage itself Microsoft.

En_4k_vs_64k_hdd

En_4k_vs_64k_ssd

So without a doubt, for transactional access in a small organization, QSAN XN8012R a great choice. What is more interesting is the support of WORM (Write Once Read Many) technology, which in simple language means write protection for some resources. This protection can be set for a limited time, or forever. It is very convenient for backups: you have saved a backup to the storage, and you can already be sure that neither the cipher virus nor the hacker will be able to delete or change your backups.

En_seq_access

To test the speed of serial access, we used 2 ports of 10 Gbit/s, but the performance did not exceed the capabilities of one. In our test configuration for 3 HDD and 3 SSD is not particularly roam, but for large installations can connect up to 8 storage expansion units.

Here we must bear in mind that you have no fault tolerance: according to the scheme, the shelves are connected in series, without a loop, so for example if you fail the 5th regiment, then turns off and the 6th, and the 7th and 8th. But in General, when connecting 8 QSAN XD5324 shelves, each of which has 24 bays for 3.5 HDD, you can fill the entire storage system 14-terabyte hard drives, to obtain 2.83 210 Winchester volume 2886 TB of space for your needs.

Cost and licensing

The cost of the head unit QSAN XN8012R without hard drives is 6250$. Note that by default, the head unit is delivered even without a sliding rails, with a minimum amount of memory and 1-Gigabit interfaces. The deduplication feature in ZFS actively uses RAM to store extent tables, so if you want to save on the number of hard drives, please configure the storage with at least 32 GB of RAM. All software features such as Tiering, virtualization and snapshots do not require license activation.

Guarantee

By default, QSAN XN8012R comes with 3-year limited warranty, which the buyer may extend for another 2 years, getting the appropriate package at any time during the basic warranty period. At the time of preparation of the review, QSAN did not have service packages for service with limited response time, such as a replacement for the next working day or the departure of a specialist to the site.

Conclusions

Today, each storage supports SSD-drives, but not every has 6 Hot-Swap compartments for solid-state drives, which you can use without damaging the HDD. Today, each storage system has support for snapshots and SSD cache, but not every supports multi-layer storage On nvme SSD with hot swap. Such functions as Inline deduplication are the main criteria for choosing a storage provider, and practically determine the choice of a solution. And if you choose QSAN, this storage system will meet you with a friendly HTML5 interface, made according to the canons of traditional storage systems. Here everything is simple and clear, so that even a child will be able to understand the configuration of the server, run virtualization and configure user access rights. Storage system has built-in security systems, including antivirus and Firewall, so that from some misfortunes it can protect itself.

Yes, of course, everything that goes beyond the ordinary, has both its pros and cons: the unusual design causes difficulties when connecting cables, there are problems with the web-interface under the Firefox browser, and 8 GB of RAM, and even typed two 4-Gigabyte modules look frivolous.

But we are not the first say that NAS s to block access due to its versatility, it will replace SAN solutions from the initial price range. And now, the cost of a top QSAN XN8012R virtualization and deduplication comparable to the price of a 2 controller SAN-s, which are not able anything, except how to be a 2 controller SAN-s, which means memory and you can add, but missing the software to run under virtual machine. Overall, with the right approach, this storage system will allow you to save on the expansion of capacity at the data growth in your organization, and it is this feature that makes you look at the series XCubeNAS today.