Enterprise Flash Memory: Overview
Enterprise Flash Memory Overview
Dr. John R Busch, CTO and Founder
Why Use Flash in Scale-Out Datacenters?
Flash memory is a high-performance computer memory that can be electronically erased and reprogrammed (non-volatile). Enterprise-class flash memory in the form of commodity solid state disks (SSDs) and PCI-e cards is available today with a broad range of architectures, performance and reliability characteristics, capacities, and price points. In this series of blog entries we provide our analysis of the state and trends of enterprise-class commodity flash memory as it relates to deployment in datacenters scale out.
The latency, bandwidth, capacity, cost, and persistence benefits of flash memory are compelling.
Flash memory offers access times that are 100x faster than those of hard disk drives (HDDs), and it requires much less space and power than HDDs. It consumes only 1/100th the power of DRAM, and can be packed much more densely—providing much higher capacities than DRAM. And flash memory is far less expensive than DRAM to both purchase and operate. Flash memory is persistent when written, whereas DRAM loses its content when its power is turned off. Flash memory can be organized into modules of different capacities, form factors, and physical and programmatic interfaces.
Challenges in Exploiting Flash Memory
Flash memory has many promising characteristics—but also many idiosyncrasies. Effectively incorporating flash memory into system architectures requires thoughtful design and optimization—starting at the application layer, and extending throughout the operating environment and down to the physical machine organization.
The algorithms and mechanisms of current operating systems and applications were not designed to exploit the very high access rates of flash memory. These algorithms and mechanisms generally lack the high degree of parallelism and granular concurrency control necessary to effectively exploit the very high IOPS of flash memory.
Flash memory access times are 1000x slower than DRAM, so to effectively utilize flash memory a memory hierarchy with intelligent, specialized DRAM caching of flash content is required .
Typical software was also not designed to deal with the idiosyncrasies of flash memory. Flash memory chips have write-access behavior that is very different than DRAM memory. At the chip level, flash memory writes can only be done into pages (~4 kB), and before writing, a block of pages (128KB) needs to be erased, which is very slow (~1.5 millisecond). As a result, smaller logical writes need to be buffered and combined into larger physical write blocks before writing (this is called write coalescing). The deletion of small logical writes leaves holes (internal fragmentation) in the physical write blocks which need to be combined in the background to free up space while remapping the location of the stored logical writes (garbage collection). Garbage collection needs to be aggressive since new data cannot be written into flash until erase blocks have been freed up and pre-erased, which is a very slow operation.
Flash memory has limits on how many times a block can be erased (~100k times for SLC, ~3000k times for MLC). As a result, block writes need to be spread uniformly across the total flash memory subsystem to maximize the effective lifetime (this is called wear leveling).
Flash memory stores permanent data, so fault tolerance is required to insure the accuracy and availability of the data when it is written. Strong error detection and correction codes are required to recover from flash cell and flash chip failures. Replication at the system level and/or at the SSD/flash card level is required to handle SSD or PCI-e flash card failures.
There are numerous architectural and product alternatives for enterprise flash memory. The selection of an appropriate flash technology and flash subsystem configuration comes down to creating a balanced system for the target workload that achieves the required system uptime at the best price/performance. This analysis requires workload characterization, system measurement, and TCO and availability modeling.
Our Schooner Labs evaluation of flash technologies analyzes alternative flash architectures, alternative flash products, and system and application level performance, availability and TCO.