Schooner just released an extended version of memslap (version 0.36) to the open source community
In this blog entry I’d like to describe the motivation behind the extensions and how we use them at Schooner. http://tangent.org
When we started developing the Schooner version of memcached we found that there was no readily available driver suitable for performance testing. In particular, no driver supported controlling the miss rate in the memcache. This is particularly important in the Schooner appliance because it implements memcache as a two level memory hierarchy: flash as the primary store, fronted by a smaller DRAM cache. Memcached “gets” that hit in the DRAM cache are easier to service than those that go to flash. We wanted to optimize Schooner memcached for workloads with both low and high miss rates. The versions of memslap and brutis that were available had no way to control miss rate.
memslap 0.36 introduces the notion of a “window” which records for each client context a history of the last N write operations (sets) to the memcached server. Each set operation either overwrites an existing key in the window or adds a new key to the window. If a new key is added, the oldest existing key is evicted so that the window size is preserved. The fraction of sets that overwrite is a configurable parameter.
When memslap issues a get, it randomly selects a key in the window. If the size of all data in the window is less than or equal to the size of the DRAM cache, most gets will hit in the cache. If the window is larger than the DRAM cache, some portion of gets will miss; varying the size of the window relative to the DRAM cache controls the average miss rate. In practice, you must make a few test runs with different window sizes to find the size that yields a desired miss rate. This same mechanism can be used to control the overall memcached miss rate in traditional memcached servers: just make the window larger than the memcached capacity.
Although miss rate control was the main reason to modify memslap, we also made other changes. The Schooner appliance targets the consolidation of multiple traditional memcached servers, so we needed a driver that could achieve very high traffic rates at large numbers of connections. We improved memslap’s efficiency so we could drive hundreds of thousands of requests per second from a single client. These changes also enabled memslap to drive hundreds of thousands of connections (some of our customers actually use this many connections!).
We also added more precise control over the mix of sets, gets and multigets, with control over the fraction of sets that perform overwrites. Overwrites are of special significance when using flash as the primary store. Since different customers use widely varying data and key sizes, we added the ability to specify detailed key and data size distributions.
We hope that others in the memcached community find memslap 0.36 useful. We would very much like to get feedback on it, and hear how others benchmark memcached servers.