Organizations are replacing older hypervisor environments and running VMs and containers on a single platform. According to Eric Herzog, CMO at Infinidat, a Lenovo company, consolidating thousands of VMs places specific demands on host configuration and storage.
As they modernize their IT infrastructure, organizations are increasingly moving away from virtualization environments built on older hypervisors. Instead, they are adopting unified, cloud-native orchestration platforms that manage traditional virtual machines (VMs) and containerized workloads through a shared control plane. Operational efficiency and lower costs are cited as advantages. In a technical article, however, Eric Herzog points out that consolidating very large numbers of VMs in host clusters creates new requirements for the data and storage architecture. Performance, availability, cyber resilience and resource efficiency at scale, he writes, require careful coordination between container orchestration and storage infrastructure.
Bottlenecks at the host level
With thousands or tens of thousands of VMs, the way hosts interact with storage systems changes, according to Herzog. In cloud-native environments, each VM disk is typically tied to its own persistent volume claim, which is dynamically mapped to a logical block device. When hundreds of VMs run on one physical host, that host must manage thousands of storage mappings, paths and I/O channels at the same time. Without fine-tuning at the operating system and driver level, bottlenecks can arise in device discovery, in kernel limits on LUNs and in storage queue depth. For stable clusters, the author names three measures: systematic tuning of kernel parameters at the host level, efficient multipathing strategies, and routines for releasing device connections so that orphaned paths and exhausted resources are avoided.
Requirements for enterprise storage
For large VM estates not to impair application responsiveness, the storage architecture must, in the author’s view, meet several criteria. The first concerns performance and latency. Random I/O spikes arise, for example, from boot storms, concurrent snapshot operations or the automated redistribution of workloads. The platform is expected to deliver predictable response times in the microsecond range across all consolidated workloads to avoid delays for business-critical applications.
The second criterion is availability and cyber resilience. If a single host fails in a densely populated environment, many VMs are affected. The storage system must therefore prevent data loss and enable seamless failover. Herzog refers to multipathing protocols with immediate path failover and load balancing across redundant storage controllers. Automated snapshots, cloning of persistent volumes and space-efficient provisioning are also needed. They are meant to instantiate VMs quickly and protect data continuously without consuming excessive physical capacity.
Workload mobility and storage provisioning
In dynamic virtualization environments, workloads are rarely static. Live migration, in which running VMs move between physical hosts without interrupting operations, places its own demands on persistent storage. Volumes must be provisioned using block-level access modes that allow several nodes to access them simultaneously during migration. In addition, provisioning workflows must integrate with the orchestrator’s native storage driver interfaces so that volumes can be created, expanded and deleted in a policy-driven way, without manual storage administration.
Context
If organizations combine optimized host kernel configurations with block storage connectivity that offers high throughput at low latency over Fibre Channel or NVMe over Fabrics, Herzog says previous scalability limits can be pushed back. Hundreds of volumes per host could then be operated efficiently. The article states these requirements from a storage vendor’s perspective and does not cite measurements.
Preview
As organizations move away from older hypervisor environments, storage connectivity moves closer to the center of planning. Those who consolidate VMs and containers on one platform need to align host configuration, multipathing and protocol choice (Fibre Channel, NVMe-oF) with the storage architecture.
Dr. Jakob Jung is Editor-in-Chief of Security Storage and Channel Germany. He has been working in IT journalism for more than 20 years. His career includes Computer Reseller News, Heise Resale, Informationweek, Techtarget (storage and data center) and ChannelBiz. He also freelances for numerous IT publications, including Computerwoche, Channelpartner, IT-Business, Storage-Insider and ZDnet. His main topics are channel, storage, security, data center, ERP and CRM.
Contact via Mail: jakob.jung@security-storage-und-channel-germany.de