Nutanix AHV End-to-End Setup -- Foundation, Prism Central, Networking, Storage, and Flow

Standalone Infrastructure | Component: Nutanix AOS 6.10+, AHV, Prism Central 2024.x | Audience: Enterprise Architects, Senior Sysadmins

Nutanix AHV is the hypervisor that comes with every Nutanix node at no additional cost, and it's increasingly the default choice for new Nutanix deployments. It's also architecturally different enough from vSphere and Hyper-V that people coming from those platforms consistently hit the same friction points: the Controller VM model, the relationship between Prism Element and Prism Central, how OVS networking works, and what "distributed storage" actually means when there's no separate storage controller.

This article covers a complete AHV cluster setup end to end: hardware requirements, AOS installation with Foundation, cluster formation, Prism Central deployment and registration, AHV networking with OVS, storage architecture, VM and image management, Nutanix Flow for microsegmentation, and HA and data protection concepts.

1. The Nutanix Architecture: What Makes It Different

Before anything else, the Controller VM (CVM) model. Every Nutanix node runs a dedicated CVM: a Linux based service VM that handles all storage I/O for that node. The CVM runs on the hypervisor but is isolated from guest VM resource contention via vCPU pinning and memory reservations. All AHV guest VMs write to and read from storage through the CVM on their local node. The distributed storage layer (Nutanix DSF or AOS Storage) coordinates across all CVMs in the cluster to provide a single, pooled, resilient storage namespace.

The consequence is that you never touch the CVM directly in normal operations. You don't manage storage by connecting to it separately. You manage compute and storage through the same interface because they're the same platform. That's the hyperconverged premise and it's why the operational model is simpler than managing a separate SAN or NAS.

The two management interfaces: Prism Element is the per-cluster management interface running on the CVM cluster itself. You access it at the cluster's virtual IP on port 9440. It handles cluster level operations: storage containers, network configuration, local VM management, and cluster health. Prism Central is a separate appliance that provides a single pane of glass across multiple clusters, as well as advanced features like Nutanix Flow microsegmentation, VM templates, playbooks, and lifecycle management at scale. Some features only exist in Prism Central. For anything beyond a single cluster, Prism Central is required.

2. Hardware Requirements and AOS Installation

Hardware

Nutanix sells hardware as nodes, either as NX-series appliances or as software on certified third party hardware (Dell XC, HPE SimpliVity, Lenovo ThinkAgile HX). For NX-series nodes the hardware configuration is fixed by model: you select a model and get a specific CPU, RAM, and drive configuration. For software on OEM hardware, the Nutanix Hardware Compatibility List defines which configurations are certified.

Minimum cluster size for production AHV with AOS is three nodes. Three nodes gives you two-way replication (replication factor 2, or RF2) where data has two copies distributed across the cluster. One node failure is tolerated with RF2. Four node clusters with RF2 give you better rebuild time after a failure because there are more nodes to participate in the rebuild. Five nodes with replication factor 3 (RF3) tolerate two simultaneous node failures.

AOS Installation with Foundation

Foundation is Nutanix's bare-metal imaging tool. It installs AOS and AHV on one or more nodes simultaneously from a single browser session. You run Foundation from the Foundation VM on one of the nodes or from a standalone Foundation appliance on your network.

1. Download Foundation from the Nutanix Portal (my.nutanix.com). You'll need a Nutanix account. Select the Foundation version compatible with your target AOS version.
2. Boot Foundation on the node network. Foundation discovers nodes on the imaging network via IPMI. You access the Foundation UI at the node's IPMI IP on port 8000, or from a browser on the management workstation.
3. Configure the imaging spec: assign CVM IP, AHV host IP, subnet mask, gateway, DNS, and hostname for each node. Upload or point to the AOS bundle (the full installer, not the upgrade package).
4. Run Foundation. Foundation installs the hypervisor, deploys the CVM, and forms the cluster automatically. For three nodes on fast storage this takes roughly 45 to 60 minutes.
5. Access Prism Element at the cluster VIP on port 9440 after Foundation completes. Default username is admin, default password is the node's IPMI password (usually printed on the node hardware tag for NX-series).

3. Cluster Formation and Node Management

Foundation handles cluster formation automatically during imaging. For clusters assembled from individual nodes outside of Foundation, you can form the cluster from the command line on any CVM.

# SSH to any CVM as the nutanix user
# Default CVM SSH password is set during Foundation or is 'nutanix/4u' on factory-default nodes

# Form the cluster with three CVM IPs
cluster -s 10.0.100.21,10.0.100.22,10.0.100.23 create

# Set the cluster name
ncli cluster edit-params new-name="prod-cluster-01"

# Set the cluster VIP (the single IP used to access Prism Element)
ncli cluster edit-params external-ip-address=10.0.100.20

# Verify cluster is healthy
cluster status
ncli cluster info

# Check all services on the local CVM
nutanix_installer --list_services

# Check cluster wide health
cluster health

# Check which CVMs are up and what their roles are
nodetool ring

# Check storage pool and container status from Prism Element CLI
ncli sp list
ncli ctr list

4. Prism Central Deployment and Registration

Prism Central is deployed as a VM (or a scale-out three-VM cluster for production environments) from Prism Element. You don't download and deploy an OVA separately. The deployment is initiated from within Prism Element and Nutanix handles the VM creation automatically.

1. In Prism Element, go to the gear icon and select Prism Central Registration.
2. Choose Deploy a new Prism Central instance. Select the size: small (16 vCPUs, 32 GB RAM, single VM) for smaller environments, large (24 vCPUs, 56 GB RAM) for larger clusters, or three node scale-out (each node at small or large) for production deployments managing multiple clusters.
3. Enter the Prism Central VM IP, subnet, gateway, and DNS. Prism Central needs a static IP. It also needs a separate IP for each VM in a scale-out deployment plus a VIP for the cluster.
4. Select the storage container and network to deploy the Prism Central VM(s) on. Prism Central itself is stored on the Nutanix DSF cluster.
5. Click Deploy. Nutanix downloads the Prism Central image from the portal (requires internet access) or from a configured dark site URL, creates the VM, and starts it. This takes 20 to 30 minutes.
6. After Prism Central starts, complete the registration: the Prism Element cluster connects to Prism Central and registers itself. Additional clusters register from their own Prism Element instances pointing to the same Prism Central VIP.

5. AHV Networking: OVS, VLANs, and Bonds

AHV uses Open vSwitch (OVS) for all networking. Unlike VMware's vSwitch or Hyper-V's virtual switch, OVS is a full production grade open source virtual switch with support for VLAN trunking, LACP bonding, and OpenFlow compatible programmatic management. Nutanix Flow uses OVS for microsegmentation enforcement at the hypervisor layer.

Virtual Switch Design

AHV creates a default virtual switch (vs0) during installation. This switch carries all traffic: management, storage (CVM), and VM traffic by default. For production environments you should create dedicated virtual switches for different traffic types or at minimum use VLAN tagged networks to separate them logically.

# Access the acli (Acropolis CLI) from any CVM
# The acli is the primary CLI for AHV host and network configuration

# From CVM SSH, enter acli
acli

# Create a bond on physical uplinks (LACP)
net.create_bond bond0 mode=active-backup uplinks="eth0 eth1"

# Or use LACP (802.3ad) if your switch supports it
net.create_bond bond0 mode=balance-slb uplinks="eth0 eth1"

# Create a virtual switch using the bond
net.create vs0 bond_uplink=bond0 mtu=1500

# Create a VLAN-tagged network for VMs on VLAN 400
net.create_vlan vm-network-400 vswitch_name=vs0 vlan_id=400

# Create an IP pool for the network (enables IP address management in Prism)
net.add_dhcp_pool vm-network-400 start_address=10.0.400.100 end_address=10.0.400.200

# List all configured networks
net.list

# Exit acli
exit

6. Storage: Containers, Storage Pools, and Volume Groups

Nutanix DSF (Distributed Storage Fabric) pools all storage across all nodes into a single namespace. You don't see individual disks or RAID sets. What you see and manage are storage pools and storage containers.

Storage Pools

A storage pool is the aggregate of all physical storage across the cluster. Nutanix creates a default storage pool during cluster formation. You don't create multiple storage pools or manage individual drives. Nutanix handles drive management internally, including intelligent tiering of hot data to SSD and cold data to HDD (on hybrid nodes) automatically using its Intelligent Tiering (IT) engine.

Storage Containers

A storage container is a logical subdivision of the storage pool with its own set of policies: replication factor, compression, deduplication, and erasure coding. VMs store their vDisks in storage containers. You can have multiple containers with different policy settings on the same cluster.

# From CVM SSH, use ncli for storage container management
ncli ctr create name=VMs-Production rf=2 compression-enabled=true

# Enable deduplication on a VDI container
ncli ctr edit name=VMs-VDI on-disk-dedup=post_process

# Enable erasure coding on an archive container (cold data only)
ncli ctr edit name=Archive erasure-code=off-fault-tolerance-preference

# List all containers with usage stats
ncli ctr list
ncli ctr get name=VMs-Production

Volume Groups

Volume groups expose Nutanix storage to external clients as iSCSI LUNs. They're used for workloads that can't use the AHV virtual disk interface directly: bare metal servers, Oracle RAC, or applications that require raw block devices. Volume groups are not the same as storage containers. A volume group is a named set of vDisks presented over iSCSI. You attach a volume group to specific VMs or to external initiators via the iSCSI interface.

7. VM and Image Management

AHV VMs are created and managed from Prism Element (for single-cluster operations) or Prism Central (for multi-cluster environments or when using templates and playbooks). The VM model in AHV is similar to other hypervisors: vCPUs, memory, virtual disks (vDisks stored in storage containers), and virtual network interfaces connected to AHV networks.

Creating VMs from the Image Service

The Image Service stores ISO and disk images for VM creation. Upload an image to the Image Service once, and it's available for VM creation across the cluster (or across all clusters in Prism Central). AHV doesn't have a separate datastore browser for ISOs like vSphere. Images live in the Image Service and are referenced by name during VM creation.

1. In Prism Element or Prism Central, go to the Image Configuration section under Settings.
2. Upload an ISO or qcow2/VMDK disk image. Images are stored in the default storage container.
3. Create a VM: specify vCPU count, memory, create a vDisk (boot disk, specify size and container), and add a CD-ROM drive pointing to the uploaded ISO for OS installation.
4. After OS installation, eject the ISO from the CD-ROM, install Nutanix Guest Tools (NGT) for improved integration (enhanced metrics, consistent snapshots, and in-guest IP address reporting to Prism), and create a snapshot to use as a template baseline.

VM Templates in Prism Central

Prism Central supports VM templates with versioning. A template captures the VM's configuration and disk state. When you deploy from a template, Prism Central creates a full copy of the disks and applies guest customization (hostname, IP, SSH keys) via cloud-init or Sysprep. Template versions let you update a template without losing the previous version. Deployments from older template versions remain linked to that specific version's disk state.

8. Nutanix Flow: Microsegmentation

Nutanix Flow is the microsegmentation feature built into AHV networking. It enforces security policies at the VM's virtual NIC rather than at a perimeter firewall, which means east west traffic between VMs on the same VLAN is subject to policy enforcement. This is the traffic that traditional perimeter firewalls don't see and that ransomware uses for lateral movement.

Flow requires Prism Central. It's not available in Prism Element alone. Flow policies are defined in Prism Central and pushed to all AHV hosts via the OVS integration. The enforcement is distributed: each AHV host enforces policy for the VMs running on it. There's no centralized traffic inspection appliance that becomes a bottleneck.

Security Policy Model

Flow uses an application centric policy model. You group VMs into categories (Web, App, Database, for example) and then write policies in terms of those categories: allow Web to App on port 443, allow App to Database on port 5432, deny all else. When VMs move between hosts (live migration) the policy moves with them because policy is attached to the VM category, not to a host, network, or port group.

• Monitor mode: Flow watches traffic and logs what would be blocked without actually blocking anything. Use this to baseline existing traffic patterns before enforcing policy. Running monitor mode for a week before enforcing gives you a clear picture of what legitimate traffic exists between VM groups.
• Enforce mode: Policy is actively enforced. Traffic that doesn't match an allow rule is blocked. Switch to enforce mode only after you've validated the allow rules cover all legitimate traffic, or you'll block production traffic.
• Quarantine mode: Isolates a specific VM to block all traffic except a designated forensics path. Used when a VM is suspected compromised. Quarantine can be applied per-VM without affecting other VMs in the same category.

9. High Availability and Data Protection

HA in AHV

Nutanix HA works differently from VMware HA. There's no separate HA admission control configuration or dedicated HA heartbeat network. When a node fails, the cluster detects the CVM going offline (typically within 30 seconds based on the Immediate rebuild mode default), and AHV live migrates or restarts VMs from the failed node on surviving nodes. Nutanix HA is enabled by default and requires no configuration beyond sizing the cluster to have enough free capacity to absorb a node failure.

The capacity reservation to absorb one node failure: divide the failed node's RAM by the number of surviving nodes to estimate the headroom each surviving node needs. A four node cluster where each node has 256 GB RAM needs roughly 85 GB of free RAM per surviving node (256 / 3) to restart all VMs from the failed node. Run the Nutanix capacity planning tool on my.nutanix.com to verify your cluster has sufficient headroom before production deployment.

Data Resilience

DSF maintains the configured replication factor across all nodes. When a node fails, DSF immediately starts rebuilding the missing data copies on surviving nodes using the Immediate rebuild mode (triggered roughly 60 seconds after a CVM goes offline). Rebuild priority is configurable in AOS 6.7 and later: you can set it to minimize the impact on running workloads at the cost of a longer rebuild window, or maximize rebuild speed at the cost of higher I/O overhead on production VMs during the rebuild period.

Protection Domains and Protection Policies

Nutanix has two approaches to VM protection and replication, and they aren't fully equivalent in capability:

• Protection Domains (Prism Element): The legacy approach. Configured per cluster in Prism Element. Supports synchronous replication (Metro, 0 RPO), NearSync (1 to 15 minute RPO), and asynchronous replication. Required for synchronous Metro replication on ESXi and Hyper-V clusters.
• Protection Policies (Prism Central): The modern approach for AHV. Configured centrally across multiple clusters. Supports the same RPO options as Protection Domains for AHV. Required for Metro synchronous replication on AHV clusters. Provides better scale and centralized management.

If you're running AHV, use Protection Policies in Prism Central for DR configuration. Protection Domains are still available but are the legacy path, and Nutanix's active development for AHV DR is in Protection Policies.