PSVM is currently based on stable/icehouse and located in the public branch.

What problem does PSVM solve?

PSVM was developed and integrated as a new feature into a fork of the OpenStack Nova project to solve a scalability issue with the underlying physical switching infrastructure. In particular with the Cisco Nexus switching line, there is a limit on the STP logical interfaces which is a product of the number of VLANs times the number of ports using that VLAN. Depending on the NX-OS version this limit ranges from 32K to 48K which can easily be reached when using trunk ports and fabric extenders (FEX).

How does PSVM work?

The new feature manages this resource by dynamically configuring which VLANs are allowed on each port in use by the OpenStack environment. PSVM makes OpenStack aware of the physical networking infrastructure by tracking all the server to switch port connections and synchronizing the OpenStack network information (VLANs in use by host) with the specified supporting switches. The sync is performed by default at an interval of 60 seconds. This can be overridden in nova.conf. PSVM adds a plugin/driver framework into nova-network. The initial release of PSVM supports the Cisco Nexus 5K line of switches and their associated FEXs. The framework makes it relatively easy to add support for other switches as needed.

Why use PSVM?

For existing deployments it is difficult to migrate networking over to the Neutron project, OpenStack’s networking service. Neutron has many subcomponents many of which have not reached production ready status and relies on complex third-party technologies such as Openvswitch. Neutron had centralized its gateway services initially but is now moving over to a decentralized approach similar to nova-network’s multi-host mode under the name distributed virtual router (DVR). The DVR feature is not production ready yet. PSVM adds the neutron-like pluggable framework into the simple yet robust networking in nova. If using Icehouse or upgrading to Icehouse it is easy to deploy and activate PSVM. (See install guide)

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Contact the PSVM team at: psvm@onop.org

Nova-Network Physical Switch VLAN Manager (PSVM)

https://blueprints.launchpad.net/nova/+spec/nova-network-physical-switch-vlan-manager

This blueprint proposes the addition of the physical switch VLAN Manager (PSVM) to nova-network. PSVM provides a pluggable framework inside of nova-network to allow nova-network to manage the physical switch infrastructure. The first plugin/driver support will be for the Cisco Nexus line of switches.

Problem description

Nova-network in multi-host mode provides reliable networking but lacks integration with the physical switching hardware. PSVM adds this management capability in a pluggable fashion to support a variety of switches.

Proposed change

PSVM Module

The physical_switch_vlan_manager module resides in nova.network and is responsible for handling the database requests through nova.objects. For environments where compute_nodes are connected to more than one switch, this module spawns separate threads for each switch.

Plugin

The first plugin will be the cisco_nexus_plugin.CiscoNEXUSPlugin. The plugins will be responsible for ordering the driver commands appropriately and handling any specialized logic for that switch. In the case of the initial plugin, CiscoNEXUSPlugin, provisioning of the physical switch hardware is accomplished via an intelligent sync. The configuration on the interface connecting the compute host executing the code is compared to the VLANs that are expected for that compute host in the nova database. Missing VLANs are provisioned while Excess VLANs are removed. There is a nova.conf option to preserve network operational VLANs such as a Native VLAN. The sync option was chosen as it was deemed more efficient to perform network operations in bulk and less expensive in terms of network calls to the switch. The sync is also more reliable than running a simple add/delete routine since it autocorrects provisioning from external causes (network administrator, switch reboots)

Driver

The first driver will be the cisco_nexus_network_driver.CiscoNEXUSDriver. The drivers are responsible for implementing the network communication to the switch. In the case of the initial CiscoNEXUSDriver this is accomplished in the driver by using Cisco’s verison of the ncclient which implements netconf XML+SSH.

Alternatives

Manually configure switches or use Neutron. The first alternative is not feasible for large deployments. The second is fine for greenfield deployments but not for existing deployments. Additionally, PSVM can work in conjunction with the neutron-migration blueprint to transition nova-network deployments to neutron.

Data model impact

There are three new tables named psvm_switch, psvm_switch_credential, and psvm_switchport_binding along with their respective shadow tables. There is no migration of existing data required. After these tables are created via the SQLAlchemy migration method, the Cloud Administrator will need to populate the tables via either the nova-manage interface or the REST interface. The physical switch infrastructure and its connections to the compute nodes are represented in these three tables.

REST API impact

PSVM CREDENTIAL Table

A new v2 API extension will be created to access data in the psvm credential table. Extension details:

• Name: Psvmcred
• Alias: os-psvmcred

The extension is defined as:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/psvmcred.py

V2 API specification:

Sample POST: A request to create a psvm credential with the user name as test_name and password as test_password is of the form:

POST: /v2/os-psvmcred -d '{ "psvmcred":
{ "user_name": "test_name", "password": "test_password" }}'

The response should have the following form:

{
    "psvmcred": {
    "created_at": "2012-10-01T18:50:27.781065",
    "deleted": false,
    "deleted_at": null,
    "id": 1,
    "password": "test_password",
    "updated_at": null,
    "user_name": "test_name"
    }
}

Sample GET: A request to retrieve information from the entry in the psvm credential table with row id 1 is of the form:

GET: /v2/os-psvmcred/1

The response will have the same data as the POST response.

Sample PUT: A request to modify the psvm credential with row-id 1 to set the user name as new_name and password as new_password would be of the form:

PUT: /v2/os-psvmcred/1 -d '{ "psvmcred":
{ "user_name": "new_name", "password": "new_password" }}'

The response should have the following form:

{
    "psvmcred": {
        "created_at": "2012-10-01T18:50:27.781065",
        "deleted": false,
        "deleted_at": null,
        "id": 1,
        "password": "new_password",
        "updated_at": "2012-10-01T18:51:27.781065",
        "user_name": "new_name"
    }
}

Sample DELETE: A request to delete row 1 in the psvm credential table is of the form:

DELETE: /v2/os-psvmcred/1

On success a 200 HTTP response code is returned.

HTTP response codes:

v2:

Normal HTTP Response Code: 200 on success

PSVM SWITCH Table

A new v2 API extension will be created to access data in the psvm switch table. Extension details:

• Name: Psvm
• Alias: os-psvm

The extension is defined as:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/psvm.py

V2 API specification:

Sample POST: A request to create a psvm switch entry with the switch IP set to 1.2.3.4 and using the row id 1 in the psvm switch credential table for accessing the switch is of the form:

POST: /v2/os-psvm -d '{"psvm": {"ip": "1.2.3.4", "switch_cred_id": 1}}'

The response should have the following form:

{
    "psvm": {
        "created_at": "2012-10-01T18:50:27.781065",
        "deleted": false,
        "deleted_at": null,
        "id": 1,
        "ip": "1.2.3.4",
        "switch_cred_id": "1",
        "updated_at": null
    }
}

Sample GET: A request to retrieve information from the entry in the psvm credential table with row id 1 is of the form:

GET: /v2/os-psvm/1

The response will have the same data as the POST response.

Sample PUT: A request to modify the psvm switch entry with row-id 1 to set the switch ip address to 2.3.4.5 and using row 2 of the credential table would be of the form:

PUT: /v2/os-psvm/1 -d '{ "psvm": { "ip": "2.3.4.5",
            "switch_cred_id": "2" } }'

The response should have the following form:

{
    "psvm": {
        "created_at": "2012-10-01T18:50:27.781065",
        "deleted": false,
        "deleted_at": null,
        "id": 1,
        "ip": "2.2.2.2",
        "switch_cred_id": "1",
        "updated_at": "2012-10-01T18:50:28.791065"
    }
}

Sample DELETE: A request to delete row 1 in the psvm table is of the form:

DELETE: /v2/os-psvm/1

On success a 200 HTTP response code is returned.

HTTP response codes: v2: Normal HTTP Response Code: 200 on success

PSVM SWITCHPORT BINDING Table

A new v2 API extension will be created to access data in the psvm port bind table. Extension details:

• Name: Psvmpbind
• Alias: os-psvmpbind

The extension is defined as: http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/psvmpbind.py

V2 API specification:

Sample POST: A request to create a psvm port bind entry with the switch id 1 (from the psvm table), the compute node id 1 and the switch port set to “e100/1/1” is of the form:

POST: /v2/os-psvmpbind -d '{ "psvmpbind": { "switch_id": 1,
            "compute_node_id": 1, "switch_port": "e100/1/1" } }'

The response should have the following form:

{
    "psvmpbind": {
        "compute_node_id": 1,
        "created_at": "2014-08-13T18:46:41.305930",
        "deleted": false,
        "deleted_at": null,
        "id": 1,
        "switch_id": 1,
        "switch_port": "e100/1/1",
        "updated_at": null
    }
}

Sample GET: A request to retrieve information on the entry in the psvm port bind table with row id 1 is of the form:

GET: /v2/os-psvmpbind/1

The response will have the same data as the POST response.

Sample PUT: A request to modify the psvm port bind entry with row-id 1 to set the switch port to “e100/2/2” would be of the form:

PUT: /v2/os-psvmpbind/1 -d '{ "psvmpbind": { "switch_id": 1,
        "compute_node_id": 1, "switch_port": "e100/2/2", } }'

The response should have the following form:

{
    "psvmpbind":
    {
        "compute_node_id": 1,
        "created_at": "2014-08-13T18:46:41.305930",
        "deleted": false,
        "deleted_at": null,
        "id": 1,
        "switch_id": 1,
        "switch_port": "e100/2/2",
        "updated_at": "2014-08-13T18:47:41.305930"
    }
}

Sample DELETE: A request to delete row 1 in the psvm switchport binding table is of the form:

DELETE: /v2/os-psvmpbind/1

HTTP response codes:

v2:

Normal HTTP Response Code: 200 on success

Security impact

None

Notifications impact

None

Other end user impact

python-novaclient has new commands for the cloud administrator which all begin with psvm. list, get, create, update, and delete commands are available for each of the three new tables.

Performance Impact

In the linux_net module the manage_vlan_on_physical_switch static method is decorated with the utils.synchronized semaphore lock. This is to ensure only one sync operation is run at a time.

In the physical_switch_vlan_manager module one thread is spawned per switch (usually 1 or 2 switches) to make changes to the physical infrastructure more efficiently. This module also calls nova.objects to retrieve database information required for this feature.

In the nova.network.manager module the _sync_physical_network method is a periodic task which is set to run every 60 seconds by default but is configurable by the Administrator in nova.conf.

With minor conflicts between the database and the physical switch the PSVM sync completes in under a second dependent on hardware. With large differences between the two as in an initial install, the sync could take up to 10 seconds dependent on hardware.

Other deployer impact

The PSVM feature must be explicitly enabled by setting psvm=True in the nova.conf default section.

Below are the new flags.

• psvm
• psvm_plugin
• psvm_driver
• psvm_ignore_vlans
• psvm_sync_interval

psvm is False by default.

psvm_plugin is set to cisco_nexus_plugin.CiscoNEXUSPlugin under nova.network.plugins which is the only plugin available at this time.

psvm_driver is set to cisco_nexus_network_driver.CiscoNEXUSDriver under nova.network.plugins which is the only driver available at this time.

psvm_ignore_vlans is set to 1 by default to support the standard native VLAN.

psvm_sync_interval is set to 60 by default so the periodic task executes every 60 seconds.

Developer impact

None

Implementation

Work Items

PSVM feature is complete.

Dependencies

ncclient==0.3.1 or >=0.4.2

Cisco’s version of the ncclient which is 0.3.1 which can be cloned from https://github.com/CiscoSystems/ncclient or install version >= 0.4.2 thru pip.

pip install -U ncclient

eventlet>=0.15.2

eventlet had a concurrency problem which was exposed often when used with paramiko. This was fixed in 0.15.1 which introduced a memory leak and was fixed in 0.15.2.

pip install -U eventlet

Testing

None

Documentation Impact

• Nova.conf flags
• REST API
• Nova-manage
• PSVM feature

References

None