VLANd support in LAVA test jobs

What is a VLAN?

In terms of VLANd and the LAVA support based upon it, a VLAN is a way to provide isolation of various network ports across switches - similar to changing the connections to provide a private network. e.g. we want to be able to connect switch 3/port 7 & switch 1/port 6 to make one VLAN, then repeat for more arbitrary selections of switch and port. Once the VLAN is created, the interfaces connected to the switch/port will only be able to access addresses within that VLAN.

VLANd and LAVA

VLANd provides a simple utility to control switches on a single site, single network basis. The LAVA integration is implemented as one of the Protocols supported by the refactoring. As such, the LAVA integration inherits the restrictions of the original design of VLANd.

Unlike the Multinode Protocol, the lava-vland protocol has only a minimal API for use during the test shell, providing static information about the network interfaces on the device. The MultiNode API is also available, as with all other multinode jobs.

VLANd test shell helpers

lava-vland-self

Prints details of the sysfs path, mac address and admin-assigned interface name for each interface on this device:

/sys/devices/platform/ocp/47400000.usb/47401c00.usb/musb-hdrc.1.auto/usb1/1-1/1-1:1.0/net/eth1
00:e0:4c:53:44:58
iface1
/sys/devices/platform/ocp/4a100000.ethernet/net/eth0
90:59:af:5e:69:fd
iface0

lava-vland-tags

Prints details of the Interfaces and link speeds:

iface1
100M
iface0
1G
iface0
100M

VLANd Restrictions

The design of VLANd set out some clear constraints on the support to be created:

  • 1 access port on 1 switch being on 1 VLAN, no more, no less.
    • aka a port-based or static or manually-created VLAN (depending on which vendor’s docs you read!).
  • No support for dynamic VLANs
    • (switch calling out to external services to determine which VLAN a newly-detected connection should be connected to)
  • No support for filtering on ports to set up VLANs by traffic analysis etc.
  • No support for egress/ingress control such that a port may interact with ports outside of its own defined VLAN.
  • No support for cross-site VLANs via QinQ or similar.
  • Ports defined in terms of the switch/port combination.
  • Some switch/port combinations are to be locked so that test jobs cannot put infrastructure devices into a test VLAN.

VLANd Design goals and considerations

  • Set up arbitrary sets of VLANs
  • Map interfaces to switch ports in the LAVA device instance configuration.
  • Run a single VLAN daemon instance per lab
  • Switches are identified by IP or hostname - DNS must work for names to be used
  • Support a regular background read-only check that the switch config is reflected in the DB

LAVA and VLANd Device considerations

Requirement for multiple interfaces

Initial support for VLANd in LAVA sets up the VLANs at the start of the job. Many test jobs will require the device to download artifacts from the dispatcher (which itself has downloaded from third party sites) using protocols like TFTP. The device therefore needs to be able to reach the dispatcher over the network and this has implications for which devices are usable with VLANd at this stage.

Devices to be used with VLANd must have multiple network interfaces. It is not required that all interfaces are enabled at boot, simply that the boot process has a usable network interface. It is up to the test job writer whether the other interface(s) are enabled at boot or enabled/disabled during the test job - VLANd has no requirement other than that the physical hardware has a cable attached to the specified switch/port.

Future changes are expected to allow for devices with only a single interface to use VLANd but this requires code changes to support setting up the VLAN after the device has downloaded files using TFTP but before the serial connection is used to run the boot commands. This could result in a test job where the device has no access to the internet or the dispatcher during the rest of the test job. LAVA continues to control the physical device using the serial connection, including to implement the MultiNode API but some test jobs may use dynamic connections made from the dispatcher - such test jobs would not be able to use VLANd on devices with only a single network interface.

LAVA and locked switch/port combinations

VLANd supports locking particular switch/port combinations to prevent test jobs interfering with critical lab infrastructure (like a PDU or the dispatcher itself). The dispatcher is serving many jobs simultaneously, so cannot be part of any VLAN created by a test job.

The lava-vland protocol will not be allowed to modify locked switch/port combinations or to lock switch/port combinations used within the test job. LAVA will control the raising and tear down of VLANs using the lava-vland protocol, so that each test job gets access only to the VLANs that the test job itself defines.

VLANd and Multinode

  • VLANd is restricted to a single mapping of a switch and port to a single interface on a device
  • A VLAN which only ever contains a single device is not typically a useful test of the networking support on that device.
  • The multinode role determines which devices go onto which named VLAN.

So the lava-vland protocol is directly tied to the lava-multinode protocol, with one additional restriction:

  • Any role used by lava-vland must only set a count of one. There is no limit to the number of roles as long as each is unique across the multinode job.

LAVA VLANd database support

Details of which interface of which board is on which port of which switch is collectively called the network map which is maintained by the lab admins. See Administering VLANd support in LAVA.

Test writers get to see which types of device support which interfaces and which vland_device_tags, together with device tags. This allows test writers to specify which devices are used for a particular test, without being tied to a set of device hostnames that may change from time to time. LAVA then maps the test writer request to a specific device, interface and switch/port combination and constructs the commands to pass to VLANd.

Test writers do not provide explicit switch/port instructions; the test job simply defines the type of device to use, the interfaces to use and any device tags required. LAVA then assembles this into a series of instructions to VLANd. This allows test jobs to be re-used without regard to whether the lab admins have had to change the physical topology of the network, as long as the same services remain available.

Assigning roles to a VLAN

The name for the VLAN, as specified by the test writer, is an arbitrary label - the actual name used by VLANd will be calculated by LAVA based on the test job ID and the multinode target group ID. In a similar way to a role, the name is used to associate different roles onto the same VLAN.

Example vland protocol YAML

All uses of the lava-vland protocol also require the Multinode Protocol, this example just looks at the vland component.

lava-vland:
  client:
    vlan_one:
      - 10G
  server:
    vlan_one:
      - 1G

Any one role can be put onto multiple vlans. Managing the routing and specifying which interface is up or down at any particular point of a test job is entirely within the remit of the test writer:

lava-vland:
  master:
    vlan_one:
      - 10G
    vlan_two:
      - 1G
  slave:
    vlan_one:
      - 1G
  soldier:
    vlan_two:
      - 1G

Example YAML for the protocols

Combining the lava-vland protocol with the lava-multinode protocol shows how the roles match up.

Note

VLANd and Multinode support dictates that the count for roles which are used by lava-vland can only ever be 1.

This example will create a single VLAN which the test writer will be able to see as vlan_one and this VLAN will contain a single beaglebone-black and a single cubietruck. The beaglebone-black is required to provide at least one interface capable of a 10G link speed (so this example is unlikely to ever find a suitable device) and the cubietruck is required to provide an interface capable of 1G. (The actual meaning of the interface tags is up to the lab admins but it is expected that most admins will use the established convention of G === gigabit per second.) In addition, this example stipulates that the beaglebone-black is to support a device tag called usb-eth and the cubietruck is to support a device tag called sata. Depending on the setup of the lab, these tags can be used to indicate that the beaglebone-black has a USB ethernet converter as well as the on-board physical ethernet support and that the cubietruck has an accessible SATA drive.

protocols:
  lava-multinode:
    roles:
      client:
        device_type: bbb
        count: 1
        tags:
        - usb-eth
      server:
         device_type: cubietruck
        count: 1
        tags:
        - sata
    timeout:
      seconds: 60
  lava-vland:
    client:
      vlan_one:
        - 10G
    server:
      vlan_one:
        - 1G