openstack中开启sriov ######################## 主体思想：先在操作系统层面修改配置文件，再去Openstack里面修改配置文件 compute node 操作系统层 ========================================== * 确认是否含有ixgbe moudle :: lsmod |grep ixgbe * 增加sriov的信息: :: vi /etc/default/grub [root@compute1-bay5 default]# cat grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet intel_iommu=on ixgbe.max_vfs=16" GRUB_DISABLE_RECOVERY="true" * 重新生成grub文件: :: grub2-mkconfig -o /boot/grub2/grub.cfg .. note:: 会覆盖原有grub.cfg文件，如有需要可以做下备份 :: [root@compute3-bay7 default]# grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... Found linux image: /boot/vmlinuz-3.10.0-514.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-514.el7.x86_64.img Found linux image: /boot/vmlinuz-0-rescue-70b8529f43024bcf8949e45e7a9140a2 Found initrd image: /boot/initramfs-0-rescue-70b8529f43024bcf8949e45e7a9140a2.img done * 所有compute node 修改完成后重启 安装sriov ========================================== :: [root@compute2-bay9 ~]# yum search openstack|grep sriov openstack-neutron-sriov-nic-agent.noarch : Neutron SR-IOV NIC agent [root@compute2-bay9 ~]# yum -y install openstack-neutron-sriov-nic-agent.noarch 安装完成后会在/etc/neutron/plugins/ml2/下生成sriov_agent.ini的配置文件，修改增加如下内容: :: [sriov_nic] # # From neutron.ml2.sriov.agent # # Comma-separated list of : tuples mapping # physical network names to the agent's node-specific physical network device # interfaces of SR-IOV physical function to be used for VLAN networks. All # physical networks listed in network_vlan_ranges on the server should have # mappings to appropriate interfaces on each agent. (list value) physical_device_mappings =physnet1:eno1,physnet4:ens2f1 #根据项目情况进行网卡的选择 安装完sriov之后，默认服务是disable的，可以通过命令`systemctl list-unit-files|grep sriov`查看, 通过命令`systemctl enable neutron-sriov-nic-agent`使服务可用，然后在start下`systemctl start neutron-sriov-nic-agent`, 开启之后要在每个compute node上创建VFs以便支持SRIOV的功能，所以要在nova.conf文件内增加pci白名单: :: #pci_passthrough_whitelist = pci_passthrough_whitelist = [{"devname":"eno1", "physical_network":"physnet1"},{"devname":"ens2f1", "physical_network":"physnet4"}] 这个白名单是根据上文中映射关系来决定的，采用全部添加策略 创建完白名单之后，修改参与sriov功能的网卡，参考结果如下: :: [root@compute1-bay5 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 TYPE=Ethernet BOOTPROTO=none //初始为dhcp DEFROUTE=yes PEERDNS=yes PEERROUTES=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=eno1 UUID=3ae393ff-318c-4f83-b5a8-2dab3e887cab DEVICE=eno1 ONBOOT=yes //初始为no` 所有compute node弄完之后在control node上重启nova以及neutron: 修改neutron node ========================================== 修改nova的conf文件，增加filters字段如下： :: scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,CoreFilter,PciPassthroughFilter 修改ml2_conf.ini文件 增加字段如下： :: mechanism_drivers =openvswitch,sriovnicswitch [ml2_sriov] supported_pci_vonder_devs=8086:10ed agent_required = True .. note:: supported_pci_vonder_devs=8086:10ed 中的8086:10ed为因特尔网卡的代码，可以通过如下命令查看：lspci -nn|grep Ethe 查看开启与否的命令如下： :: ip link show|more [root@compute3-bay7 ~]# ip link show|more 1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eno1: mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether 38:ea:a7:17:01:5c brd ff:ff:ff:ff:ff:ff vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off vf 15 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off