Since I have not yet been able to resolve my complex networking issues in RHEL 5, I decided to upgrade 1 of my cluster nodes to 5.1 beta and take this new libvirt networking for a spin. I'm not sure if this is going to make my issue easier to resolve, or add to the complexity so I'm writing this e-mail to hopefully get some suggestions as how to proceed.
My network looks like this: eth2 & eth3 comprise bond1 bond1 has bond1.48 - my public VLAN interface with an IP for dom0 bond1 has bond1.20 - my VLAN for some domU systems, dom0 has no IP on this VLAN bond1 has bond1.21 - my VLAN for some domU systems, dom0 has no IP on this VLAN
I want to bridge to bond1.20 from some domU systems, and bond1.21 from other domU systems. dom0 should not have an IP on these VLANs.
1) do I need to define a network interface in /etc/libvirt/qemu/networks/ for each VLAN? (bond1.20 and bond1.21) 1a) If yes, do I need to have an IP address for the bond1.20 and bond1.21 interface defined in /etc/sysconfig/network-scripts/ifcfg-bond1.20 and bond1.21? 1b) Do I also need to define an IP address on that VLAN in the .xml file for that network?
or
2) Can I just have 1 default.xml network and then use iptables to forward to a specific VLAN based on bond1.20 and bond1.21 configured in /etc/sysconfig/network-scripts/ ??
#2 would be best, if possible. Unfortunately there's not a lot of documentation out on the new networking methods. I've read http://watzmann.net/blog/index.php/2007/04/27/networking_with_kvm_and_libvir... but that doesn't get in to all the VLANs and bonds.
Any help is greatly appreciated!
Wow, I didn't even know that /etc/libvirt/qemu/networks/ existed. Now I know how to remove that annoying 192.168 network. Thanks :)
I'm on F7 and not doing any bonding, but I am using bridges on VLANs and I setup all my interfaces and do my filtering the old fashioned way in /etc/sysconfig.
Sorry if I'm stating the obvious below...
[root@helix sysconfig]# grep forward /etc/sysctl.conf # Controls IP packet forwarding net.ipv4.ip_forward = 1
[root@helix sysconfig]# grep phys iptables -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
[root@helix network-scripts]# cat ifcfg-eth0 DEVICE=eth0 MTU=1504 BOOTPROTO=none ONBOOT=yes BRIDGE=br101
[root@helix network-scripts]# cat ifcfg-vlan6 DEVICE=eth0.6 BRIDGE=br6 VLAN=yes MTU=1500 ONBOOT=yes BOOTPROTO=none
[root@helix network-scripts]# cat ifcfg-br6 DEVICE=br6 TYPE=Bridge BOOTPROTO=static ONBOOT=yes MTU=1500 IPADDR=1.1.1.1 NETMASK=255.255.255.0 NETWORK=1.1.1.0 BROADCAST=1.1.1.255
And set (network-script /bin/true) in xend-config.sxp. Change your domU bridge like so:
# virsh shutdown <imgname> # virsh dumpxml <imgname> | sed s/br101/br6/ > <imgname>.xml # virsh define <imgname>.xml # virsh startup <imgname>
-- Dale Bewley - Unix Administrator - Shields Library - UC Davis GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD 1753 064D 2583 B098 A0F3
----- "Mark Nielsen" mnielsen@redhat.com wrote:
Since I have not yet been able to resolve my complex networking issues
in RHEL 5, I decided to upgrade 1 of my cluster nodes to 5.1 beta and
take this new libvirt networking for a spin. I'm not sure if this is going to make my issue easier to resolve, or add to the complexity so
I'm writing this e-mail to hopefully get some suggestions as how to proceed.
My network looks like this: eth2 & eth3 comprise bond1 bond1 has bond1.48 - my public VLAN interface with an IP for dom0 bond1 has bond1.20 - my VLAN for some domU systems, dom0 has no IP on
this VLAN bond1 has bond1.21 - my VLAN for some domU systems, dom0 has no IP on
this VLAN
I want to bridge to bond1.20 from some domU systems, and bond1.21 from
other domU systems. dom0 should not have an IP on these VLANs.
- do I need to define a network interface in
/etc/libvirt/qemu/networks/ for each VLAN? (bond1.20 and bond1.21) 1a) If yes, do I need to have an IP address for the bond1.20 and bond1.21 interface defined in /etc/sysconfig/network-scripts/ifcfg-bond1.20 and bond1.21? 1b) Do I also need to define an IP address on that VLAN in the .xml file for that network?
or
- Can I just have 1 default.xml network and then use iptables to
forward to a specific VLAN based on bond1.20 and bond1.21 configured in /etc/sysconfig/network-scripts/ ??
#2 would be best, if possible. Unfortunately there's not a lot of documentation out on the new networking methods. I've read http://watzmann.net/blog/index.php/2007/04/27/networking_with_kvm_and_libvir...
but that doesn't get in to all the VLANs and bonds.
Any help is greatly appreciated!
Dale Bewley wrote:
Wow, I didn't even know that /etc/libvirt/qemu/networks/ existed. Now I know how to remove that annoying 192.168 network. Thanks :)
I'm on F7 and not doing any bonding, but I am using bridges on VLANs and I setup all my interfaces and do my filtering the old fashioned way in /etc/sysconfig.
Sorry if I'm stating the obvious below...
Thanks for the help :) I had some issue with the BRIDGE= lines, the kernel was saying that it didn't support bridging. Rather than try to deviate any farther from the supported enterprise installation in place here, I just went back to RHEL 5 on the node I had upgrade to 5.1 and made all the bonded VLANs work with the Xen bridges. It's pretty complex, but I've developed a how-to that I'm hoping to get out soon. (if you need help before then, feel free to e-mail me).
Mark