Purpose
This document wants to give some suggestions and hints on how to configure equipment directly connected to MIX L2 switches. We will refer to configuration scenarios described in MIX-302 document. Some configuration examples are available for the most popular vendors grouped by equipment type (Layer 2 Switches, Routers and Layer 3 Switches) and also some guidance on how to configure Link Aggregation on multiple peering ports.
Switch L2 Configuration Examples
The following examples are referred to the ‘Member Switch’ Equipment in the picture above.
Switch Cisco Catalyst 29xx e 35xx
When connecting a Catalyst switch to MIX the following protocols/services must be disabled:
- VTP (VLAN Trunking Protocol)
- DTP (Dynamic Trunking Protocol)
- LLDP (Link Layer Discovery Protocol)
- UDLD (Unidirectional Link Detection)
Type in the following commands in global config mode (IOS sw)
vtp mode transparent
!
no spanning-tree vlan 100
! global disable of LLDP
no lldp run
! Global dsable of CDP
no cdp run
!
vlan 100
name MIX
!
interface /IfIdent/
description MIX Interface
switchport access vlan 100
switchport mode access
switchport nonegotiate
no keepalive
speed nonegotiate
no udld enable
! If CDP can/has not been disabile globally
no cdp enable
! If LLDP can/has not been disabile globally
no lldp receive
no lldp transmit
! If Spanning Tree can’t be disabile globally
spanning-tree bpdufilter enable
end
Cisco Catalyst 6500 Family
CatOS software is still very used on such equipment: the folowing commands apply:
set vtp mode off
set port name /IfIdent/ Porta MIX
set cdp disable /IfIdent/
set udld disable /IfIdent/
set trunk /IfIdent/ off dot1q
set spantree bpdu-filter /IfIdent/ enable
set vlan 100 name VLAN su MIX
set vlan 100 /IfIdent
If VTP can not be disabile globally the only known workaround is using l2pt thus disabling it on a per port basis:
set port l2protocol-tunnel /IfIdent/ vtp enable
Keep attention: this command could not be available on certain CatOS releases.
Extreme Networks L2 Switch
The configuration fragment which follows is related to a L2 Switch directly connected to MIX belonging also to a EAPS ring. In the example, port 1 is connected to MIX, and port 2 and 3 belong to the ring. Teh peering router is connected to the ring on the MIX Vlan. All ports are Gigabit Ethernet.
create vlan “ring”
configure vlan “ring” tag 1700 # VLAN-ID=0x6a4 Global Tag 9
configure vlan “ring” qosprofile “QP8”
configure vlan “ring” add port 2 tagged
configure vlan “ring” add port 3 tagged
create vlan “mix”
configure vlan “mix” tag 1200 # VLAN-ID=0x4b0 Global Tag 3
configure vlan “mix” add port 1 untagged
configure vlan “mix” add port 2 tagged
configure vlan “mix” add port 3 tagged
configure port 1 auto off speed 1000 duplex full
configure port 2 auto off speed 1000 duplex full
configure port 3 auto off speed 1000 duplex full
disable edp port 1
disable igmp snooping
disable igmp snooping with-proxy
create eaps “ring-eaps”
configure eaps “ring-eaps” mode transit
configure eaps “ring-eaps” primary port 2
configure eaps “ring-eaps” secondary port 3
configure eaps “ring-eaps” add control vlan “ring”
configure eaps “ring-eaps” add protect vlan “mix”
enable eaps “ring-eaps”
Brocade L2 Switch
Here follows a configuration fragment for a Brocade BigIron switch used as access device towards MIX LAN. In this example the Peering Router is connected to another porto f thesame switch.
! Define ua VLAN for MIX port and the peering Router
vlan number name “MIX” by port
no spanning-tree
untagged ethernet if-verso-switch-MIX
untagged ethernet if-verso-router-peering
Switch L3 / Peering Router Configuration Hints
Here follows some configuration templates for Layer 3 switches or Routers acting as Peering Border Router, according to the scheme visible in the picture above.
Cisco
The following configuration fragmnts aims at disabling all functionalities which generate un wanted and undesirable traffic over MIX Peering LAN;
Autoconfiguration Protocol
- DHCP
- BOOTP
- TFTP of the configuration through MIX
Other Protocols and Services which traffic must not be sent through MIX LAN
- CDP
- DEC MOP
- IP redirects
- IP directed broadcasts
- proxy ARP
- IPv6 RouterAdvertisements
- L2keepalive
Global Configuration
! Remove DHCP
no service dhcp!Older Release may require instead: ip bootp server
! NDo not allow Tftp download of configuration
no service config
! Global Disable of Cisco Discovery Protocol (CDP)
no cdp run
Interface Configuration
! Disable IP redirects
no ip redirects
! Disable proxy ARP
no ip proxy-arp
! If CDP can not be disabled globally do it on MIX port
no cdp enable
! Disable directed broadcasts
no ip directed-broadcast
! If DEC/MOC can not be disabled globally do it on MIX
no mop enable
! (Fast)Ethernet Ports: no auto-negotiation.
no negotiation auto
! fix the duplex mode
duplex full
! L2 keepalives does not have reason to exist on MIX peering LAN
no keepalive
Extreme Networks
The folIowing example shows the details of a L3 router/switch connected through port 1 on vlan ‘mix’ (untagged).
# Config information for VLAN mix. # create vlan “mix” configure vlan “mix” tag 1200 configure vlan “mix” protocol “IP” configure vlan “mix” ipaddress 217.29.66./Y/ 255.255.254.0 configure vlan “mix” add port 1 untagged # configure port 1 display-string “MIX” disable edp port 1 # enable ipforwarding vlan “mix” disable ipforwarding broadcast vlan “mix” disable ipforwarding fast-direct-broadcast vlan “mix” disable ipforwarding ignore-broadcast vlan “mix” disable ipforwarding lpm-routing vlan “mix” disable isq vlan “mix” disable irdp vlan “mix” disable icmp unreachable vlan “mix” disable icmp redirects vlan “mix” disable icmp port-unreachables vlan “mix” disable icmp time-exceeded vlan “mix” disable icmp parameter-problem vlan “amsix” disable icmp timestamp vlan “mix” disable icmp address-mask vlan “mix” disable subvlan-proxy-arp “mix” configure ip-mtu 1500 vlan “mix” # IP Route Configuration # configure iproute add blackhole default disable icmpforwarding vlan “mix” disable igmp vlan “mix”
Force 10
This example shows a configuration fragment for a router/switch L3 Force10 connected to MIX through a 10 Gbit port.
! Disable proxy-arp over MIX interface
Force10(conf)#interface tengigabitethernet 0/0
Force10(conf-if-te-0/0)#no ip proxy-arp
! Disable Ipv6 ND RA
Force10(conf-if-te-0/0)#ipv6 nd suppress-ra
! ARP timeout is 4 hrs by default but can be changed
Force10(conf)#interface tengigabitethernet 0/0
Force10(conf-if-te-0/0)#arp timeout /minutes/
Brocade
Follows a configuration fragment for a Layer 3 Switch Brocade directly connected to MIX.
! MIX interface configuration
interface ethernet if
port-name “MIX”
! Activate the port at Layer 3 only
route-only
no spanning-tree
! Disable IPv6 ND-RA (Router Advertisements)
ipv6 nd suppress-ra
! Disable VLAN auto discovery.
no vlan-dynamic-discovery
! IP address
ip address 216.29.6X.Y 255.255.254.0
! No redirects
no ip redirect
no ipv6 redirect
! MIX recommends at least 2 hrs as ARP timeouts value
ip arp-age 120
! fast-ethernet ports: fix speed and duplex
speed-duplex 100-full
In some Ironware releases default settings for ICMPv6 ND was at 1 second, this value ha sto be changed to a much reasonable value of 1 hr, with the following command:
!Se IPv6 nd timeout at 1 hr.
ipv6 nd ns-retransmit 3600
Juniper
For Juniper routers be sure that you are announcing only unicast routes overMIX peering LAN by adding the following command to all neighbor, groups and prefix-limits:
set family inet unicast
Even just one neighbor configured with family inet ‘any’ will activate multi cast routing and MBGP over the peering LAN
IPv4 ARP Cache Timeout
Juniper ARP cache timeout is 20 minutes: to reduce the amount of ARP broadcast traffic we recommend to raise this value up to 4 hrs. Follow the configuration commands to achieve this.
> “configure”
Entering configuration mode
[edit]
you@juniper# edit system arp
[edit system arp]
you@juniper# set aging-timer 240
[edit system arp]
you@juniper# show | compare
[edit system arp]
aging-timer 240;
[edit system arp]
you@juniper# commit and-quit
commit complete
Exiting configuration mode
Starting from release JUNOS 9.4 ARP cache timeout is configurable on each single interface:
[edit system arp aging-timer interface interface-name] [edit system arp aging-timer interface interface-name] aging-timer-minutes;aging-timer-minutes;
Link Aggregation on Multiple MIX Ports
Link Aggregation (LAG) is available as a solution on MIX Peering LAN, according to the scheme visible in the picture below. To finalize this configuration is always mandatory to contact MIX Technical Department. Somo guideleines on how to configure it on the member side follow.
The service is available on Gigabit and 10 Gigabit Ethernet ports, up to a maximum of 8.
Cisco Catalyst 6500
The Port Channel on the Cisco side must be configured in mode on, and not as negotiate or desirable. MIX switches does not enable LACT or PaGP as default: LACP is possible asking this feature to the MIX Technical Department. Some Interface modules may have limitations in the amount of traffic injected on the LAG.
Please verify your technical documentation or ask the vendor. MIX static MAC address assignment might be a problem so contact MIX Technical Department. MIX static MAC address assignment might be a problem so contact MIX Technical Department.
! Port Channel MIX: Config Example
interface GigabitEthernet1/1
description MIX Link 1
no ip address
no ip redirects
no ip proxy-arp
no keepalive
no cdp enable
channel-group 1 mode on
!
interface GigabitEthernet1/2
description MIX Link 2
no ip address
no ip redirects
no ip proxy-arp
no keepalive
no cdp enable
channel-group 1 mode on
!
interface Port-channel1
description MIX aggregated link
ip address 217.29.6x.y 255.255.254.0
no ip redirects
no ip proxy-arp
no keepalive
!
Cisco GSR
MIX static MAC address assignment could be a problem with Cisco GSR: keep in touch with MIX Technical Department in this case
! MIX Port Channel config example:
!
interface Port-channel1
description MIX Port Channel
ip address 217.29.6x.y 255.255.254.0
no ip redirects
no ip directed-broadcast
no ip proxy-arp
channel-group minimum active 1
no channel-group bandwidth control-propagation
hold-queue 150 in
!
interface GigabitEthernet1/2/1
no keepalive
no negotiation auto
channel-group 1
no cdp enable
!
interface GigabitEthernet1/2/2
no keepalive
no negotiation auto
channel-group 1
no cdp enable
!
Switch L3 Brocade
Old platforms like BigIron Jetcore and IronCore have limitations on the LAG port configuration. Please check your technical documentation. On BiGIron 15000 slot n. 8 is not available for LAG together with his neighbouring slots.
! BigIron JetCore LAG configuration example
trunk server ethernet slot/port to slot/port+1
BigIron RX e MLX/XMR does not have any limitation on LAG ports.
! RX/MLX/XMR MIX port channel config example
trunk ethe slot/port to slot/port ethe otherslot/otherport to otherslot/otherport
Juniper M-Series
There are no nown problem with LAGs configuration starting from JuonOS release 6.0. A configuration example follows:
—
[edit]
ops@junix# show chassis
aggregated-devices {
ethernet {
device-count 1;
}
}
—
[edit]
ops@junix# show interfaces ge-2/1/0
gigether-options {
802.3ad ae0;
}
[edit]
ops@junix# show interfaces ge-3/1/0
gigether-options {
802.3ad ae0;
}
—
[edit]
ops@junix# show interfaces ae0
description “MIX”;
unit 0 {
family inet {
filter {
input MIX-in;
output MIX-out;
}
address 217.29.6x.y/23;
}
family inet6 {
address 2001:7F8:B:100:1D1:A5Dx:xxxx:y/64;
}
}
Optionally a more fine grained load balancing strategy is available:
#
routing-options {
autonomous-system abcde;
forwarding-table {
export [ load-balance ];
}
}
policy-options {
policy-statement load-balance {
then {
load-balance per-packet;
}
}
}
forwarding-options {
hash-key {
family inet {
layer-3;
layer-4;
}
}
}
Just in case this should not be sufficient the hashing algorithm can be modified through some undocumented options introduced starting from JunOS 7.0
—
hash-key {
family inet {
layer-3 {
destination-address;
protocol;
source-address;
}
layer-4 {
destination-port;
source-port;
type-of-service;
}
}
}
Also the minimum number of active links is configurable: when the minimum number of active links is reached the aggregation will be removed, being the LAG not able anymore to carry all the expected traffic.
—
aggregated-ether-options {
minimum-links 2;
link-speed 1g;
}
—
Tagged ports configuration on MIX
Mix ports can be also configured to be active on multiple VLANS other than the public Peering broadcast domain, followiing the 802.1Q standard. To accomplish this task and gain access to this functionality that allows the implementation of different type of services over the Mix ports the contact and coordination with the MIX NOC is always a requirement, after the subscription of the additional service(s) with the MIX Sales team. Find enclosed in the following some technical guidelines on how to configure this functionality. There are many scenarios where Q-tagging can be employed, here follows some of the most common uses. Additional information and case-by case detail will be provided by the NOC upon configuration of the service. Again, here are presented the most common scenarios:
- Tagged peering port over two VLANs, one the main Public Peering VLAN and another VLAN to interconnect privately with another member or a Closed User Group of several members ( each of them must have subscribed the service for the Q-tagging of the ports involved )
- Interconnection ports for direct peering configured exclusively on dedicated VLANs3.
- Access ports to other services provided by other MIX Members in appropriately segregated environments (private VLANs)
The following are the configuration outlines for the most common Vendors
Cisco Configuration Commands
int gig 1.100
encap dot1q 100
description – Mix Milano –
ip addr 217.29.6x.y 255.255.254.0
int gig 1.501
description – MIX OHM –
encap dot1q 501
ip addr 185.1.186.x 255.255.255.0
Juniper Configuration Commands
user@host# set interfaces ge-0/0/0 vlan-tagging
user@host# set interfaces ge-0/0/0 unit 0 alias “MIX-MILANO”
user@host# set interfaces ge-0/0/0.0 vlan-id 100
user@host# set interfaces ge-0/0/0.0 family inet address 217.29.6x.y/23
user@host# set interfaces ge-0/0/0 unit 1 alias “MIX-OHM”
user@host# set interfaces ge-0/0/0.1 vlan-id 501
user@host# set interfaces ge-0/0/0.1 family inet address 185.1.186.x/24
Mikrotik Configuration Commands
/interface vlan
add interface=sfp-sfpplus1 name=MIX-MILANO vlan-id=100
/ip address
add address=217.29.6x.y/23 interface=MIX-MILANO network=217.29.66.0
/interface vlan
add interface=sfp-sfpplus1 name=MIX-OHM vlan-id=501
/ip address
add address=185.1.186.x/24 interface=MIX-OHM network=185.1.186.0
BGP Routing between Tagged ports (VLAN MIX and other MIX managed IXes)
In this Chapter the operational scenario of a Peering Port connected at the MIX Open Hub Med POP in Carini (PA) or at Mxi Bologna PoP and active both on the MIX Milan public Peering LAN and the local Public Peering VLAN is assessed. At the moment for this configuration the subscription of MIX additional service is not requested, so this can be done directly by sending a configuration request directly to the MIX NOC.Remote Local Peering VLANs are not extended to any other MIX PoP, being introduced to define a precise geographical location for the peering interconnection dedicated to the traffic being originated and terminated locally.
For further information regarding the 802.1Q configuration of such a port please refer to the instruction provided in the previous Chapter.
As a general rule, MIX Remote VLANs like MIX-OHM or MIX-BO ones must be used as the primary traffic exchange route between members both connected in OHM. As an example in the following picture a basic scenario is presented: ISP1 has multiple MIX connections, one in Milano and one in Palermo while ISP2 is just connected at MIX OHM premises. In normal condition all the traffic between ISP 1 and ISP must use Yellow WLAN and thus follw the green dashed line. When a fault occurs to ISP1 OHM PoP port, the traffic can be routed over red VLAN (Mix Milano Peering VLAN) over the orange dashed path. Same scenario applies if Mix Bologna is involved instead of MIX Palermo.
In this scenario the requested routing behaviour is obtainable by ISP2 raising the LP (Local Preference) attribute over the MIX OHM VLAN received announcements coming from ISP1, and by ISP1 in the same way around, raising ISP2 announcement LP over the MIX OHM VLAN. Upon a ISP1 port fault in OHM, the traffic between ISP1 and ISP2 will traverse the MIX Milano Peering VLAN going back to normal upon ISP1 OHM port restoration.