                          Broadcom Corporation
                          16215 Alton Parkway,
                          Irvine, CA 92619-7013
			  
                           Installation Notes
            Broadcom's Advanced Server Features Program For Netware
                              7/24/2007
                             Version 2.28

Table of Contents
=================

  Introduction
  Balance Modes and Limitations
  NESL Compliance
  Installation
  Using InetCfg
  Balance Mode Selection
  Loading Frame Types
  Hot Standby
  VLANs
  Jumbo Frames
  Additional Command Line Keywords
  Unloading
  
			       
Introduction
============

BASP.LAN is Broadcom's virtual ethernet driver for Netware 4.x and 5.x that
provides Load-balancing, fault-tolerance, and VLAN features. These features
are provided by creating teams (virtual adapters) that consist of multiple
NIC interfaces. A team can consist of one to six (eight on some systems) NIC 
interfaces and each interface can be designated primary or standby*. All 
primary interfaces in a team will participate in Load-balancing operations by 
sending and receiving a portion of the total traffic**. Standby interfaces will 
take over in the event that all primary interfaces have lost their links. 
VLANs can be added to a team to allow multiple VLANs with different VLAN IDs 
to share the virtual adapter.

Load-balancing and fault-tolerance features will work with any third party's
NIC adapters. VLANs only work with Broadcom or Alteon NIC adapters.


*Standby can only be used in Smart Load-Balance mode (See below).

**In 802.3ad mode, the number of NIC interfaces aggregated is automatically
determined through LACP (See below).


Balance Modes and Limitations
=============================

Smart Load-balance (SLB) is a protocol specific scheme and the level of
support for IP, IPX, and other protocols are listed below.
   

                    Load-balancing       Fault-tolerance
   -----------------------------------------------------
   IP                   Yes                   Yes
   IPX                  Yes*                  Yes**
   Other protocols      No                    Yes**

*Only outbound load-balancing for IPX.

**Only for Broadcom or Alteon NICs. Alteon's driver ALT.LAN must be version
2.05b or newer released by Broadcom.


Smart Load-balance (SLB) mode works with all ethernet switches without
configuring the switch ports to any special trunking mode. Only IP traffic
will be load-balanced in both inbound and outbound directions. IPX traffic
will be load-balanced in outbound direction only. Other protocol packets will
be sent and received through one primary NIC only. Fault-tolerance for
non-IP traffic is only supported using Broadcom or Alteon NICs.

The Generic Trunking mode requires the ethernet switch to support some
form of port trunking mode (e.g. Cisco's Gigabit EtherChannel or other
switch vendor's link aggregation mode). Trunking mode must be statically
configured on the switch ports that are connected to the team. This mode is
protocol-independent and all traffic should be load-balanced and
fault-tolerant.

802.3ad mode requires the ethernet switch to support 802.3ad with LACP
(Link Aggregation Control Protocol). LACP will try to configure the maximum
number of NICs in the team that are compatible for link aggregation. If LACP
determines that some NICs are not able to aggregate (because of some
restrictive limitations or configurations on the switch), the remaining NICs
that cannot aggregate will be idle. If LACP is completely disabled on the
switch, then only one of the NICs in the team will be used. Some switches
require the LACP ports to be in full duplex mode for them to work. LACP
can be configured active or passive on the team and most switches allow
active or passive selections on a per port basis. At least one side of
each connection must be active otherwise the connection will never be
selected for aggregation. This mode is also protocol-independent and all
traffic should be load-balanced and fault-tolerant.


NESL Compliance
===============

For optimum fault tolerance and recovery operations, BASP.LAN relies on
the NIC drivers to generate NESL (Netware Event Sevice Layer) events during
link changes and other failure events. NESL is an optional feature in the
ODI driver specification and not all drivers support it. For NESL events to
propagate properly to BASP.LAN, ODINEB.NLM must be loaded before the
NESL compliant ODI drivers.

Do the following to check if a NIC driver supports NESL events. Load BASP.LAN
and create a team by binding the NIC adapter to the virtual slot (See
instructions and examples below). In the "Virtual Adapter X Team Members"
screen of the BASP.LAN's menu interface, the Link status of all bound NIC
adapters are shown. Disconnect or connect the NIC adapter's cable and the
link status shown on the screen should change immediately if the NIC driver
supports NESL events.


Installation
============

1. Load BASP.LAN just like a standard LAN driver with all necessary frame
types for the team. BASP.LAN requires a special VSLOT parameter to specify
the virtual slot. The virtual slot can be viewed as a team number and the
valid range is from 1 to 4.

e.g.
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII VSLOT=1

2. Load the network drivers for the NIC adapters that will be part of the
team. The frame types loaded should be the same for all adapters in the team
and same as those loaded for BASP.LAN in step 1. Do not bind protocols
directly to these adapters. Be sure to load ODINEB.NLM (a Novell supplied
NLM) before all network drivers.

e.g.
        LOAD ODINEB.NLM
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_1_EII SLOT=1
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_2_EII SLOT=2

3. Bind BASP.LAN to the NIC adapters in the team by using a custom
BASP BIND command at the console.

e.g.
        BASP BIND BASP_1_EII B57_1_EII
        BASP BIND BASP_1_EII B57_2_EII

Note that if there are multiple frame types loaded on the virtual and the
physical adapters, it is only necessary to bind one frame type on the
virtual adapter to the same frame type on the physical adapter. The other
frame types will be automatically bound.

Alternatively, BASP can be bound to NIC adapters like a protocol using
the standard Netware BIND command. The protocol name for BASP is BASPn where
n is the vslot number.

e.g.
        BIND BASP1 B57_1_EII
        BIND BASP1 B57_2_EII
	
Note that if there are multiple frame types loaded on the virtual and the
physical adapters, it is only necessary to bind BASP to any one frame type
on the physical adapter. The other frame types will be automatically bound.


4. Bind protocols to BASP.LAN.

e.g.
        BIND IP BASP_1_EII ADDR=x.x.x.x MASK=x.x.x.x


Note that the recommended sequence is to load BASP.LAN before the network
drivers as outlined above. This allows BASP.LAN to determine the initial
link state of the bound adapters without delay.



Using InetCfg
=============

load inetcfg.nlm
<accept configuration, then - "Internetworking Configuration" window is
displayed>
1  select "Board" <enter>
2  press <ins> key
3  select physical device driver, i.e. "B57"
4  press <enter> then enter name, i.e. "B57_1"
5  press <Esc> key the select "yes" to save

Repeat steps 2 through 5 to configure another adapter, you may name 
subsequent boards "B57_2", "B57_3" etc.

6  press <ins> key
7  select virtual device driver, i.e. "BASP"
8  press <enter> then enter name, i.e. "BASP_1".  In this window,
   Virtual slot number, Balance Mode and VLAN ID can be selected

Note: 1. The first Virtual adapter (TEAM) should have Virtual slot of "1".
      2. VLAN "0" (zero) is a default and this VLAN is untagged.

9  press <Esc> key, then select "yes" to save
10 scroll down to "Protocols" and press <enter> to select
11 scroll down to "User-specified Protocol" and press <enter> to select
12 press <ins> key
13 enter the name for the Protocol,
   i.e. "BASP1" for TEAM 1, BASP2 for TEAM 2, BASP3 for TEAM 3 and
         BASP4 for TEAM 4.
14 press <Esc> keys to go back to "Internetworking Configuration" window
15 select "Bindings"
16 press <ins> to configure protocol
17 select "TCP/IP" then press <enter>
18 select "A Network Interface" then press <enter>
19 select virtual adapter i.e. BASP....
20 define IP address
21 press <Esc>, select "yes" to save

Repeat steps 16 through 21 if desired to configure IPX and multiple frame types.
Note: physical adapters must be configured with the same frame types
      as the Virtual TEAM.

22 press <ins> to bind the physical adapter to a virtual team
23 select "User-specified Protocol"
24 select "A Network Interface"
25 select a physical adapter i.e. B57.....
26 enter "BASP1" for Protocol, select desired frame type,
   enter special parameters if desired (please refer to B57.LAN readme.txt
   for parameters), and press <Esc> then select "yes" to save

Note: You may use names such as "BASP1" for TEAM 1, "BASP2" for TEAM 2, 
	  "BASP3" for TEAM 3 and "BASP4" for TEAM 4.

Repeat steps 22 through 26 to configure an additional physical driver, and to
configure additional frame types.

The configuration can be viewed at "View Configuration" - "All INETCFG
Commands".


Balance Mode Selection
======================

Use "MODE=SLB" for Smart Load-Balance mode, "MODE=TRUNK" for Generic
Trunking mode, or "MODE=802.3AD" for 802.3ad mode. The default is SLB.

e.g.
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII VSLOT=1 MODE=TRUNK
	
In 802.3ad mode, untagged ethernet II frame type must be loaded before LACP
frames can be transmitted and received. LACP will default to active for all
NICs in the team. Use the parameter "LACP=PASSIVE" to change LACP to passive
mode for all NICs in the team. Note that at least one side (server or switch)
must be in LACP active mode for it to work.

e.g.
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII VSLOT=1 MODE=802.3AD
        LACP=PASSIVE



Loading Frame Types
===================

After one or more NIC adapters are bound to a virtual adapter, additional
frame types can only be loaded in the virtual adapter if the corresponding
frame types are also loaded in the bound adapters. For example,
ETHERNET_802.2 can be loaded in BASP VSLOT 1 if ETHERNET_802.2 are loaded
in B57.LAN SLOT 1 and 2 in the above example. 

Similarly, a virtual adapter can only be bound to a physical adapter if
the physical adapter has all the frame types loaded in the virtual adapter.


Hot Standby
===========

In Smart Load-Balance (SLB) mode, one or more NIC adapters can be designated
as hot standbys. Use the keyword "STANDBY" in the BASP BIND command to
indicate binding a NIC adapter as a hot standby.

e.g.
        BASP BIND BASP_1_EII B57_1_EII
        BASP BIND BASP_1_EII B57_2_EII STANDBY
	
In the above example, B57_1_EII and B57_2_EII are bound as primary and
hot standby adapters respectively. Note that standby is only valid for
Smart Load-Balance mode.


VLANs
===== 

To add VLANs to a team, do the following:

1. Load BASP.LAN with the all necessary frame types and specify the
VLAN ID for each frame type. You can specify a maximum of 64 VLAN IDs and
each VLAN ID can be loaded up to 4 times with 4 different frame types.

e.g. VLAN ID 2 for Ethernet II:

        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_V2_EII VSLOT=1 VLAN=2
	
2. Load the network drivers for the NIC adapters in the team with all the
frame types specified in step 1. Note that the one or more VLAN IDs
specified in step 1 do not have to be specifed when loading the network
drivers. And each frame type loaded in step 1 only needs to be loaded once
for each network driver even if it is loaded multiple times with different
VLAN IDs in step 1. Only Broadcom and Alteon NIC adapters can be used.
ALT.LAN must be version 2.05b or newer released by Broadcom and must include
the keyword FORVLANS.

e.g. Broadcom adapters:

        LOAD ODINEB.NLM
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_1_EII SLOT=1
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_2_EII SLOT=2

or Alteon adapters:

        LOAD ODINEB.NLM
        LOAD ALT.LAN FRAME=ETHERNET_II NAME=ALT_1_EII SLOT=1 FORVLANS
        LOAD ALT.LAN FRAME=ETHERNET_II NAME=ALT_2_EII SLOT=2 FORVLANS
	
3. Bind BASP.LAN to the NIC adapters in the team.

e.g.
        BASP BIND BASP_1_V2_EII B57_1_EII
        BASP BIND BASP_1_V2_EII B57_2_EII
	
Note that if there are multiple VLANs (each with one or more frame types)
loaded on the virtual adapter, it is only necessary to bind one frame type
on one VLAN on the virtual adapter to the same frame type on the physical
adapter. The other VLANs will be automatically bound.

	
4. Bind protocols to BASP.LAN.

e.g.
        BIND IP BASP_1_V2_EII ADDR=x.x.x.x MASK=x.x.x.x
  
 
This example creates a team with 2 adapters using VLAN ID 2. Outbound
packets will be tagged with VLAN ID 2 and only similarly tagged packets
will be received by the NIC adapters in the team. Additional VLANs
with different VLAN IDs can be created in the same team. The Maximum number
of VLANs per virtual slot is 64. The valid range of VLAN IDs is from 1 to
4094. VLAN=0 indicates the VLAN is untagged and is the default. Use decimal
numbers to specifiy the VLAN ID.


Jumbo Frames
============

Jumbo Frames are supported in all balance modes. The maximum frame size will
be automatically set to the smallest maximum frame size of all NICs in the
team. Use appropriate keywords to enable jumbo frames when loading the NIC
drivers.

e.g.
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII VSLOT=1
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_1_EII SLOT=1 JUMBO=9000
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_2_EII SLOT=2 JUMBO=9000
        BASP BIND BASP_1_EII B57_1_EII
        BASP BIND BASP_1_EII B57_2_EII
	

Additional Command Line Keywords
================================

CHECKSUM=ON

        Enables BASP.LAN to offload TCP/UDP and IP checksums to the bound
        NIC adapters if supported. This will improve performance if some or
        all NIC adapters in the team support hardware checksums. Be sure to
        load the NIC drivers with hardware checksums enabled.
	
        e.g., a team of two BCM5700 NICs with hardware checksums enabled.
	
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_1_EII CHECKSUM=ON SLOT=1
        LOAD B57.LAN FRAME=ETHERNET_II NAME=B57_2_EII CHECKSUM=ON SLOT=2
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII CHECKSUM=ON VSLOT=1
        BASP BIND BASP_1_EII B57_1_EII
        BASP BIND BASP_1_EII B57_2_EII


NOSCREEN

        Disables the menu-driven screen when BASP.LAN is loaded for the first
        time.
	
        e.g.
	
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_EII VSLOT=1 NOSCREEN
	
	
GVRP

        Enables GVRP (Garp VLAN Registration Protocol) for the VLAN that is
        loaded. An untagged 802.2 frame type must be loaded in the virtual
        adapter and all bound physical adapters for GVRP to take effect.
        This is necessary because GVRP uses untagged 802.2 frames to advertise
        VLAN memberships. Use VLAN=0 FRAME=ETHERNET_802.2 in the LOAD command
        to specify untagged 802.2 frame type.
	
        e.g.
	
        LOAD BASP.LAN FRAME=ETHERNET_II NAME=BASP_1_V2_EII VSLOT=1 VLAN=2
        GVRP
        LOAD BASP.LAN FRAME=ETHERNET_802.2 NAME=BASP_1_V2_EII VSLOT=1 VLAN=0


Unloading
=========

BASP.LAN can only be unloaded from the console command line after all
bound adapters in every VSLOT have been unbound.


