ASR9K IPoE BNG and DHCP configuration.

Here is the complete configuration guide for ASR9K IPOE BNG, including support for PPPoE.

dhcp ipv4
profile ipoe proxy
allow-move
helper-address vrf LANDING 172.16.254.2 giaddr 0.0.0.0
broadcast-flag policy unicast-always
relay information option
relay information policy keep
relay information option allow-untrusted
relay information option remote-id enable
relay information option circuit-id enable
!
interface TenGigE0/2/0/23 proxy profile ipoe
interface TenGigE0/2/0/23.100 proxy profile ipoe
!

interface TenGigE0/2/0/23.100
vrf LANDING
ipv4 point-to-point
ipv4 unnumbered Loopback1111
arp learning disable
service-policy type control subscriber PPP-IPOE
pppoe enable bba-group PPPOE
ipsubscriber ipv4 l2-connected
initiator dhcp
!
encapsulation ambiguous dot1q 100 second-dot1q any
!

policy-map type control subscriber PPP-IPOE
 event session-activate match-first
  class type control subscriber PPP do-until-failure
   10 authenticate aaa list PPPOE
  ! 
 ! 
 event session-start match-first
  class type control subscriber PPP do-all
   10 activate dynamic-template PPP_TPL
  ! 
  class type control subscriber IPOE do-until-failure
   10 activate dynamic-template ipoe-profile
  ! 
 ! 
 end-policy-map

 dynamic-template
 type ipsubscriber ipoe-profile
  vrf LANDING
  ipv4 unnumbered Loopback1111
 !

class-map type control subscriber match-any IPOE
 match protocol dhcpv4 
 end-class-map

interface Loopback1111
vrf LANDING
ipv4 address 10.155.63.254 255.255.192.0

Debugging Commands:

General Checks:
“show drops all location all”
“show drops”
“show asic-errors all location 0/0/CPU0”
“show asic-errors all location 0/RSP0/CPU0”

Look for active PFM alarms on LC as well as RSP:
“show pfm location all”

Check FPGA Software Is Updated:
“show controllers np summary all”
“admin show hw-module fpd location all”

Check Hardware Diagnostics:
“admin show diagnostic result location all”

Clearing Counters:
“clear counters all”
“clear controller np counters all”
“clear controller fabric fia location”
“clear controller fabric crossbar-counters location”

Checking Interface Stats:
“show interface Gi0/0/0/0 | inc rate” ! Note that this will exclude Ethernet headers and include payload only

Check LC NP Stats:
“sh controller np ports all loc 0/0/cpu0”
“show controllers pm vqi location all”
“show controllers pm interface Te0/0/0/0”
“show controllers pm location 0/0/CPU0 | i “name|switch”
“show controllers np fabric-counters all all”
Some interesting counters for this command are:
xaui_a_t_transmited_packets_cnt — Num pkt sent by NPU to bridge
xaui_a_r_received_packets_cnt — Num pkt sent by bridge to NPU

When using “show controllers np fabric-counters all all location 0/0/CPU0”, all zero counts can be an indication there are Tx/Rx problems between the NP and FIA.

“show controllers np counters all”
Some interesting counters for this command are:
800 PARSE_ENET_RECEIVE_CNT — Num of packets received from external interface
970 MODIFY_FABRIC_TRANSMIT_CNT — Num of packets sent to fabric
801 PARSE_FABRIC_RECEIVE_CNT — Num of packets received from fabric
971 MODIFY_ENET_TRANSMIT_CNT — Num of packets sent to external interface

When using “show controller np counters all loc 0/0/CPU0” the output “No non-zero data counters found” for an NP can be an indication it has locked up.

Check LC FIA & Bridge Stats:
“show controllers fabric fia link-status location 0/0/CPU0”
“show controllers fabric fia stats location 0/0/CPU0”
“show controllers fabric fia q-depth location 0/0/CPU0”
“show controllers fabric fia drops ingress location 0/0/CPU0”
“show controllers fabric fia drops egress location 0/0/CPU0”
“show controllers fabric fia errors ingress location 0/0/CPU0”
“show controllers fabric fia errors egress location 0/0/CPU0”
“show controllers fabric fia bridge *” Trident LC Only

Check RSP Abriter and Xbar Stats:
“show controllers fabric arbiter serders location …”
“show controllers fabric crossbar link-status instance 0 location 0/RSP0/CPU0”
“show controllers fabric crossbar link-status instance 1 location 0/RSP0/CPU0”
“show controllers fabric crossbar statistics instance 0 location 0/RSP0/CPU0”
“show controllers fabric ltrace crossbar last 100 location all”

Check how the interface policy is applied in hardware:
“show qos capability location 0/0/cpu0”
“show qos interface Te0/0/2/1 output”
“show policy-map interface Te0/0/2/1 output”  ! Note that this will inlcude Ethernet SA+DA+ETYPE+VLANS+PAYLOAD+CRC
“show qoshal resource summary location 0/0/CPU0”
“show qoshal default-queue interface Te0/0/2/1”

# Reboot the machine soon after a kernel panic
kernel.panic=10

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Protects against creating or following links under certain conditions
fs.protected_hardlinks=1
fs.protected_symlinks=1

#Enable ExecShield protection
#Set value to 1 or 2 (recommended) 
kernel.exec-shield = 2
kernel.randomize_va_space=2

# increase system file descriptor limit    
fs.file-max = 65535

#Allow for more PIDs 
kernel.pid_max = 65536

#Disable zone reclaim
vm.zone_reclaim_mode = 0

#Reduce swap usage
vm.swappiness = 10

###############################################
########## IPv4 networking start ##############
###############################################

# Send redirects, if router, but this is just server
# So no routing allowed 
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

# Accept packets with SRR option? No
net.ipv4.conf.all.accept_source_route = 0

# Accept Redirects? No, this is not router
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 1

#Ignore bad ICMP errors
net.ipv4.icmp_ignore_bogus_error_responses=1

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# TCP window scaling tries to avoid saturating the network adapter with
# incoming packets.
net.ipv4.tcp_window_scaling = 1

# If enabled, assume that no receipt of a window-scaling option means that the
# remote TCP is broken and treats the window as a signed quantity.  If
# disabled, assume that the remote TCP is not broken even if we do not receive
# a window scaling option from it.
net.ipv4.tcp_workaround_signed_windows = 1

# TCP SACK and FACK refer to options found in RFC 2018 and are also documented
# back to Linux Kernel 2.6.17 with an experimental "TCP-Peach" set of
# functions. These are meant to get you your data without excessive losses.
net.ipv4.tcp_sack = 1
net.ipv4.tcp_fack = 1

# The latency setting is 1 if you prefer more packets vs bandwidth, or 0 if you
# prefer bandwidth. More packets are ideal for things like Remote Desktop and
# VOIP: less for bulk downloading.
#net.ipv4.tcp_low_latency = 0

# I found RFC 2923, which is a good review of PMTU. IPv6 uses PMTU by default
# to avoid segmenting packets at the router level, but its optional for
# IPv4. PMTU is meant to inform routers of the best packet sizes to use between
# links, but its a common admin practice to block ICMP ports that allow
# pinging, thus breaking this mechanism. Linux tries to use it, and so do I: if
# you have problems, you have a problem router, and can change the "no" setting
# to 1. "MTU probing" is also a part of this: 1 means try, and 0 means don't.
#net.ipv4.ip_no_pmtu_disc = 0
#net.ipv4.tcp_mtu_probing = 1

# FRTO is a mechanism in newer Linux kernels to optimize for wireless hosts:
# use it if you have them; delete the setting, or set to 0, if you don't.
#net.ipv4.tcp_frto = 2
#net.ipv4.tcp_frto_response = 2

# Log packets with impossible addresses to kernel log? yes
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.secure_redirects = 0

# Ignore all ICMP ECHO and TIMESTAMP requests sent to it via broadcast/multicast
net.ipv4.icmp_echo_ignore_broadcasts = 1

#Increase system IP port limits
net.ipv4.ip_local_port_range = 15000 65000

# Disable TCP slow start on idle connections
net.ipv4.tcp_slow_start_after_idle = 0

# Enable TCP/IP SYN cookies, see http://lwn.net/Articles/277146/
# Note: This may impact IPv6 TCP sessions too.
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 5

# Enable source validation by reversed path, as specified in RFC1812, which
# turn on Source Address Verification in all interfaces to prevent some
# spoofing attacks.
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1 

# RFC 1337, TIME-WAIT Assassination Hazards in TCP, a fix written in 1992
# for some theoretically-possible failure modes for TCP connections. To this
# day this RFC still has people confused if it negatively impacts performance
# or not or is supported by any decent router. Murphy's Law is that the only
# router that it would even have trouble with, is most likely your own.
net.ipv4.tcp_rfc1337 = 1

###############################################
########## IPv6 networking start ##############
###############################################

# Uncomment the next line to enable packet forwarding for IPv6.  Enabling this
# option disables Stateless Address Autoconfiguration based on Router
# Advertisements for this host.
#net.ipv6.conf.all.forwarding = 0

# Number of Router Solicitations to send until assuming no routers are present.
# This is host and not router
net.ipv6.conf.default.router_solicitations = 0

# Accept packets with SRR option? No
net.ipv6.conf.all.accept_source_route = 0

# Accept Router Preference in RA?
net.ipv6.conf.default.accept_ra_rtr_pref = 0

# Learn Prefix Information in Router Advertisement
net.ipv6.conf.default.accept_ra_pinfo = 0

# Setting controls whether the system will accept Hop Limit settings from a router advertisement
net.ipv6.conf.default.accept_ra_defrtr = 0

#router advertisements can cause the system to assign a global unicast address to an interface
net.ipv6.conf.default.autoconf = 0

#how many neighbor solicitations to send out per address?
net.ipv6.conf.default.dad_transmits = 0

# How many global unicast IPv6 addresses can be assigned to each interface?
net.ipv6.conf.default.max_addresses = 1

# Do not accept ICMP redirects (prevent MITM attacks)
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.all.secure_redirects = 1

############################################
##### TCP Tuning ###########################
############################################

# Increase Linux autotuning TCP buffer limits
# Set max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE
# Don't set tcp_mem itself! Let the kernel scale it based on RAM.
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Make room for more TIME_WAIT sockets due to more clients,
# and allow them to be reused if we run out of sockets
# Also increase the max packet backlog
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10
net.core.netdev_max_backlog = 60000
net.core.netdev_budget = 60000
net.core.netdev_budget_usecs = 6000

# If your servers talk UDP, also up these limits
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192

# Change Congestion Control Algorithm to BBR
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr