3cd4b6c919
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
SuvarnaMeenakshi
|
90dc254656
|
[SNMP]: Modify minigraph parser to update SNMP_AGENT_ADDRESS_CONFIG table (#17045)
#### Why I did it SNMP query over IPv6 does not work due to issue in net-snmp where IPv6 query does not work on multi-nic environment. To get around this, if snmpd listens on specific ipv4 or ipv6 address, then the issue is not seen. We plan to configure Management IP and Loopback IP configured in minigraph.xml as SNMP_AGENT_ADDRESS in config_db., based on changes discussed in https://github.com/sonic-net/SONiC/pull/1457. ##### Work item tracking - Microsoft ADO **(number only)**:26091228 #### How I did it Modify minigraph parser to update SNMP_AGENT_ADDRESS_CONFIG with management and Loopback0 IP addresses. Modify snmpd.conf.j2 to use SNMP_AGENT_ADDRESS_CONFIG table if it is present in config_db, if not listen on any IP. Main change: 1. if minigraph.xml is used to configure the device, then snmpd will listen on mgmt and loopback IP addresses, 2. if config_db is used to configure the device, snmpd will listen IP present in SNMP_AGENT_ADDRESS_CONFIG if that table is present, if table is not present snmpd will listen on any IP. #### How to verify it config_db.json created from minigraph.xml for single asic VS image with mgmt and Loopback IP addresses. ``` "SNMP_AGENT_ADDRESS_CONFIG": { "10.1.0.32|161|": {}, "10.250.0.101|161|": {}, "FC00:1::32|161|": {}, "fec0::ffff:afa:1|161|": {} }, ..... snmpd listening on the above IP addresses: admin@vlab-01:~$ sudo netstat -tulnp | grep 161 tcp 0 0 127.0.0.1:3161 0.0.0.0:* LISTEN 71522/snmpd udp 0 0 10.250.0.101:161 0.0.0.0:* 71522/snmpd udp 0 0 10.1.0.32:161 0.0.0.0:* 71522/snmpd udp6 0 0 fec0::ffff:afa:1:161 :::* 71522/snmpd udp6 0 0 fc00:1::32:161 :::* 71522/snmpd ``` |
||
|
SuvarnaMeenakshi
|
9864dfeaa1
|
[SNMP][IPv6]: Fix SNMP IPv6 reachability issue in certain scenarios (#15487)
Modify snmpd.conf to start snmpd to listen on specific management and loopback ips instead of listening on any ip. #### Why I did it SNMP over IPv6 is not working for all scenarios for a single asic platforms. The expectation is that SNMP query over IPv6 should work over Management or Loopback0 addresses. **Specific scenario where this issue is seen** In case of Lab T0 device, when SNMP request is sent from a directly connected T1 neighbor over Loopback IP, SNMP response was not received. This was because the SRC IP address in SNMP response was not Loopback IP, it was the PortChannel IP connected to the neighboring device. ``` 23:18:51.620897 In 22:26:27:e6:e0:07 ethertype IPv6 (0x86dd), length 105: fc00::72.41725 > **fc00:1::32**.161: C="msft" **GetRequest**(28) .1.3.6.1.2.1.1.1.0 23:18:51.621441 Out 28:99:3a:a0:97:30 ethertype IPv6 (0x86dd), length 241: **fc00::71**.161 > fc00::72.41725: C="msft" **GetResponse**(162) .1.3.6.1.2.1.1.1.0="SONiC Software Version: SONiC.xxx - HwSku: xx - Distribution: Debian 10.13 - Kernel: 4.19.0-12-2-amd64" ``` In case of IPv4, the SRC IP in SNMP response was correctly set to Loopback IP. ``` 23:25:32.769712 In 22:26:27:e6:e0:07 ethertype IPv4 (0x0800), length 85: 10.0.0.57.56701 > **10.1.0.32**.161: C="msft" **GetRequest**(28) .1.3.6.1.2.1.1.1.0 23:25:32.975967 Out 28:99:3a:a0:97:30 ethertype IPv4 (0x0800), length 221: **10.1.0.32**.161 > 10.0.0.57.56701: C="msft" **GetResponse**(162) .1.3.6.1.2.1.1.1.0="SONiC Software Version: SONiC.xxx - HwSku: xx - Distribution: Debian 10.13 - Kernel: 4.19.0-12-2-amd64" ``` **Sequence of SNMP request and response** 1. SNMP request will be sent with SRC IP fc00::72 DST IP fc00:1::32 2. SNMP request is received at SONiC device is sent to snmpd which is listening on port 161 :::161/ 3. snmpd process will parse the request create a response and sent to DST IP fc00::72. snmpd process does not track the DST IP on which the SNMP request was received, which in this case is Loopback IP. snmpd process will only keep track what is tht IP to which the response should be sent to. 4. snmpd process will send the response packet. 5. Kernel will do a route look up on destination IP and find the best path. ip -6 route get fc00::72 fc00::72 from :: dev PortChannel101 proto kernel src fc00::71 metric 256 pref medium 5. Using the "src" ip from about, the response is sent out. This SRC ip is that of the PortChannel and not the device Loopback IP. The same issue is seen when SNMP query is sent from a remote server over Management IP. SONiC device eth0 --------- Remote server SNMP request comes with SRC IP <Remote_server> DST IP <Mgmt IP> If kernel finds best route to Remote_server_IP is via BGP neighbors, then it will send the response via front-panel interface with SRC IP as Loopback IP instead of Management IP. Main issue is that in case of IPv6, snmpd ignores the IP address to which SNMP request was sent, in case of IPv6. In case of IPv4, snmpd keeps track of DST IP of SNMP request, it will keep track if the SNMP request was sent to mgmt IP or Loopback IP. Later, this IP is used in ipi_spec_dst as SRC IP which helps kernel to find the route based on DST IP using the right SRC IP. https://github.com/net-snmp/net-snmp/blob/master/snmplib/transports/snmpUDPBaseDomain.c#L300 ipi.ipi_spec_dst.s_addr = srcip->s_addr Reference: https://man7.org/linux/man-pages/man7/ip.7.html ``` If IP_PKTINFO is passed to sendmsg(2) and ipi_spec_dst is not zero, then it is used as the local source address for the routing table lookup and for setting up IP source route options. When ipi_ifindex is not zero, the primary local address of the interface specified by the index overwrites ipi_spec_dst for the routing table lookup. ``` **This issue is not seen on multi-asic platform, why?** on multi-asic platform, there exists different network namespaces. SNMP docker with snmpd process runs on host namespace. Management interface belongs to host namespace. Loopback0 is configured on asic namespaces. Additional inforamtion on how the packet coming over Loopback IP reaches snmpd process running on host namespace: https://github.com/sonic-net/sonic-buildimage/pull/5420 Because of this separation of network namespaces, the route lookup of destination IP is confined to routing table of specific namespace where packet is received. if packet is received over management interface, SNMP response also is sent out of management interface. Same goes with packet received over Loopback Ip. ##### Work item tracking - Microsoft ADO **17537063**: #### How I did it Have snmpd listen on specific Management and Loopback IPs specifically instead of listening on any IP for single-asic platform. Before Fix ``` admin@xx:~$ sudo netstat -tulnp | grep 161 udp 0 0 0.0.0.0:161 0.0.0.0:* 15631/snmpd udp6 0 0 :::161 :::* 15631/snmpd ``` After fix ``` admin@device:~$ sudo netstat -tulnp | grep 161 udp 0 0 10.1.0.32:161 0.0.0.0:* 215899/snmpd udp 0 0 10.3.1.1:161 0.0.0.0:* 215899/snmpd udp6 0 0 fc00:1::32:161 :::* 215899/snmpd udp6 0 0 fc00:2::32:161 :::* 215899/snmpd ``` **How this change helps with the issue?** To see snmpd trace logs, modify snmpd to start using the below parameters, in supervisord.conf file ``` /usr/sbin/snmpd -f -LS0-7i -Lf /var/log/snmpd.log ``` When snmpd listens on any IP, snmpd binds to IPv4 and IPv6 sockets as below: ``` netsnmp_udpbase: binding socket: 7 to UDP: [0.0.0.0]:0->[0.0.0.0]:161 trace: netsnmp_udp6_transport_bind(): transports/snmpUDPIPv6Domain.c, 303: netsnmp_udpbase: binding socket: 8 to UDP/IPv6: [::]:161 ``` When IPv4 response is sent, it goes out of fd 7 and IPv6 response goes out of fd 8. When IPv6 response is sent, it does not have the right SRC IP and it can lead to the issue described. When snmpd listens on specific Loopback/Management IPs, snmpd binds to different sockets: ``` trace: netsnmp_udpipv4base_transport_bind(): transports/snmpUDPIPv4BaseDomain.c, 207: netsnmp_udpbase: binding socket: 7 to UDP: [0.0.0.0]:0->[10.250.0.101]:161 trace: netsnmp_udpipv4base_transport_bind(): transports/snmpUDPIPv4BaseDomain.c, 207: netsnmp_udpbase: binding socket: 8 to UDP: [0.0.0.0]:0->[10.1.0.32]:161 trace: netsnmp_register_agent_nsap(): snmp_agent.c, 1261: netsnmp_register_agent_nsap: fd 8 netsnmp_udpbase: binding socket: 10 to UDP/IPv6: [fc00:1::32]:161 trace: netsnmp_register_agent_nsap(): snmp_agent.c, 1261: netsnmp_register_agent_nsap: fd 10 netsnmp_ipv6: fmtaddr: t = (nil), data = 0x7fffed4c85d0, len = 28 trace: netsnmp_udp6_transport_bind(): transports/snmpUDPIPv6Domain.c, 303: netsnmp_udpbase: binding socket: 9 to UDP/IPv6: [fc00:2::32]:161 ``` When SNMP request comes in via Loopback IPv4, SNMP response is sent out of fd 8 ``` trace: netsnmp_udpbase_send(): transports/snmpUDPBaseDomain.c, 511: netsnmp_udp: send 170 bytes from 0x5581f2fbe30a to UDP: [10.0.0.33]:46089->[10.1.0.32]:161 on fd 8 ``` When SNMP request comes in via Loopback IPv6, SNMP response is sent out of fd 10 ``` netsnmp_ipv6: fmtaddr: t = (nil), data = 0x5581f2fc2ff0, len = 28 trace: netsnmp_udp6_send(): transports/snmpUDPIPv6Domain.c, 164: netsnmp_udp6: send 170 bytes from 0x5581f2fbe30a to UDP/IPv6: [fc00::42]:43750 on fd 10 ``` #### How to verify it Verified on single asic and multi-asic devices. Single asic SNMP query with Loopback ``` ARISTA01T1#bash snmpget -v2c -c xxx 10.1.0.32 1.3.6.1.2.1.1.1.0 SNMPv2-MIB::sysDescr.0 = STRING: SONiC Software Version: SONiC.xx - HwSku: Arista-7260xx - Distribution: Debian 10.13 - Kernel: 4.19.0-12-2-amd64 ARISTA01T1#bash snmpget -v2c -c xxx fc00:1::32 1.3.6.1.2.1.1.1.0 SNMPv2-MIB::sysDescr.0 = STRING: SONiC Software Version: SONiC.xx - HwSku: Arista-7260xxx - Distribution: Debian 10.13 - Kernel: 4.19.0-12-2-amd64 ``` On multi-asic -- no change. ``` sudo netstat -tulnp | grep 161 udp 0 0 0.0.0.0:161 0.0.0.0:* 17978/snmpd udp6 0 0 :::161 :::* 17978/snmpd ``` Query result using Loopback IP from a directly connected BGP neighbor ``` ARISTA01T2#bash snmpget -v2c -c xxx 10.1.0.32 1.3.6.1.2.1.1.1.0 SNMPv2-MIB::sysDescr.0 = STRING: SONiC Software Version: SONiC.xx - HwSku: xx - Distribution: Debian 9.13 - Kernel: 4.9.0-14-2-amd64 ARISTA01T2#bash snmpget -v2c -c xxx fc00:1::32 1.3.6.1.2.1.1.1.0 SNMPv2-MIB::sysDescr.0 = STRING: SONiC Software Version: SONiC.xx - HwSku: xx - Distribution: Debian 9.13 - Kernel: 4.9.0-14-2-amd64 ``` <!-- If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012. --> |
||
nmoray
|
f978b2bb53
|
Timezone sync issue between the host and containers (#14000)
#### Why I did it To fix the timezone sync issue between the containers and the host. If a certain timezone has been configured on the host (SONIC) then the expectation is to reflect the same across all the containers. This will fix [Issue:13046](https://github.com/sonic-net/sonic-buildimage/issues/13046). For instance, a PST timezone has been set on the host and if the user checks the link flap logs (inside the FRR), it shows the UTC timestamp. Ideally, it should be PST. |
||
Travis Van Duyn
|
d769ef2abd
|
[snmp]: updated to support snmp config from redis configdb (#6134)
**- Why I did it** I'm updating the jinja2 template to support getting SNMP information from the redis configdb. I'm using the format approved here: https://github.com/Azure/SONiC/pull/718 This will pave the way for us to decrement using the snmp.yml in the future. Right now we will still be using both the snmp.yml and configdb to get variable information in order to create the snmpd.conf via the sonic-cfggen tool. **- How I did it** I first updated the SNMP Schema in PR #718 to get that approved as a standardized format. Then I verified I could add snmp configs to the configdb using this standard schema. Once the configs were added to the configdb then I updated the snmpd.conf.j2 file to support the updates via the configdb while still using the variables in the snmp.yml file in parallel. This way we will have backward compatibility until we can fully migrate to the configdb only. By updating the snmpd.conf.j2 template and running the sonic-cfggen tool the snmpd.conf gets generated with using the values in both the configdb and snmp.yml file. Co-authored-by: trvanduy <trvanduy@microsoft.com> |
||
|
Travis Van Duyn
|
6efc0a885f
|
Convert snmp.yml to configdb (#6205)
This PR is in preparation to move from snmp.yml to configdb. This will more closely align with other commands in sonic and use configdb as the source of truth for snmp configuration. Note: This is the first of 2 PR's to enable this. This PR will not change any functionality but will allow the snmp.yml file info to be put into the configdb. Created a script that takes the snmp.yml variables and converts them to the configdb format. Added file to dockerfile.j2 so that file is copied in the container. Updated start.sh file to automatically run the python conversion script each time the docker container is restarted. |
||
Renuka Manavalan
|
ba02209141
|
First cut image update for kubernetes support. (#5421)
* First cut image update for kubernetes support.
With this,
1) dockers dhcp_relay, lldp, pmon, radv, snmp, telemetry are enabled
for kube management
init_cfg.json configure set_owner as kube for these
2) Each docker's start.sh updated to call container_startup.py to register going up
As part of this call, it registers the current owner as local/kube and its version
The images are built with its version ingrained into image during build
3) Update all docker's bash script to call 'container start/stop/wait' instead of 'docker start/stop/wait'.
For all locally managed containers, it calls docker commands, hence no change for locally managed.
4) Introduced a new ctrmgrd service, that helps with transition between owners as kube & local and carry over any labels update from STATE-DB to API server
5) hostcfgd updated to handle owner change
6) Reboot scripts are updatd to tag kube running images as local, so upon reboot they run the same image.
7) Added kube_commands.py to handle all updates with Kubernetes API serrver -- dedicated for k8s interaction only.
|
||
Tamer Ahmed
|
cc970627d0
|
[snmp]: Reduce Calls to SONiC Cfggen (#5166)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to sonic-cfggen to once calle during snmp startup singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> |
||
Joe LeVeque
|
5d5d5739c2
|
[dockers] Rename 'docker-snmp-sv2' to 'docker-snmp' (#4699)
The `-sv2` suffix was used to differentiate SNMP Dockers when we transitioned from "SONiCv1" to "SONiCv2", about four years ago. The old Docker materials were removed long ago; there is no need to keep this suffix. Removing it aligns the name with all the other Dockers. Also edit Monit configuration to detect proper snmp-subagent command line in Buster, and make snmpd command line matching more robust. |