#### Why I did it
This fixed memory leak in ETHERLIKE-MIB. The fix is not part of net-snmp(5.7.3 version). This PR includes the patch to fix memory leak issue.
```
ke->name in stdup-ed at line 297: n->name = strdup(RTA_DATA(tb[IFLA_IFNAME]));
```
#### How I did it
patched the fix.
[net-snmp] upstream fix link -> [snmpd|upstream link](ed4e48b5fa)
#### How to verify it
**Before The fix**
used valgrind to find memory leak.
```
root@lnos-x1-a-csw06:/# grep "definitely lost" valgrind-out.txt
==493== 4 bytes in 1 blocks are definitely lost in loss record 1 of 333
==493== 16 bytes in 1 blocks are definitely lost in loss record 25 of 333
==493== 757 bytes in 71 blocks are definitely lost in loss record 214 of 333
==493== 1,168 (32 direct, 1,136 indirect) bytes in 1 blocks are definitely lost in loss record 293 of 333
==493== 1,168 (32 direct, 1,136 indirect) bytes in 1 blocks are definitely lost in loss record 294 of 333
==493== 1,168 (32 direct, 1,136 indirect) bytes in 1 blocks are definitely lost in loss record 295 of 333
==493== 1,168 (32 direct, 1,136 indirect) bytes in 1 blocks are definitely lost in loss record 296 of 333
==493== definitely lost: 905 bytes in 77 blocks
```
_we can see the memory leak see in stack trace._
-> dot3stats_linux -> get_nlmsg -> strdup
https://github.com/net-snmp/net-snmp/blob/v5.7.3/agent/mibgroup/etherlike-mib/data_access/dot3stats_linux.chttps://github.com/net-snmp/net-snmp/blob/v5.7.3/agent/mibgroup/etherlike-mib/data_access/dot3stats_linux.c#L277
```
n = malloc(sizeof(*n));
memset(n, 0, sizeof(*n));
n->ifindex = ifi->ifi_index;
n->name = strdup(RTA_DATA(tb[IFLA_IFNAME]));
memcpy(&n->stats, RTA_DATA(tb[IFLA_STATS]), sizeof(n->stats));
n->next = kern_db;
kern_db = n;
return 0;
```
we were not freeing space for EtherLike-MIB.AS interface mib queries were getting increased, we see memory increment.
```
kern_db = ke->next;
free(ke);
```
https://github.com/net-snmp/net-snmp/blob/v5.7.3/agent/mibgroup/etherlike-mib/data_access/dot3stats_linux.c#L467
```
==55== 757 bytes in 71 blocks are definitely lost in loss record 186 of 299
==55== at 0x483577F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==55== by 0x4EB6E49: strdup (strdup.c:42)
==55== by 0x493F278: get_nlmsg (dot3stats_linux.c:299)
==55== by 0x493F529: rtnl_dump_filter_l.constprop.3 (dot3stats_linux.c:370)
==55== by 0x493FD7A: rtnl_dump_filter (dot3stats_linux.c:401)
==55== by 0x493FD7A: _dot3Stats_netlink_get_errorcntrs (dot3stats_linux.c:424)
==55== by 0x494009F: interface_dot3stats_get_errorcounters (dot3stats_linux.c:530)
==55== by 0x48F6FDA: dot3StatsTable_container_load (dot3StatsTable_data_access.c:330)
==55== by 0x485E76B: _cache_load (cache_handler.c:700)
==55== by 0x485FA37: netsnmp_cache_helper_handler (cache_handler.c:638)
==55== by 0x48720BC: netsnmp_call_handler (agent_handler.c:526)
==55== by 0x48720BC: netsnmp_call_next_handler (agent_handler.c:640)
==55== by 0x4865F75: table_helper_handler (table.c:717)
==55== by 0x4871B66: netsnmp_call_handler (agent_handler.c:526)
==55== by 0x4871B66: netsnmp_call_handlers (agent_handler.c:611)
757 bytes in 71 blocks are definitely lost in loss record 214 of 333
==493== at 0x483577F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==493== by 0x4EB6E49: strdup (strdup.c:42)
==493== by 0x493F278: ??? (in /usr/lib/x86_64-linux-gnu/libnetsnmpmibs.so.30.0.3)
==493== by 0x493F529: ??? (in /usr/lib/x86_64-linux-gnu/libnetsnmpmibs.so.30.0.3)
==493== by 0x493FD7A: _dot3Stats_netlink_get_errorcntrs (in /usr/lib/x86_64-linux-gnu/libnetsnmpmibs.so.30.0.3)
==493== by 0x494009F: interface_dot3stats_get_errorcounters (in /usr/lib/x86_64-linux-gnu/libnetsnmpmibs.so.30.0.3)
==493== by 0x48F6FDA: dot3StatsTable_container_load (in /usr/lib/x86_64-linux-gnu/libnetsnmpmibs.so.30.0.3)
==493== by 0x485E76B: _cache_load (cache_handler.c:700)
==493== by 0x485FA37: netsnmp_cache_helper_handler (cache_handler.c:638)
==493== by 0x48720BC: netsnmp_call_handler (agent_handler.c:526)
==493== by 0x48720BC: netsnmp_call_next_handler (agent_handler.c:640)
==493== by 0x4865F75: table_helper_handler (table.c:717)
==493== by 0x4871B66: netsnmp_call_handler (agent_handler.c:526)
==493== by 0x4871B66: netsnmp_call_handlers (agent_handler.c:611)
```
```
**After The fix**
no memory leak in valgrind stack trace related to etherlike MIB.
```
* [snmpd]: Update to 5.9+dfsg-4+deb11u1 to match Debian version
This brings in some security fixes.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Update snmpd makefile
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Remove binNMU for snmpd
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Debian actually did a binNMU for snmpd, so to match the package version
we're building with the version in the offiical repos, that version
needs to be manually specified in the changelog.
Buster still needs 5.7.3, because there's a ABI change between 5.7.3 and
5.9 for libsnmp, so for Buster, make sure that 5.7.3 is built, and for
Bullseye, make sure that 5.9 is built.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
I have been seeing intermittent (~40%) build failures with the same error described in PR https://github.com/Azure/sonic-buildimage/pull/6592, even with that fix present
```
/usr/bin/ld: mibgroup/ip-forward-mib/ipCidrRouteTable/.libs/ipCidrRouteTable_interface.o: file not recognized: file truncated
...
libtool: error: 'mibgroup/ip-forward-mib/inetCidrRouteTable/inetCidrRouteTable_interface.lo' is not a valid libtool object
make[5]: *** [Makefile:1020: libnetsnmpmibs.la] Error 1
make[5]: *** Waiting for unfinished jobs....
```
#### How I did it
Use `-j1` for the libsnmp build regardless of the value of `$(MULTIARCH_QEMU_ENVIRON)`
#### How to verify it
Performed 10 builds of the libsnmp target (`target/debs/buster/libsnmp-base_5.7.3+dfsg-5_all.deb`) with and without this change. Without the change, hit the error 40% of the time. With the change did not see the error at all
Signed-off-by: Justin Sherman <jusherma@cisco.com>
- Support compile sonic arm image on arm server. If arm image compiling is executed on arm server instead of using qemu mode on x86 server, compile time can be saved significantly.
- Add kernel argument systemd.unified_cgroup_hierarchy=0 for upgrade systemd to version 247, according to #7228
- rename multiarch docker to sonic-slave-${distro}-march-${arch}
Co-authored-by: Xianghong Gu <xgu@centecnetworks.com>
Co-authored-by: Shi Lei <shil@centecnetworks.com>
snmpd's compile is always failed with file truncated on ARM64 arch, the error log is like "/usr/bin/ld: mibgroup/ip-forward-mib/inetCidrRouteTable/.libs/inetCidrRouteTable_interface.o: file not recognized: file truncated"
Co-authored-by: Xianghong Gu <xgu@centecnetworks.com>
Fix#119
when parallel build is enable, multiple dpkg-buildpackage
instances are running at the same time. /var/lib/dpkg is shared
by all instances and the /var/lib/dpkg/updates could be corrupted
and cause the build failure.
the fix is to use overlay fs to mount separate /var/lib/dpkg
for each dpkg-buildpackage instance so that they are not affecting
each other.
Signed-off-by: Guohan Lu <lguohan@gmail.com>
- Add .gitignore files in each subdirectory of src/, so as to reduce the size of the .gitignore file in the project root, and also make it easier to maintain (i.e., if a directory in src/ is removed, there will not be outdated entries in the root .gitignore file.
- Also add missing .gitignore entries and remove outdated entries and duplicates.
* [SNMP] management VRF SNMP support
This commit adds SNMP support for Management VRF using l3mdev.
The patch included provides VRF support, there is no single
"listendevice" configuration, rather multiple agentaddress
config options can each have their own "interface" to bind to
using "ip%interface". The snmpd.conf file is accordingly
generated using the snmp.yml file and redis database info.
Adding below the comments of SNMP patch 1376
--------------------------------------------
Since the Linux kernel added support for Virtual Routing
and Forwarding (VRF) in version 4.3
(Note: these won't compile on non-linux platforms)
https://www.kernel.org/doc/Documentation/networking/vrf.txt
Linux users could not use snmpd in its current form to
bind specific listening IP addresses to specific VRF
devices. A simplified description of a VRF inteface
is an interface that is a master (a container of sorts)
that collects a set of physicalinterfaces to form a
routing table.
This set of two patches (one for V5-7-patches and one
for V5-8-patches branches) is almost identical to patch
single "listendevice" configuration. Rather, multiple
agentAddress config options can each have their own
"interface" to bind to using the <ip>%<interface>
syntax.</interface></ip>
-------------------------------------------
Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
Upon snmpd closes a netsnmp_agent_session due to snmp_timeout
there is a possibility of crash due to stale memory access.
This is a patch from source-forge:net-snmp. commit-id #793d59
* Enable debug image build for kvm image.
* Fix a bug in cleaning up requests referring the netsmp_session being closed to avoid a crash, during netsnmp_session close/abort due to timeout.
RCA:
The netsnmp_agent_session requests use netsmp_subtree objects that matches the associated variable name.
The netsnmp_subtrees created through a netsnmp_session are tied to that session.
When a subagent connection is closed/dropped due to timeout, all associated netsnmp_subtree objects are fred.
Hence during a netsnmp_session close, all the delegated netsnmp_agent_sessions are scanned for requets that could be using netsnmp_subtree objects associated with this netsnmp_sesssion or its subsession. For each of the found request, they are explicitly marked to fail and a call is made to complete them.
But due to the bug in scanning, it leaves behind requests and hence later when the requests get processed, they refer the *now* freed netsnmp_subtree. As often these requests gets completed pretty soon, they escape crashing. But if it so happens that the freed memory happened to complete a memory unit, hence returned to kernel or it got reallocated & changed enough to crash, the snmpd crashes.
* Revert the changes
* Revert
ARM Architecture support in SONIC
make configure platform=[ASIC_VENDOR_ARCH] PLATFORM_ARCH=[ARM_ARCH]
SONIC_ARCH: default amd64
armhf - arm32bit
arm64 - arm64bit
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* Added debug symbols to many debug dockers.
* For debug images *only*:
1) Archive source files into debug image
2) Archived source is copied into /src
3) Created an empty dir /debug
4) Mount both /src as ro & /debug as rw into every docker
5) Login banner will give some details on /src & /debug
6) Devs can copy core file into /debug and view it from inside a container.
7) Dev may create all gdb logs and other data directly into /debug.
* Dropped redundant REDIS_TOOLS per review comments.
* Added debug symbols to frr package and hence FRR based BGP docker.
* 1) Moved dbg_files.sh to scripts/
2) Src directories to archive are now collected from individual Makefiles.
3) Added few more debug symbols
4) Added few more debug dockers.
Here after no more changes except per review comments.
To debug:
Install required version of debug image in Switch or VM.
Copy core file into /debug of host
Get into Docker
gdb /usr/bin/<daemon> -c /debug/<your core file>
set directory /src/... <-- inside gdb to get the source
For non-in-depth debugging:
Download corresponding debug Docker image (docker-...-dbg.gz) to your VM
Load the image
Run image with entrypoint as 'bash' with dir containing core mapped in.
Run gdb on the core.
* [dockers]: Upgrade SNMP docker to stretch build
* Removed patch-> 0003-CHANGES-BUG-2743-snmpd-crashes-when-receiving-a-GetN.patch
* update platform-common submodule
* adding PyYAML package to stretch dockerfile
* Installing redis package via pip in stretch dockerfile
* again updating platform-common submodule
* revert back the snmpd version to 5.7.3+dfsg-1.5
* upgrading the snmpd version to 5.7.3+dfsg-1.7+deb9u1 with openssl backport support
* update sonic-snmpagent submodule
* purge libdpkg-perl package in Dockerfile
* revert back the snmpd version to 5.7.3+dfsg-1.5
* minor change in series file
* [config]: Add SONIC_CONFIG_MAKE_JOBS
This config option allows user to specify -j value that will be passed
to each package build.
Signed-off-by: marian-pritsak <marianp@mellanox.com>
* Build improvements
Fix dependencies
Add configuration options
Automatically build sonic-slave
* Set default number of jobs to 1
* Auto generate target/debs directory
Signed-off-by: marian-pritsak <marianp@mellanox.com>
* Automatically remove sonic-slave container after exit
* Silence clean-logs
* Add SONIC_CLEAN_TARGETS to clean
* Use second expansion for clean dependencies
* Avoid creating empty log files
Remove log file on flush instead of writing empty string
* Put dpkg install inside lock
Use same lock as debian install targets do to avoid
race condition in dpkg installation
* Remove redirect to log from docker save
* Add .platform dependency to all and clean targets
* Remove header and footer from clean targets
* Disable messages for SONIC_CLEAN_TARGETS
* Exit with error if dpkg-buildpackage fails
* Set new location for debs in build_debian.sh
* Add recipe for docker-database
* Update redis version to 3.2.4
* Add support for p4 platform
* Add recipe for snmpd
* Add slave targets to phony and make all target default
* Remove build.sh from thrift
* Add versioning to team, nl, hiredis and initramfs
* Change sonic-slave to support snmpd build from sources
* Remove src/tenjin
* Add recipe for lldpd
* Add recipe for mpdecimal
* Remove hiredis directory on rebuild
* Add recipe for Mellanox hw management
* Remove generic image from all targets for Mellanox
* Add support for python wheels
* Add lldp and snmp dockers
* Sync docker-database to include libjemalloc
* Fix asyncsnmp variable name
* Change default build configuration
Redirect output to log files by default
Set number of jobs to nproc value
Do not print dependencies
Fix logging to print log of failed job into console
* Use docker inspect to check if sonic-slave image exists
* Use config in slave.mk directly
* Disable color output by default
* Remove sswsdk dependency from lldp and snmp dockers
* Fix comment in py wheels install targets
* Add dependency between two versions of sswsdk
* Add containers to mellanox platform
lldp, snmp and database containers
* Add recipe for team docker
* Add team docker to mellanox platform
* Encrypt password passed to build_debian.sh
* Update mellanox SAI version
Make version and revision setting only in main recipe
* Fix error handling in makefiles
As makefiles use .ONESHELL we should add -e
option to shell options in order to exit after any command fails
* Add recipe for platform monitor image
* Add platfotm monitor to mellanox targets
* Ignore submodules when building base image
