matthew/sonic-buildimage
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[build]: Build sonic-broadcom.bin using debug dockers for all stretch based dockers (#2833)

Browse Source 
* Updated Makefile infrastructure to build debug images.
As a sample, platform/broadcom/docker-orchagent-brcm.mk is updated to add a docker-orchagent-brcm-dbg.gz target.

Now "BLDENV=stretch make target/docker-orchagent-brcm-dbg.gz" will build the debug image.

NOTE: If you don't specify NOSTRETcH=1, it implicitly calls "make stretch", which builds all stretch targets and that would include debug dockers too.

This debug image can be used in any linux box to inspect core file. If your module's external dependency can be suitably mocked, you my even manually run it inside.

"docker run -it --entrypoint=/bin/bash e47a8fb8ed38"

You may map the core file path to this docker run.

* Dropped the regular binary using DBG_PACKAGES and a small name change to help readability.

* Tweaked the changes to retain the existing behavior w.r.t INSTALL_DEBUG_TOOLS=y.

When this change ('building debug docker image transparently') is extended to all dockers, this flag would become redundant. Yet, there can be some test based use cases that rely on this flag.

Until after all the dockers gets their debug images by default and we switch all use cases of this flag to use the newly built debug images, we need to maintain the existing behavior.

* 1) slave.mk - Dropped unused Docker build args
2) Debug template builder: renamed build_dbg_j2.sh to build_debug_docker_j2.sh
3) Dropped insignifcant statement CMD from debug Docker file, as base docker has Entrypoint.

* Reverted some changes, per review comments.
"User, uid, guid, frr-uid & frr-guid" are required for all docker images, with exception of debug images.

* Get in sync with the new update that filters out dockers to be built (SONIC_STRETCH_DOCKERS_FOR_INSTALLERS) and build debug-dockers only for those to be built and debug target is available.

* Mkae a template for each target that can be shared by all platforms.
Where needed a platform entry can override the template.
This avoids duplication, hence easier to maintain.

* A small change, that can fit better with other targets too.
Just take the platform code and do the rest in template.

* Extended debug to all stretch based docker images

* 1) Combined all orchagent makefiles into one platform independent make under rules/docker-orchagent.mk
2) Extened debug image to all stretch dockers

* Changes per review comments:
1) Dropped LIBSAIREDIS_DBG from database, teamd, router-advertiser, telemetry, and platform-monitor docker*.mk files from _DBG_DEPENDS list
2) W.r.t docker make for syncd, moved DEPENDS from template to specific makefile and let the template has stuff that is applicable to all.

* 1) Corrected a copy/paste mistake

* Fixed a copy/paste bug

* The base syncd dockers follow a template, which defines the base docker as DOCKER_SYNCD_BASE instead of DOCKER_SYNCD_<platform code>. Fix the docker-syncd-<mlnx, bfn>.mk to use the new one.

[Yet to be tested locally]

* Fixed spelling mistake

* Enable build of dbg-sonic-broadcom.bin, which uses dbg-dockers in place of regular dockers, for dockers that build debug version. For dockers that do not build debug version, it uses the regular docker.

This debug bin is installable and usable in a DUT, just like a regular bin.

* Per review comments:
  1) Share a single rule for final image for normal & debug flavors (e.g. sonic-broadcom.bin & sonic-broadcom-dbg.bin)
  2) Put dbg as suffix in final image name.
  3) Compared target/sonic-broadcom.bin.logs with & w/o fix to verify integrity of sonic-broadcom.bin
  4) Compared target/sonic-broadcom.bin.logs with sonic-broadcom-dbg.bin.log for verification

This fix takes care of ONIE image only. The next PR will cover the rest.
The next PR, will also make debug image conditional with flag.

* Updated per comments.
Now that debug dockers are available, do not need a way to install debug symbols in regular dockers.

With this commit, when INSTALL_DEBUG_TOOLS=y is set, it builds debug dockers (for dockers that enable debug build) and the final image uses debug dockers. For dockers that do not enable debug build, regular dockers get used in the final image.

Note:
The debug dockers are explicitly named as <docker name>-dbg.gz. But there is no "-dbg" suffix for image.
Hence if you make two runs with and w/o INSTALL_DEBUG_TOOLS=y, you have complete set of regular dockers + debug dockers. But the image gets overwritten.
Hence if both regular & debug images are needed, make two runs, as one with INSTALL_DEBUG_TOOLS=y and one w/o. Make sure to copy/rename the final image, before making the second run.
This commit is contained in:
Renuka Manavalan 2019-06-12 01:36:21 -07:00 committed by lguohan
parent 44bc083cde
commit cdca062693
3 changed files with 26 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
files/build_templates/sonic_debian_extension.j2
View File

@ -288,6 +288,11 @@ sudo chroot $FILESYSTEM_ROOT docker info
{% set imagename = imagefilename.split('.')|first -%}
sudo LANG=C chroot $FILESYSTEM_ROOT docker load < {{image}}
sudo LANG=C chroot $FILESYSTEM_ROOT docker tag {{imagename}}:latest {{imagename}}:$(sonic_get_version)
{% if imagename.endswith('-dbg') %}
{% set imagebasename = imagename.replace('-dbg', '') -%}
sudo LANG=C chroot $FILESYSTEM_ROOT docker tag {{imagename}}:latest {{imagebasename}}:$(sonic_get_version)
sudo LANG=C chroot $FILESYSTEM_ROOT docker tag {{imagename}}:latest {{imagebasename}}:latest
{% endif %}
{% endfor %}
sudo chroot $FILESYSTEM_ROOT service docker stop
sudo rm $FILESYSTEM_ROOT/etc/init.d/docker

8
platform/broadcom/one-image.mk
View File

@ -40,5 +40,11 @@ $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(DELL_S6000_PLATFORM_MODULE) \
$(ALPHANETWORKS_SNH60A0_320FV2_PLATFORM_MODULE) \
$(ALPHANETWORKS_SNH60B0_640F_PLATFORM_MODULE) \
$(BRCM_XLR_GTS_PLATFORM_MODULE)
$(SONIC_ONE_IMAGE)_DOCKERS += $(SONIC_INSTALL_DOCKER_IMAGES)
ifeq ($(INSTALL_DEBUG_TOOLS),y)
$(SONIC_ONE_IMAGE)_DOCKERS += $(SONIC_INSTALL_DOCKER_DBG_IMAGES)
$(SONIC_ONE_IMAGE)_DOCKERS += $(filter-out $(patsubst %-$(DBG_IMAGE_MARK).gz,%.gz, $(SONIC_INSTALL_DOCKER_DBG_IMAGES)), $(SONIC_INSTALL_DOCKER_IMAGES))
else
$(SONIC_ONE_IMAGE)_DOCKERS = $(SONIC_INSTALL_DOCKER_IMAGES)
endif
SONIC_INSTALLERS += $(SONIC_ONE_IMAGE)

27
slave.mk
View File

@ -541,7 +541,8 @@ $(addprefix $(TARGET_PATH)/, $(DOCKER_DBG_IMAGES)) : $(TARGET_PATH)/%-$(DBG_IMAG
./build_debug_docker_j2.sh $* $(subst -,_,$(notdir $($*.gz_PATH)))_dbg_debs $(subst -,_,$(notdir $($*.gz_PATH)))_image_dbgs > $($*.gz_PATH)/Dockerfile-dbg.j2
j2 $($*.gz_PATH)/Dockerfile-dbg.j2 > $($*.gz_PATH)/Dockerfile-dbg
docker info $(LOG)
docker build --squash --no-cache \
docker build \
$(if $($*.gz_DBG_DEPENDS), --squash --no-cache, --no-cache) \
--build-arg http_proxy=$(HTTP_PROXY) \
--build-arg https_proxy=$(HTTPS_PROXY) \
--build-arg docker_container_name=$($*.gz_CONTAINER_NAME) \
@ -617,19 +618,19 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_INSTALLERS)) : $(TARGET_PATH)/% : \
$(foreach docker, $($*_DOCKERS),\
export docker_image="$(docker)"
export docker_image_name="$(basename $(docker))"
export docker_container_name="$($(docker)_CONTAINER_NAME)"
$(eval $(docker)_RUN_OPT += $($(docker)_$($*_IMAGE_TYPE)_RUN_OPT))
export docker_image_run_opt="$($(docker)_RUN_OPT)"
j2 files/build_templates/docker_image_ctl.j2 > $($(docker)_CONTAINER_NAME).sh
if [ -f files/build_templates/$($(docker)_CONTAINER_NAME).service.j2 ]; then
j2 files/build_templates/$($(docker)_CONTAINER_NAME).service.j2 > $($(docker)_CONTAINER_NAME).service
export docker_container_name="$($(docker:-dbg.gz=.gz)_CONTAINER_NAME)"
$(eval $(docker:-dbg.gz=.gz)_RUN_OPT += $($(docker:-dbg.gz=.gz)_$($*_IMAGE_TYPE)_RUN_OPT))
export docker_image_run_opt="$($(docker:-dbg.gz=.gz)_RUN_OPT)"
j2 files/build_templates/docker_image_ctl.j2 > $($(docker:-dbg.gz=.gz)_CONTAINER_NAME).sh
if [ -f files/build_templates/$($(docker:-dbg.gz=.gz)_CONTAINER_NAME).service.j2 ]; then
j2 files/build_templates/$($(docker:-dbg.gz=.gz)_CONTAINER_NAME).service.j2 > $($(docker:-dbg.gz=.gz)_CONTAINER_NAME).service
fi
chmod +x $($(docker)_CONTAINER_NAME).sh
chmod +x $($(docker:-dbg.gz=.gz)_CONTAINER_NAME).sh
)
export installer_start_scripts="$(foreach docker, $($*_DOCKERS),$(addsuffix .sh, $($(docker)_CONTAINER_NAME)))"
export installer_services="$(foreach docker, $($*_DOCKERS),$(addsuffix .service, $($(docker)_CONTAINER_NAME)))"
export installer_extra_files="$(foreach docker, $($*_DOCKERS), $(foreach file, $($(docker)_BASE_IMAGE_FILES), $($(docker)_PATH)/base_image_files/$(file)))"
export installer_start_scripts="$(foreach docker, $($*_DOCKERS),$(addsuffix .sh, $($(docker:-dbg.gz=.gz)_CONTAINER_NAME)))"
export installer_services="$(foreach docker, $($*_DOCKERS),$(addsuffix .service, $($(docker:-dbg.gz=.gz)_CONTAINER_NAME)))"
export installer_extra_files="$(foreach docker, $($*_DOCKERS), $(foreach file, $($(docker:-dbg.gz=.gz)_BASE_IMAGE_FILES), $($(docker:-dbg.gz=.gz)_PATH)/base_image_files/$(file)))"
j2 -f env files/initramfs-tools/union-mount.j2 onie-image.conf > files/initramfs-tools/union-mount
j2 -f env files/initramfs-tools/arista-convertfs.j2 onie-image.conf > files/initramfs-tools/arista-convertfs
@ -652,8 +653,8 @@ $(addprefix $(TARGET_PATH)/, $(SONIC_INSTALLERS)) : $(TARGET_PATH)/% : \
./build_image.sh $(LOG)
$(foreach docker, $($*_DOCKERS), \
rm -f $($(docker)_CONTAINER_NAME).sh
rm -f $($(docker)_CONTAINER_NAME).service
rm -f $($(docker:-dbg.gz=.gz)_CONTAINER_NAME).sh
rm -f $($(docker:-dbg.gz=.gz)_CONTAINER_NAME).service
)
$(if $($*_DOCKERS),