matthew/sonic-buildimage
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6d3e9e8b83
sonic-buildimage/Makefile.work
Greg Paussa 48c77f85a1 [build] Add option to avoid Docker base image :latest tag (#3124) 
Define slave_base_tag_ref variable in Makefile.work containing
specific base image tag to use, rather than always defaulting
to :latest.

Add an ARG command before FROM statement in Dockerfile.user
for sonic-slave and sonic-slave-stretch. ARG variable defaults
to latest if slave_base_tag_ref not specified in Makefile.work.

The presumption to always refer to the :latest tagged Docker base
image when creating the user image causes problems in a shared
build server environment, where the most recently created base
image (i.e. the current :latest tag) may not be compatible with
the current build. For example, different users working in
different branches may all be sharing the same build server.

Signed-off-by: Greg Paussa greg.paussa@broadcom.com

- What I did
Added a DOCKER_AVOID_BASE_TAG_LATEST build option to rules/config that forces the Docker user image creation to refer to its base image by a specific tag rather than rely on the :latest tag. This is needed in a shared build server environment where builds from different developers and/or different SONiC branches all converge on the same Docker daemon instance running on the build server. The :latest tag is always assigned to the most recent base image built, which might not correspond to the base image needed for a particular build, thus causing various build errors that mostly manifest as missing Debian packages or package version mismatches.

NOTE TO REVIEWERS: This PR relies on Docker support of "ARG before FROM," which was first introduced in Docker version 17.05.1-ce. Although there is no mention of a minimum required Docker version for the build server in the SONiC Building Guide pages, please consider whether it is reasonable to assume that Docker 17.05.1-ce or later must be used for SONiC build hosts before approving this PR.

- How I did it
Added an ARG before the FROM statement at the top of the sonic-slave/Dockerfile.user and sonic-slave-stretch/Dockerfile.user files. The ARG variable defaults to latest, but can be overridden in Makefile.work to reference the SLAVE_BASE_TAG so that it refers to the specific, matching base image for the build. This override is activated by un-commenting the DOCKER_AVOID_BASE_TAG_LATEST = y line in rules/config.
2019-07-13 12:43:45 -07:00

201 lines
8.7 KiB
Makefile
Raw Blame History

###############################################################################
## Wrapper for starting make inside sonic-slave container
#
# Supported parameters:
#
# * PLATFORM: Specific platform we wish to build images for.
# * BUILD_NUMBER: Desired version-number to pass to the building-system.
# * ENABLE_DHCP_GRAPH_SERVICE: Enables get-graph service to fetch minigraph files
# through http.
# * SHUTDOWN_BGP_ON_START: Sets admin-down state for all bgp peerings after restart.
# * ENABLE_PFCWD_ON_START: Enable PFC Watchdog (PFCWD) on server-facing ports
# * by default for TOR switch.
# * ENABLE_SYNCD_RPC: Enables rpc-based syncd builds.
# * INSTALL_DEBUG_TOOLS: Install debug tools and debug symbol packeages.
# * USERNAME: Desired username -- default at rules/config
# * PASSWORD: Desired password -- default at rules/config
# * KEEP_SLAVE_ON: Keeps slave container up and active after building process concludes.
# * Note that rm=true is still set, so once user quits from the docker
# * session, the docker will be removed.
# * Please note that with current Stretch build structure,
# * user of KEEP_SLAVE_ON feature will have to be conscious
# * about which docker to stay inside after build is done.
# * - If user desires to stay inside Jessie docker, please issue
# * make KEEP_SLAVE_ON=yes jessie
# * - If user desires to stay inside Stretch docker, please issue
# * make NOJESSIE=1 KEEP_SLAVE_ON=yes <any target>
# * SOURCE_FOLDER: host path to be mount as /var/$(USER)/src, only effective when KEEP_SLAVE_ON=yes
# * SONIC_BUILD_JOBS: Specifying number of concurrent build job(s) to run
# * VS_PREPARE_MEM: Prepare memory in VS build (drop cache and compact).
# * Default: yes
# * Values: yes, no
# * KERNEL_PROCURE_METHOD: Specifying method of obtaining kernel Debian package: download or build
#
###############################################################################
SHELL = /bin/bash
USER := $(shell id -un)
PWD := $(shell pwd)
ifeq ($(USER), root)
$(error Add your user account to docker group and use your user account to make. root or sudo are not supported!)
endif
# Check for minimum Docker version on build host
# Note: Using the greater of CE (17.05.0) and EE (17.06.1) versions that support ARG before FROM
docker_min := 17.06.1
docker_min_ver := $(shell echo "$(docker_min)" | awk -F. '{printf("%d%03d%03d\n",$$1,$$2,$$3);}' 2>/dev/null)
docker_ver := $(shell docker info 2>/dev/null | grep -i "server version" | cut -d' ' -f3 | awk -F. '{printf("%d%03d%03d\n",$$1,$$2,$$3);}' 2>/dev/null)
docker_is_valid := $(shell if [ $(docker_ver) -lt $(docker_min_ver) ] ; then echo "0"; else echo "1"; fi)
ifeq (0,$(docker_is_valid))
$(error SONiC requires Docker version $(docker_min) or later)
endif
# Remove lock file in case previous run was forcefully stopped
$(shell rm -f .screen)
MAKEFLAGS += -B
ifeq ($(BLDENV), stretch)
SLAVE_BASE_TAG = $(shell sha1sum sonic-slave-stretch/Dockerfile | awk '{print substr($$1,0,11);}')
SLAVE_TAG = $(shell cat sonic-slave-stretch/Dockerfile.user sonic-slave-stretch/Dockerfile | sha1sum | awk '{print substr($$1,0,11);}')
SLAVE_BASE_IMAGE = sonic-slave-stretch-base
SLAVE_IMAGE = sonic-slave-stretch-$(USER)
SLAVE_DIR = sonic-slave-stretch
else
SLAVE_BASE_TAG = $(shell sha1sum sonic-slave/Dockerfile | awk '{print substr($$1,0,11);}')
SLAVE_TAG = $(shell cat sonic-slave/Dockerfile.user sonic-slave/Dockerfile | sha1sum | awk '{print substr($$1,0,11);}')
SLAVE_BASE_IMAGE = sonic-slave-base
SLAVE_IMAGE = sonic-slave-$(USER)
SLAVE_DIR = sonic-slave
endif
OVERLAY_MODULE_CHECK := \
lsmod | grep -q "^overlay " &>/dev/null || \
zgrep -q 'CONFIG_OVERLAY_FS=y' /proc/config.gz &>/dev/null || \
grep -q 'CONFIG_OVERLAY_FS=y' /boot/config-$(shell uname -r) &>/dev/null || \
(echo "ERROR: Module 'overlay' not loaded. Try running 'sudo modprobe overlay'."; exit 1)
BUILD_TIMESTAMP := $(shell date +%Y%m%d\.%H%M%S)
ifeq ($(DOCKER_BUILDER_MOUNT),)
override DOCKER_BUILDER_MOUNT := "$(PWD):/sonic"
endif
ifeq ($(DOCKER_BUILDER_WORKDIR),)
override DOCKER_BUILDER_WORKDIR := "/sonic"
endif
DOCKER_RUN := docker run --rm=true --privileged \
-v $(DOCKER_BUILDER_MOUNT) \
-w $(DOCKER_BUILDER_WORKDIR) \
-e "http_proxy=$(http_proxy)" \
-e "https_proxy=$(https_proxy)" \
-i$(if $(TERM),t,)
include rules/config
ifeq ($(SONIC_CONFIG_USE_NATIVE_DOCKERD_FOR_BUILD), y)
DOCKER_RUN += -v /var/run/docker.sock:/var/run/docker.sock
endif
DOCKER_BASE_BUILD = docker build --no-cache \
-t $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG) \
--build-arg http_proxy=$(http_proxy) \
--build-arg https_proxy=$(https_proxy) \
$(SLAVE_DIR)
DOCKER_BUILD = docker build --no-cache \
--build-arg user=$(USER) \
--build-arg uid=$(shell id -u) \
--build-arg guid=$(shell id -g) \
--build-arg hostname=$(shell echo $$HOSTNAME) \
--build-arg slave_base_tag_ref=$(SLAVE_BASE_TAG) \
-t $(SLAVE_IMAGE):$(SLAVE_TAG) \
-f $(SLAVE_DIR)/Dockerfile.user \
$(SLAVE_DIR)
SONIC_BUILD_INSTRUCTION := make \
-f slave.mk \
BLDENV=$(BLDENV) \
PLATFORM=$(PLATFORM) \
BUILD_NUMBER=$(BUILD_NUMBER) \
BUILD_TIMESTAMP=$(BUILD_TIMESTAMP) \
ENABLE_DHCP_GRAPH_SERVICE=$(ENABLE_DHCP_GRAPH_SERVICE) \
SHUTDOWN_BGP_ON_START=$(SHUTDOWN_BGP_ON_START) \
SONIC_ENABLE_PFCWD_ON_START=$(ENABLE_PFCWD_ON_START) \
SONIC_ENABLE_SYNCD_RPC=$(ENABLE_SYNCD_RPC) \
SONIC_INSTALL_DEBUG_TOOLS=$(INSTALL_DEBUG_TOOLS) \
PASSWORD=$(PASSWORD) \
USERNAME=$(USERNAME) \
SONIC_BUILD_JOBS=$(SONIC_BUILD_JOBS) \
SONIC_USE_DOCKER_BUILDKIT=$(SONIC_USE_DOCKER_BUILDKIT) \
VS_PREPARE_MEM=$(VS_PREPARE_MEM) \
KERNEL_PROCURE_METHOD=$(KERNEL_PROCURE_METHOD) \
HTTP_PROXY=$(http_proxy) \
HTTPS_PROXY=$(https_proxy) \
SONIC_ENABLE_SYSTEM_TELEMETRY=$(ENABLE_SYSTEM_TELEMETRY) \
EXTRA_JESSIE_TARGETS=$(EXTRA_JESSIE_TARGETS) \
$(SONIC_OVERRIDE_BUILD_VARS)
.PHONY: sonic-slave-build sonic-slave-bash init reset
.DEFAULT_GOAL := all
%::
@$(OVERLAY_MODULE_CHECK)
@docker inspect --type image $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG) &> /dev/null || \
{ echo Image $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG) not found. Building... ; \
$(DOCKER_BASE_BUILD) ; }
@docker inspect --type image $(SLAVE_IMAGE):$(SLAVE_TAG) &> /dev/null || \
{ echo Image $(SLAVE_IMAGE):$(SLAVE_TAG) not found. Building... ; \
$(DOCKER_BUILD) ; }
ifeq "$(KEEP_SLAVE_ON)" "yes"
ifdef SOURCE_FOLDER
@$(DOCKER_RUN) -v $(SOURCE_FOLDER):/var/$(USER)/src $(SLAVE_IMAGE):$(SLAVE_TAG) bash -c "$(SONIC_BUILD_INSTRUCTION) $@; /bin/bash"
else
@$(DOCKER_RUN) $(SLAVE_IMAGE):$(SLAVE_TAG) bash -c "$(SONIC_BUILD_INSTRUCTION) $@; /bin/bash"
endif
else
@$(DOCKER_RUN) $(SLAVE_IMAGE):$(SLAVE_TAG) $(SONIC_BUILD_INSTRUCTION) $@
endif
sonic-slave-build :
$(DOCKER_BASE_BUILD)
$(DOCKER_BUILD)
sonic-slave-bash :
@$(OVERLAY_MODULE_CHECK)
@docker inspect --type image $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG) &> /dev/null || \
{ echo Image $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG) not found. Building... ; \
$(DOCKER_BASE_BUILD) ; }
@docker inspect --type image $(SLAVE_IMAGE):$(SLAVE_TAG) &> /dev/null || \
{ echo Image $(SLAVE_IMAGE):$(SLAVE_TAG) not found. Building... ; \
$(DOCKER_BUILD) ; }
@$(DOCKER_RUN) -t $(SLAVE_IMAGE):$(SLAVE_TAG) bash
showtag:
@echo $(SLAVE_IMAGE):$(SLAVE_TAG)
@echo $(SLAVE_BASE_IMAGE):$(SLAVE_BASE_TAG)
init :
@git submodule update --init --recursive
@git submodule foreach --recursive '[ -f .git ] && echo "gitdir: $$(realpath --relative-to=. $$(cut -d" " -f2 .git))" > .git'
reset :
@echo && echo -n "Warning! All local changes will be lost. Proceed? [y/N]: "
@read ans && \
if [ $$ans == y ]; then \
echo "Resetting local repository. Please wait..."; \
$(DOCKER_RUN) $(SLAVE_IMAGE):$(SLAVE_TAG) sudo rm -rf fsroot; \
git clean -xfdf; \
git reset --hard; \
git submodule foreach --recursive git clean -xfdf; \
git submodule foreach --recursive git reset --hard; \
git submodule update --init --recursive; \
echo "Reset complete!"; \
else \
echo "Reset aborted"; \
fi
Reference in New Issue View Git Blame Copy Permalink