From 826851f948ee5eac81b65d5cf2be51256459073c Mon Sep 17 00:00:00 2001 From: Marian Pritsak Date: Tue, 20 Dec 2016 12:34:05 +0200 Subject: [PATCH] Update README.buildsystem.md Add section with recipes and target groups Add section with tips & tricks --- README.buildsystem.md | 126 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 117 insertions(+), 9 deletions(-) diff --git a/README.buildsystem.md b/README.buildsystem.md index 95476c88d4..947ee6b54c 100644 --- a/README.buildsystem.md +++ b/README.buildsystem.md @@ -1,6 +1,6 @@ # SONiC Buildimage Guide ## Overview -SONiC build system is a *GNU make* based environment for build process automation. +SONiC buildimage is a *GNU make* based environment for build process automation. It consists of two main parts: * Backend - collection of makefiles and other helpers that define generic target groups, used by recipes * Frontend - collection of recipes, that define metadata for each build target @@ -38,10 +38,11 @@ sonic-buildimage/ ### Backend **Makefile**, **slave.mk** and **sonic-slave/Dockerfile** are the backend of buildimage. *slave.mk* is the actual makefile. It defines a set of rules for *target groups* (more on that later). -You can find a make rules for every target that is defined in recipe there. -*Makefile* is a wrapper over sonic-slave docker image. +You can find a make rule for every target that is defined in recipe there. +*Makefile* is a wrapper over sonic-slave docker image. + Every part of build is executed in a docker container called sonic-slave, specifically crafted for this environment. -If build is started for the first time, a new sonic-slave image will be built form *sonic-slave/Dockerfile* on the machine. +If build is started for the first time on a particular host, a new sonic-slave image will be built form *sonic-slave/Dockerfile* on the machine. It might take some time, so be patient. After that all subsequent make commands will be executed inside this container. *Makefile* takes every target that is passed to make command and delegates it as an entry point to a container, @@ -60,13 +61,16 @@ Every **platform/[VENDOR]/** directory is a derived part of buildimage frontend, ### Build output **target/** is basically a build output. You can find all biuld artifacts there. + ## Recipes and target groups Now let's go over a definition of recipes and target groups. -*Recipe* is a small makefile that defines a target and set of variables for building it. -*Target group* is a set of targets that are built according to the same rulels. +**Recipe** is a small makefile that defines a target and set of variables for building it. +If you want to add a new target to buildimage (.deb package or docker image), you have to create a recipe for this target. +**Target group** is a set of targets that are built according to the same rulels. +Every recipe sets a target group to which this target belongs. ### Recipe example -Lets take a recipe for swss as an example: +Lets take a recipe for swsscommon as an example: ```make # libswsscommon package @@ -91,6 +95,110 @@ At the end we define a dev package for swsscommon and make it derived from main Using **add_derived_package** just makes a deep copy of package's metadata, so that we don't have to repeat ourselves. ### Target groups -**TODO** +**SONIC_DPKG_DEBS** +Main target group for building .deb packages. +Define: +```make +SOME_NEW_DEB = some_new_deb.deb # name of your package +$(SOME_NEW_DEB)_SRC_PATH = $(SRC_PATH)/project_name # path to directory with sources +$(SOME_NEW_DEB)_DEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # build dependencies +$(SOME_NEW_DEB)_RDEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # runtime dependencies +SONIC_DPKG_DEBS += $(SOME_NEW_DEB) # add package to this target group +``` + +**SONIC_PYTHON_STDEB_DEBS** +Same as above, but instead of building package using dpkg-buildpackage it executes `python setup.py --command-packages=stdeb.command bdist_deb`. +Define: +```make +SOME_NEW_DEB = some_new_deb.deb # name of your package +$(SOME_NEW_DEB)_SRC_PATH = $(SRC_PATH)/project_name # path to directory with sources +$(SOME_NEW_DEB)_DEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # build dependencies +$(SOME_NEW_DEB)_RDEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # runtime dependencies +SONIC_PYTHON_STDEB_DEBS += $(SOME_NEW_DEB) # add package to this target group +``` + +**SONIC_MAKE_DEBS** +This is a bit more flexible case. +If you have to do some specific type of build or apply pathes prior to build, just define your owm Makefile and add it to buildomage. +Define: +```make +SOME_NEW_DEB = some_new_deb.deb # name of your package +$(SOME_NEW_DEB)_SRC_PATH = $(SRC_PATH)/project_name # path to directory with sources +$(SOME_NEW_DEB)_DEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # build dependencies +$(SOME_NEW_DEB)_RDEPENDS = $(SOME_OTHER_DEB1) $(SOME_OTHER_DEB2) ... # runtime dependencies +SONIC_MAKE_DEBS += $(SOME_NEW_DEB) # add package to this target group +``` + +If some packages have to be built locally due to some legal issues or they are already prebuilt and available online, you might find next four target groups useful. + +**SONIC_COPY_DEBS** +Those packages will be just copied from specified location on your machine. +Define: +```make +SOME_NEW_DEB = some_new_deb.deb # name of your package +$(SOME_NEW_DEB)_PATH = path/to/some_new_deb.deb # path to file +SONIC_COPY_DEBS += $(SOME_NEW_DEB) # add package to this target group +``` + +**SONIC_COPY_FILES** +Same as above, but applicable for regular files. Use when you need some regular files for installation in docker container. +Define: +```make +SOME_NEW_FILE = some_new_file # name of your file +$(SOME_NEW_FILE)_PATH = path/to/some_new_file # path to file +SONIC_COPY_FILES += $(SOME_NEW_FILE) # add file to this target group +``` + +**SONIC_ONLINE_DEBS** +Target group for debian packages that should be fetched from an online source. +Define: +```make +SOME_NEW_DEB = some_new_deb.deb # name of your package +$(SOME_NEW_DEB)_URL = https://url/to/this/deb.deb # path to file # URL for downloading +SONIC_ONLINE_DEBS += $(SOME_NEW_DEB) # add file to this target group +``` + +**SONIC_ONLINE_FILES** +Target group for regular files that should be fetched from an online source. +Define: +```make +SOME_NEW_FILE = some_new_file # name of your file +$(SOME_NEW_FILE)_URL = https://url/to/this/file # URL for downloading +SONIC_ONLINE_FILES += $(SOME_NEW_FILE) # add file to this target group +``` + +Docker images also have their target groups. +**SONIC_SIMPLE_DOCKER_IMAGES** +As you see from a name of the group, it is intended to build a docker image from a regular Dockerfile. +Define: +```make +SOME_DOCKER = some_docker.gz # name of your docker +$(SOME_DOCKER)_PATH = path/to/your/docker # path to your Dockerfile +SONIC_SIMPLE_DOCKER_IMAGES += $(SOME_DOCKER) # add docker to this group +``` + +**SONIC_DOCKER_IMAGES** +This one is a bit more sophisticated. You can define debian packages from buildimage that will be installed to it, and corresponding Dockerfile will be dinamically generated from a template. +Define: +```make +SOME_DOCKER = some_docker.gz # name of your docker +$(SOME_DOCKER)_PATH = path/to/your/docker # path to your Dockerfile +$(SOME_DOCKER)_DEPENDS += $(SOME_DEB1) $(SOME_DEB2) # .deb packages to install into image +$(SOME_DOCKER)_PYTHON_WHEELS += $(SOME_WHL1) $(SOME_WHL2) # python wheels to install into image +$(SOME_DOCKER)_LOAD_DOCKERS += $(SOME_OTHER_DOCkER) # docker image from which this one is built +SONIC_DOCKER_IMAGES += $(SOME_DOCKER) # add docker to this group +``` + ## Tips & Tricks -**TODO** +Although every target is built inside a sonic-slave container, which exits at the end of build, you can enter bash of sonic-slave using this command: +``` +$ sonic-slave-bash +``` +It is very useful for debugging when you add a new target and facing some trubles. + +sonic-slave environment is built only once, but if sonic-slave/Dockerfile was updated, you can rebuild it with this command: +``` +$ sonic-slave-build +``` + +All target groups are used by one or another recipe, so use those recipes as a reference when adding new ones.