Commit Graph

36 Commits

Author SHA1 Message Date
Shilong Liu
a3aa8041b9 [build] Fix issues found in reproducible build. (#10407) 2022-04-02 00:01:41 +00:00
xumia
00c13b430f [build][Bug]: Fix the command set_reproducible_mirrors not found issue (#10398)
Why I did it
Fix the command set_reproducible_mirrors not found issue during the build.
2022-04-01 01:25:16 +00:00
xumia
59af63f557 [Build]: Fix host image debian package version issue (#10358)
Why I did it
Fix host image debian package version issue.
The package dependencies may have issue, when some of debian packages of the base image are upgraded. For example, libc is installed in base image, but if the mirror has new version, when running "apt-get upgrade", the package will be upgraded unexpected. To avoid such issue, need to add the versions when building the host image.

How I did it
The package versions of host-image should contain host-base-image.
2022-03-30 09:29:19 +00:00
Shilong Liu
54f80a6127
Add a config variable to override default container registry instead of dockerhub. (#10166) (#10260)
* Add variable to reset default docker registry
* fix bug in docker version control
2022-03-19 00:09:42 +08:00
xumia
c4e8955e69 [Bug][Build]: Fix the package url not found when package name encoded issue (#9683)
When the package name with special characters, such as +, the package name may be encoded as %2b, the package url will not be found when reproducible build enabled.
2022-01-07 13:38:36 +00:00
xumia
0da287ab60 [Build]: Fix tmpfs space not enough issue when building vs image (#9438)
Fix no space left on device issue in tmpfs.
2021-12-01T06:30:40.1651742Z cp: write error: No space left on device
2021-12-01T06:30:40.1652225Z Failure: local_fs_run():/dev/vdb Unable to copy /tmp/tmp.gl4Sgp/onie-installer.bin to tmpfs
2021-12-08 21:29:22 +00:00
xumia
8b333bea87
Fix armhf version issue (#9382)
Why I did it
Fix some of the version files not used issue.
One of example version file version-py3-all-armhf, when building marvell-armhf, the version is used as expected, but it not use.
2021-12-01 07:35:07 +08:00
LuiSzee
5b284767f6 Update Centec platform support for Bullseye and 5.10 kernel (#7)
1. Fix build for armhf and arm64
2. upgrade centec tsingma bsp support to 5.10 kernel
3. modify centec platform driver for linux 5.10

Co-authored-by: Shi Lei <shil@centecnetworks.com>
2021-11-10 15:27:22 -08:00
Chris Ward
f8688bef8d
[kvm] improve the handling of compact_memory during kvm image build (#9184)
Build failed on a Ubuntu 20.04 system with kvm kernel, which does not have the /proc/sys/vm/compact_memory 
Should check if compact_memory is writeable before doing it.

Signed-off-by: Chris Ward <tjcw@uk.ibm.com>
2021-11-10 08:31:03 -08:00
Shilong Liu
c44dbf0fe0
[build] Fix reproducible build issues (#8548)
* [build] Fix reproducible build issues
2021-08-26 18:21:18 +08:00
Shilong Liu
ab75260d20
Reproducible build add docker image debian* to white list. (#8330)
#### Why I did it
1. Add version control for debian* docker image to white list.
2. Always record docker image sha256 value, regardless of white list.
2021-08-09 19:55:49 -07:00
Shilong Liu
4b3b4c0ce1
Reproducible build support docker image (#8039)
when build docker image, we need a base image. now base image is specified by tag. we can use sha256 to specify docker image.
2021-07-09 14:30:26 +08:00
xumia
9d66b6118d
Fix the apt-get download package disorder issue (#8021)
Why I did it
Multiple build failed in 202012 branch
It is caused by the disorder of the package urls retrieved from the command "apt-get download --print-urls "
2021-06-30 15:42:57 +08:00
xumia
129f803e11 Fix the /var wrong permission issue (#7571)
Why I did it
Build broken when reproducible build enabled, relative to backport package systemd.
The root cause is the bad permission of /var.
2021-05-12 07:54:28 -07:00
Stepan Blyshchak
cd2c86eab6
[dockers] label SONiC Docker with manifest (#5939)
Signed-off-by: Stepan Blyschak stepanb@nvidia.com

This PR is part of SONiC Application Extension

Depends on #5938

- Why I did it
To provide an infrastructure change in order to support SONiC Application Extension feature.

- How I did it
Label every installable SONiC Docker with a minimal required manifest and auto-generate packages.json file based on
installed SONiC images.

- How to verify it
Build an image, execute the following command:

admin@sonic:~$ docker inspect docker-snmp:1.0.0 | jq '.[0].Config.Labels["com.azure.sonic.manifest"]' -r | jq
Cat /var/lib/sonic-package-manager/packages.json file to verify all dockers are listed there.
2021-04-26 13:51:50 -07:00
guxianghong
6fe6d7394d
[arm] support compile sonic arm image on arm server (#7285)
- Support compile sonic arm image on arm server. If arm image compiling is executed on arm server instead of using qemu mode on x86 server, compile time can be saved significantly.
- Add kernel argument systemd.unified_cgroup_hierarchy=0 for upgrade systemd to version 247, according to #7228
- rename multiarch docker to sonic-slave-${distro}-march-${arch}

Co-authored-by: Xianghong Gu <xgu@centecnetworks.com>
Co-authored-by: Shi Lei <shil@centecnetworks.com>
2021-04-18 08:17:57 -07:00
Yilan
d3fae0080e
[build]: Update versions_manager.py to make versions map key unique (#7146)
py2/py3/deb packages names are case insensitive, and the versions map
key should be the same for packages whose name can have different cases.

For example, in files/build/versions/default/versions-py3, package
"click==7.1.2" is pinned; and in
files/build/versions/dockers/docker-sonic-vs/versions-py3, package
"Click==7.0" is pinned.
Without this fix, the aggregated versions-py3 file used for building
docker-sonic-vs looks like below:
...
click==7.1.2
Click==7.0
...
However, we actually want "click==7.0" to overwrite "click==7.1.2" for
docker-sonic-vs build.
2021-03-30 08:34:25 -07:00
Guohan Lu
3001bbcd28 [kvm]: print running process when build kvm fails
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-02-06 15:11:31 -08:00
Guohan Lu
83c51e4803 [kvm]: install net-tools package for debug
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-01-30 17:38:33 -08:00
lguohan
9b2bc880de
[kvm]: add debug cmd for build_kvm_image.sh (#6472)
dump netstat info on error

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-01-17 11:05:33 -08:00
xumia
95936805e0
Install the latest version of the sonic build hooks in slave container (#6348) 2021-01-05 19:05:13 +08:00
xumia
36fbc01a59
Fix the hostimage version path permission issue (#6337) 2021-01-03 17:32:20 +08:00
xumia
0a36de3a89
Recover "Support SONiC Reproduceable Build-debian/pip/web packages (#6255)
* Revert "Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)""

This reverts commit 17497a65e3.

* Revert "Revert "Remove unnecessary sudo authority in build Makefile (#6237)""

This reverts commit 163b7111b5.
2020-12-21 15:31:10 +08:00
Guohan Lu
17497a65e3 Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)"
This reverts commit 55a707586b.
2020-12-18 23:37:27 -08:00
xumia
55a707586b
Support SONiC Reproduceable Build-debian/pip/web packages (#5718)
* Support SONiC reproduceable build for deb/py2/py3/web

* Remove j2 files

* Fix bug

* Fix some issues

1. Change some code format issues
2. Fix curl calling wget command, pip2 calling pip3 issue
3. Fix wget/curl downloading multiple urls issue

* Fix some code format issue

* Fix bug

* Fix bug

* Fix command path hard code in build info scripts issue

* Add debian package sonic-build-tools

* Fix auto debian package removed issue

* Change build debian package name, and change the folder

* Collect the pre-versions and post-versions

* Change to use debian:buster

* Remove apt-mark and improve code

* Remove set_build_hooks

* Change docker trusted gpg files

* Fix docker build COPY directory name issue

* Move the trusted gpg files into the sonic-build-hooks package
2020-12-17 13:06:53 +08:00
xumia
918ce2a633
Simplify the function to covert pfx certificate (#5086) 2020-08-01 20:07:20 +08:00
xumia
81e0777b47
Support to convert certificate from pfx to pem format (#5073)
* Support to convert certificate from pfx to pem format

* Fix code issue
2020-07-31 12:24:54 +08:00
xumia
e28cb8f58e
[Secure Boot] Support to sign swi image (#4627)
* [secure boot] Support to sign swi image

* Fix build issue

* fix tab format issue

* Fix typing issue

* Change the sign_image.sh command line

* Remove SONIC_CETIFICATE_PATH

* Fix bugs
2020-06-09 15:25:17 +08:00
Guohan Lu
f1cc577c75 [build_kvm]: mount /proc to show memory
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-17 04:51:51 +00:00
lguohan
5299644dc0
[kvmbuild]: print out kvm_log on error (#4384)
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-07 12:31:20 -07:00
lguohan
20260ceb1d
[build]: add SONIC_CONFIG_BUILD_LOG_TIMESTAMP to add timestamp in build log (#4269)
add timestamp in each job build log

example:

   [01:39:21] dh clean  --with autotools-dev
   [01:39:22]    dh_auto_clean
   [01:39:27]      make -j16 distclean

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-21 14:21:26 -07:00
Guohan Lu
bd72844f75 [kvm]: increase the kvm installer size to 2G for dbg image
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-12-28 21:21:09 +00:00
lguohan
8d4516ca3d
[kvm]: increase mem to 3G to avoid OOM during onie installation (#3811)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-11-23 12:30:22 -08:00
Renuka Manavalan
14458b79ba
Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y (#3323)
* Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y

* Dropped -v (verbose flag) from tar command
2019-08-28 09:29:48 -07:00
Renuka Manavalan
76bf5a0bc4 [build]: Added debug symbols to many debug dockers. (#3098)
* Added debug symbols to many debug dockers.

* For debug images *only*:
1) Archive source files into debug image
2) Archived source is copied into /src
3) Created an empty dir /debug
4) Mount both /src as ro & /debug as rw into every docker
5) Login banner will give some details on /src & /debug
6) Devs can copy core file into /debug and view it from inside a container.
7) Dev may create all gdb logs and other data directly into /debug.

* Dropped redundant REDIS_TOOLS per review comments.

* Added debug symbols to frr package and hence FRR based BGP docker.

* 1) Moved dbg_files.sh to scripts/
2) Src directories to archive are now collected from individual Makefiles.
3) Added few more debug symbols
4) Added few more debug dockers.

Here after no more changes except per review comments.

To debug:
Install required version of debug image in Switch or VM.
Copy core file into /debug of host
Get into Docker
gdb /usr/bin/<daemon> -c /debug/<your core file>
set directory /src/... <-- inside gdb to get the source

For non-in-depth debugging:

Download corresponding debug Docker image (docker-...-dbg.gz) to your VM
Load the image
Run image with entrypoint as 'bash' with dir containing core mapped in.
Run gdb on the core.
2019-07-03 22:13:55 -07:00
lguohan
40330bcbdf
[build]: wait up to 60 seconds for docker engine to start (#3067)
* [build]: wait 60 seconds for docker engine to start

On some platforms, it can take more than 1 second for docker
engine to start.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-06-22 14:40:05 -07:00