Commit Graph

19 Commits

Author SHA1 Message Date
Kalimuthu-Velappan
aaeafa8411
02.Version cache - docker cache build framework (#12001)
During docker build, host files can be passed to the docker build through
docker context files. But there is no straightforward way to transfer
the files from docker build to host.

This feature provides a tricky way to pass the cache contents from docker
build to host. It tar's the cached content and encodes them as base64 format
and passes it through a log file with a special tag as 'VCSTART and VCENT'.

Slave.mk in the host, it extracts the cache contents from the log and stores them
in the cache folder. Cache contents are encoded as base64 format for
easy passing.

<!--
     Please make sure you've read and understood our contributing guidelines:
     https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md

     ** Make sure all your commits include a signature generated with `git commit -s` **

     If this is a bug fix, make sure your description includes "fixes #xxxx", or
     "closes #xxxx" or "resolves #xxxx"

     Please provide the following information:
-->

#### Why I did it

#### How I did it

#### How to verify it
2022-12-02 08:28:45 +08:00
xumia
9056232bfd
[Build] Cleanup the version deb preference file after build (#11414)
Why I did it
Cleanup the version deb preference file after build.
The version file is no use after build.

How I did it
Remove the no use version file.
2022-07-12 17:57:16 +08:00
xumia
b9ecaa3234
[Build]: Support to use the base image version when a package version not specified (#10971)
Why I did it
It is to fix issue: #10952
[Build]: Support to use the base image version when a package version not specified
2022-05-30 12:06:32 +08:00
xumia
e6aa3b8751
[Build]: Fix pip version constraint conflict issue (#10525)
Why I did it
[Build]: Fix pip version constraint conflict issue
When a version is specified in the constraint file, if upgrading the version in build script, it will have conflict issue.

How I did it
If a specified version has specified in pip command line, then the version constraint will be skipped.
2022-04-13 11:40:06 +08:00
Saikrishna Arcot
9451a9d43e
Check to see that the py2 and py3 version files exist before trying to sort them (#10325)
For Bullseye, Python 2 isn't present at all. This means that in certain
build cases (such as building something only for Bullseye), the version
file may not exist, and so the sort command would fail.

For most normal build commands, this probably won't be an issue, because
the SONiC build will start with Buster (which has both Python 2 and
Python 3 wheels built), and so the py2 and py3 files will be present
even during the Bullseye builds.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
2022-03-22 21:57:15 -07:00
xumia
92363eadbe
[Build]: support to collect version when purging debian package (#10287)
Why I did it
support to collect version when purging debian package
Support to collect version multiple times

How I did it
Add the collection action before purging.
2022-03-21 17:10:33 +08:00
xumia
ed4f19fb28
[Build][Bug]: fix the warning message not printed as expected issue (#10278)
Fix the warning message not printed as expected issue
2022-03-19 21:48:27 +08:00
xumia
7178c668dd
[Build]: Fix installing dpkg packages in parallel issue (#10175)
Why I did it
Fix the debian packages installing in parallel issue.
Add apt hook command to support apt to print no version control info.
2022-03-14 11:20:20 +08:00
Shilong Liu
94b778c39b
[build] docker-sonic-mgmt replace USER by whoami (#9702) 2022-01-11 16:48:04 +08:00
Shilong Liu
3844ae8290
Fix docker-sonic-mgmt reproducible related issue. (#9647)
Reproducible build script breaks docker-sonic-mgmt build.
2022-01-05 11:54:19 +08:00
xumia
471a31fce0
[Bug][Build]: fix the file not found issue caused by the relative pat… (#9443)
Fix the nodesource.list cannot read issue, it is cased by the full path not used.

```
2021-12-03T06:59:26.0019306Z Removing intermediate container 77cfe980cd36
2021-12-03T06:59:26.0020872Z  ---> 528fd40e60f6
2021-12-03T06:59:26.0021457Z Step 81/81 : RUN post_run_buildinfo
2021-12-03T06:59:26.0841136Z  ---> Running in d804bd7e1b06
2021-12-03T06:59:29.1626594Z DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
2021-12-03T06:59:34.2960105Z /usr/bin/sed: can't read nodesource.list: No such file or directory
2021-12-03T06:59:34.5094880Z The command '/bin/sh -c post_run_buildinfo' returned a non-zero code: 2
```

Co-authored-by: Ubuntu <xumia@xumia-vm1.jqzc3g5pdlluxln0vevsg3s20h.xx.internal.cloudapp.net>
2021-12-03 21:23:09 -08:00
xumia
5947406a45
[Build]: Cleanup the reproducible mirrors when build complete (#9132)
Why I did it
The reproducible build mirrors are only used during the build, the mirrors can be removed after that.
2021-12-02 09:04:40 +08:00
xumia
848a1710b0
[Build]: Fix the version not found issue (#9331)
When we update the a sai package downing from a remote server, we need to update the version file as well currently, but the reproducible build feature is not enabled in master, it can only be detected when merging the code into the release branches, such as 202106, 202012, etc.
The reproducible feature is to reduce the build failure, not need to break the build when the version not specified. If version not specified, the best choice is to accept the version from remote server.

Co-authored-by: Ubuntu <xumia@xumia-vm1.jqzc3g5pdlluxln0vevsg3s20h.xx.internal.cloudapp.net>
2021-11-22 12:18:32 -08:00
xumia
ef7bde18b9
Skip the web proxy when downloading from proxy endpoint (#7592)
Why I did it
Skip to use the web proxy when the packages have been in the proxy server.
For sai packages or the other packages, we will upload the the proxy server directly, the reproducible will skip to check the site, not necessary to change the version files.
2021-05-18 14:28:32 +08:00
xumia
1dcab4d1e3
Fix py3 version changed even version control enabled issue (#6422)
* Fix py3 version changed even version control enabled issue

* Add some comments and simplify the script

* Add the comment to explain how to get the not hooked command
2021-01-13 18:40:39 +08:00
xumia
4ef3f1376f
[arista]: Fix web package md5 hash not correct issue (#6411)
Need to add the follow redirection option -L when downloading the package with redirection.
2021-01-11 10:33:08 -08:00
xumia
0a36de3a89
Recover "Support SONiC Reproduceable Build-debian/pip/web packages (#6255)
* Revert "Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)""

This reverts commit 17497a65e3.

* Revert "Revert "Remove unnecessary sudo authority in build Makefile (#6237)""

This reverts commit 163b7111b5.
2020-12-21 15:31:10 +08:00
Guohan Lu
17497a65e3 Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)"
This reverts commit 55a707586b.
2020-12-18 23:37:27 -08:00
xumia
55a707586b
Support SONiC Reproduceable Build-debian/pip/web packages (#5718)
* Support SONiC reproduceable build for deb/py2/py3/web

* Remove j2 files

* Fix bug

* Fix some issues

1. Change some code format issues
2. Fix curl calling wget command, pip2 calling pip3 issue
3. Fix wget/curl downloading multiple urls issue

* Fix some code format issue

* Fix bug

* Fix bug

* Fix command path hard code in build info scripts issue

* Add debian package sonic-build-tools

* Fix auto debian package removed issue

* Change build debian package name, and change the folder

* Collect the pre-versions and post-versions

* Change to use debian:buster

* Remove apt-mark and improve code

* Remove set_build_hooks

* Change docker trusted gpg files

* Fix docker build COPY directory name issue

* Move the trusted gpg files into the sonic-build-hooks package
2020-12-17 13:06:53 +08:00