sonic-buildimage/dockers/docker-base-buster/Dockerfile.j2

128 lines
5.1 KiB
Docker
Raw Normal View History

{% set prefix = DEFAULT_CONTAINER_REGISTRY %}
{% from "dockers/dockerfile-macros.j2" import install_debian_packages, install_python_wheels, copy_files %}
Ported Marvell armhf build on amd64 host for debian buster to use cross-comp… (#8035) * Ported Marvell armhf build on x86 for debian buster to use cross-compilation instead of qemu emulation Current armhf Sonic build on amd64 host uses qemu emulation. Due to the nature of the emulation it takes a very long time, about 22-24 hours to complete the build. The change I did to reduce the building time by porting Sonic armhf build on amd64 host for Marvell platform for debian buster to use cross-compilation on arm64 host for armhf target. The overall Sonic armhf building time using cross-compilation reduced to about 6 hours. Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Fixed final Sonic image build with dockers inside * Update Dockerfile.j2 Fixed qemu-user-static:x86_64-aarch64-5.0.0-2 . * Update cross-build-arm-python-reqirements.sh Added support for both armhf and arm64 cross-build platform using $PY_PLAT environment variable. * Update Makefile Added TARGET=<cross-target> for armhf/arm64 cross-compilation. * Reviewer's @qiluo-msft requests done Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Added new radius/pam patch for arm64 support * Update slave.mk Added missing back tick. * Added libgtest-dev: libgmock-dev: to the buster Dockerfile.j2. Fixed arm perl version to be generic * Added missing armhf/arm64 entries in /etc/apt/sources.list * fix libc-bin core dump issue from xumia:fix-libc-bin-install-issue commit * Removed unnecessary 'apt-get update' from sonic-slave-buster/Dockerfile.j2 * Fixed saiarcot895 reviewer's requests * Fixed README and replaced 'sed/awk' with patches * Fixed ntp build to use openssl * Unuse sonic-slave-buster/cross-build-arm-python-reqirements.sh script (put all prebuilt python packages cross-compilation/install inside Dockerfile.j2). Fixed src/snmpd/Makefile to use -j1 in all cases * Clean armhf cross-compilation build fixes * Ported cross-compilation armhf build to bullseye * Additional change for bullseye * Set CROSS_BUILD_ENVIRON default value n * Removed python2 references * Fixes after merge with the upstream * Deleted unused sonic-slave-buster/cross-build-arm-python-reqirements.sh file * Fixed 2 @saiarcot895 requests * Fixed @saiarcot895 reviewer's requests * Removed use of prebuilt python wheels * Incorporated saiarcot895 CC/CXX and other simplification/generalization changes Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Fixed saiarcot895 reviewer's additional requests * src/libyang/patch/debian-packaging-files.patch * Removed --no-deps option when installing wheels. Removed unnecessary lazy_object_proxy arm python3 package instalation Co-authored-by: marvell <marvell@cpss-build3.marvell.com> Co-authored-by: marvell <marvell@cpss-build2.marvell.com>
2022-07-21 16:15:16 -05:00
{% if CONFIGURED_ARCH == "armhf" and (MULTIARCH_QEMU_ENVIRON == "y" or CROSS_BUILD_ENVIRON == "y") %}
FROM {{ prefix }}multiarch/debian-debootstrap:armhf-buster
Ported Marvell armhf build on amd64 host for debian buster to use cross-comp… (#8035) * Ported Marvell armhf build on x86 for debian buster to use cross-compilation instead of qemu emulation Current armhf Sonic build on amd64 host uses qemu emulation. Due to the nature of the emulation it takes a very long time, about 22-24 hours to complete the build. The change I did to reduce the building time by porting Sonic armhf build on amd64 host for Marvell platform for debian buster to use cross-compilation on arm64 host for armhf target. The overall Sonic armhf building time using cross-compilation reduced to about 6 hours. Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Fixed final Sonic image build with dockers inside * Update Dockerfile.j2 Fixed qemu-user-static:x86_64-aarch64-5.0.0-2 . * Update cross-build-arm-python-reqirements.sh Added support for both armhf and arm64 cross-build platform using $PY_PLAT environment variable. * Update Makefile Added TARGET=<cross-target> for armhf/arm64 cross-compilation. * Reviewer's @qiluo-msft requests done Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Added new radius/pam patch for arm64 support * Update slave.mk Added missing back tick. * Added libgtest-dev: libgmock-dev: to the buster Dockerfile.j2. Fixed arm perl version to be generic * Added missing armhf/arm64 entries in /etc/apt/sources.list * fix libc-bin core dump issue from xumia:fix-libc-bin-install-issue commit * Removed unnecessary 'apt-get update' from sonic-slave-buster/Dockerfile.j2 * Fixed saiarcot895 reviewer's requests * Fixed README and replaced 'sed/awk' with patches * Fixed ntp build to use openssl * Unuse sonic-slave-buster/cross-build-arm-python-reqirements.sh script (put all prebuilt python packages cross-compilation/install inside Dockerfile.j2). Fixed src/snmpd/Makefile to use -j1 in all cases * Clean armhf cross-compilation build fixes * Ported cross-compilation armhf build to bullseye * Additional change for bullseye * Set CROSS_BUILD_ENVIRON default value n * Removed python2 references * Fixes after merge with the upstream * Deleted unused sonic-slave-buster/cross-build-arm-python-reqirements.sh file * Fixed 2 @saiarcot895 requests * Fixed @saiarcot895 reviewer's requests * Removed use of prebuilt python wheels * Incorporated saiarcot895 CC/CXX and other simplification/generalization changes Signed-off-by: marvell <marvell@cpss-build3.marvell.com> * Fixed saiarcot895 reviewer's additional requests * src/libyang/patch/debian-packaging-files.patch * Removed --no-deps option when installing wheels. Removed unnecessary lazy_object_proxy arm python3 package instalation Co-authored-by: marvell <marvell@cpss-build3.marvell.com> Co-authored-by: marvell <marvell@cpss-build2.marvell.com>
2022-07-21 16:15:16 -05:00
{% elif CONFIGURED_ARCH == "arm64" and (MULTIARCH_QEMU_ENVIRON == "y" or CROSS_BUILD_ENVIRON == "y") %}
FROM {{ prefix }}multiarch/debian-debootstrap:arm64-buster
{% else %}
FROM {{ prefix }}{{DOCKER_BASE_ARCH}}/debian:buster
{% endif %}
# Clean documentation in FROM image
RUN find /usr/share/doc -depth \( -type f -o -type l \) ! -name copyright | xargs rm || true
# Clean doc directories that are empty or only contain empty directories
RUN while [ -n "$(find /usr/share/doc -depth -type d -empty -print -exec rmdir {} +)" ]; do :; done && \
rm -rf \
/usr/share/man/* \
/usr/share/groff/* \
/usr/share/info/* \
/usr/share/lintian/* \
/usr/share/linda/* \
/var/cache/man/* \
/usr/share/locale/*
# Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive
# Configure data sources for apt/dpkg
COPY ["dpkg_01_drop", "/etc/dpkg/dpkg.cfg.d/01_drop"]
COPY ["sources.list.{{ CONFIGURED_ARCH }}", "/etc/apt/sources.list"]
COPY ["no_install_recommend_suggest", "/etc/apt/apt.conf.d"]
COPY ["no-check-valid-until", "/etc/apt/apt.conf.d"]
Improve remote fetch (#12795) ### Why I did it To fix those errors: One: ``` Connecting to urm.nvidia.com (urm.nvidia.com)|*.*.*.*|:443... connected. GnuTLS: Error in the pull function. Unable to establish SSL connection. Error 4 make[1]: Leaving directory '/sonic/src/smartmontools' [ target/debs/bullseye/smartmontools_6.6-1_amd64.deb ] ``` Second: ``` Get:90 https://debian-mirror-url buster/main amd64 librrd-dev amd64 1.7.1-2 [284 kB] Get:91 https://debian-mirror-url buster/main amd64 psmisc amd64 23.2-1+deb10u1 [126 kB] Get:92 https://debian-mirror-url buster/main amd64 python-smbus amd64 4.1-1 [12.2 kB] Get:93 https://debian-mirror-url buster/main amd64 python3.7-dev amd64 3.7.3-2+deb10u3 [510 kB] Get:94 https://debian-mirror-url buster/main amd64 python3-dev amd64 3.7.3-1 [1264 B] Get:95 https://debian-mirror-url buster/main amd64 python3-smbus amd64 4.1-1 [12.5 kB] Get:96 https://debian-mirror-url buster/main amd64 rrdtool amd64 1.7.1-2 [485 kB] Fetched 122 MB in 12s (9976 kB/s) E: Failed to fetch https://debian-mirror-url/pool/main/p/python-defaults/python2-minimal_2.7.16-1_amd64.deb 500 Internal Server Error [IP: *.*.*.* 443] E: Failed to fetch https://debian-mirror-url/pool/main/f/fontconfig/fontconfig-config_2.13.1-2_all.deb 500 Internal Server Error [IP: *.*.*.* 443] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? The command '/bin/sh -c apt-get update && apt-get install -y build-essential python3-dev ipmitool librrd8 librrd-dev rrdtool python-smbus python3-smbus dmidecode i2c-tools psmisc libpci3' returned a non-zero code: 100 [ target/docker-platform-monitor.gz ] Error 1 ``` #### How I did it Add retry mechanism to apt, wget, and curl hooks
2023-09-23 20:07:04 -05:00
COPY ["apt-retries-count", "/etc/apt/apt.conf.d"]
# Update apt cache and
# pre-install fundamental packages
RUN apt-get update && \
apt-get -y install \
curl \
less \
perl \
procps \
python3 \
python3-distutils \
python3-pip \
rsyslog \
vim-tiny \
# Install dependencies of redis-tools
libatomic1 \
libjemalloc2 \
liblua5.1-0 \
lua-bitop \
lua-cjson \
# common dependencies
libdaemon0 \
libdbus-1-3 \
libjansson4 \
# ip and ifconfig utility missing in docker for arm arch
iproute2 \
net-tools \
# for processing/handling json files in bash environment
jq \
# for sairedis zmq rpc channel
libzmq5
# Install redis-tools
{% if CONFIGURED_ARCH == "armhf" %}
RUN curl -k -o redis-tools_6.0.6-1~bpo10+1_armhf.deb "https://sonicstorage.blob.core.windows.net/packages/redis/redis-tools_6.0.6-1_bpo10+1_armhf.deb?sv=2015-04-05&sr=b&sig=67vHAMxsl%2BS3X1KsqhdYhakJkGdg5FKSPgU8kUiw4as%3D&se=2030-10-24T04%3A22%3A40Z&sp=r"
RUN dpkg -i redis-tools_6.0.6-1~bpo10+1_armhf.deb || apt-get install -f -y
RUN rm redis-tools_6.0.6-1~bpo10+1_armhf.deb
{% elif CONFIGURED_ARCH == "arm64" %}
RUN curl -o redis-tools_6.0.6-1~bpo10+1_arm64.deb "https://sonicstorage.blob.core.windows.net/packages/redis/redis-tools_6.0.6-1_bpo10+1_arm64.deb?sv=2015-04-05&sr=b&sig=GbkJV2wWln3hoz27zKi5erdk3NDKrAFrQriA97bcRCY%3D&se=2030-10-24T04%3A22%3A21Z&sp=r"
RUN dpkg -i redis-tools_6.0.6-1~bpo10+1_arm64.deb || apt-get install -f -y
RUN rm redis-tools_6.0.6-1~bpo10+1_arm64.deb
{% else %}
RUN curl -o redis-tools_6.0.6-1~bpo10+1_amd64.deb "https://sonicstorage.blob.core.windows.net/packages/redis/redis-tools_6.0.6-1~bpo10+1_amd64.deb?sv=2015-04-05&sr=b&sig=73zbmjkf3pi%2Bn0R8Hy7CWT2EUvOAyzM5aLYJWCLySGM%3D&se=2030-09-06T19%3A44%3A59Z&sp=r"
RUN dpkg -i redis-tools_6.0.6-1~bpo10+1_amd64.deb || apt-get install -f -y
RUN rm redis-tools_6.0.6-1~bpo10+1_amd64.deb
{% endif %}
# Upgrade pip via PyPI and uninstall the Debian version
RUN pip3 install --upgrade pip
RUN apt-get purge -y python3-pip
# setuptools and wheel are necessary for installing some Python wheel packages
RUN pip3 install --no-cache-dir setuptools==49.6.00
RUN pip3 install --no-cache-dir wheel==0.35.1
# For templating
RUN pip3 install j2cli
# Install supervisor
RUN pip3 install supervisor==4.2.1
# Add support for supervisord to handle startup dependencies
RUN pip3 install supervisord-dependent-startup==1.4.0
RUN mkdir -p /var/log/supervisor /etc/supervisor/conf.d
RUN apt-get -y purge \
exim4 \
exim4-base \
exim4-config \
exim4-daemon-light
{% if docker_base_buster_debs.strip() -%}
# Copy locally-built Debian package dependencies
{{ copy_files("debs/", docker_base_buster_debs.split(' '), "/debs/") }}
# Install built Debian packages and implicitly install their dependencies
{{ install_debian_packages(docker_base_buster_debs.split(' ')) }}
{%- endif %}
# Clean up apt
# Remove /var/lib/apt/lists/*, could be obsoleted for derived images
RUN apt-get clean -y && \
apt-get autoclean -y && \
apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/* /tmp/* ~/.cache/
COPY ["etc/rsyslog.d/*", "/etc/rsyslog.d/"]
COPY ["root/.vimrc", "/root/.vimrc"]
RUN ln /usr/bin/vim.tiny /usr/bin/vim
COPY ["etc/supervisor/supervisord.conf", "/etc/supervisor/"]
COPY ["etc/supervisor/containercfgd.conf", "/etc/supervisor/conf.d/"]