sonic-buildimage/dockers/docker-ptf/Dockerfile.j2
Zhaohui Sun 1effff9836
Enable system-site-packages for ptf docker and install thrift for test_qos_sai (#12094)
Why I did it
test_sai_qos failed because of the following error:

"stderr_lines": [
        "Traceback (most recent call last):", 
        "  File \"/usr/bin/ptf\", line 522, in <module>", 
        "    test_modules = load_test_modules(config)", 
        "  File \"/usr/bin/ptf\", line 413, in load_test_modules", 
        "    mod = imp.load_module(modname, *imp.find_module(modname, [root]))", 
        "  File \"saitests/switch.py\", line 19, in <module>", 
        "    import switch_sai_thrift", 
        "ImportError: No module named switch_sai_thrift"
    ], 

It's because test_sai_qos runs ptf script which imports switch_sai_thrift, switch_sai_thrift is installed from python-saithrift_0.9.4_amd64.deb.
For master image, the deb file is for python3, but ptf only has virtual python3 environment, that's why we add --system-site-packages to allow virtual env to access system site-packeges.

Add thrift package in docker ptf virtual python3 env, because currently env-python3 doesn't have thrift module which is needed in switch_sai_thrift.

How I did it
Enable --system-site-packages for virtual py3 env in ptf docker and install thrift for test_qos_sai

How to verify it
load and login ptf conatiner
dpkg - i python-saithrift_0.9.4_amd64.deb
source /root/env-python3/bin/activate
python
import switch_sai_thrift.switch_sai_rpc
Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
2022-09-17 13:33:53 +08:00

229 lines
7.1 KiB
Django/Jinja

{% set prefix = DEFAULT_CONTAINER_REGISTRY %}
{% if CONFIGURED_ARCH == "armhf" and MULTIARCH_QEMU_ENVIRON == "y" %}
FROM {{ prefix }}multiarch/debian-debootstrap:armhf-stretch
{% elif CONFIGURED_ARCH == "arm64" and MULTIARCH_QEMU_ENVIRON == "y" %}
FROM {{ prefix }}multiarch/debian-debootstrap:arm64-stretch
{% else %}
FROM {{ prefix }}debian:buster
{% endif %}
{% from "dockers/dockerfile-macros.j2" import install_python_wheels, copy_files %}
USER root
WORKDIR /root
MAINTAINER Pavel Shirshov
RUN echo "deb [arch=amd64] http://debian-archive.trafficmanager.net/debian buster-backports main" >> /etc/apt/sources.list
## Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive
## Set the apt source, update package cache and install necessary packages
## TODO: Clean up this step
RUN sed --in-place 's/httpredir.debian.org/debian-archive.trafficmanager.net/' /etc/apt/sources.list \
&& apt-get update \
&& apt-get upgrade -y \
&& apt-get dist-upgrade -y \
&& apt-get install -y \
autoconf \
openssh-server \
vim \
telnet \
net-tools \
traceroute \
lsof \
tcpdump \
ethtool \
unzip \
pkg-config \
binutils \
build-essential \
libssl-dev \
libffi-dev \
wget \
cmake \
libqt5core5a \
libqt5network5 \
libboost-atomic1.71.0 \
less \
git \
iputils-ping \
hping3 \
curl \
tmux \
python \
python-dev \
python-libpcap \
python-scapy \
python-six \
python3 \
python3-venv \
python3-pip \
python3-dev \
python3-scapy \
python3-six \
libpcap-dev \
tacacs+ \
rsyslog \
ntp \
ntpstat \
ntpdate \
arping \
bridge-utils \
libteam-utils \
gdb \
automake \
iproute2
# Install all python modules from pypi. python-scapy is exception, ptf debian package requires python-scapy
# TODO: Clean up this step
RUN rm -rf /debs \
&& apt-get -y autoclean \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/* \
&& wget --https-only https://bootstrap.pypa.io/pip/2.7/get-pip.py \
&& python get-pip.py \
&& rm -f get-pip.py \
&& pip install setuptools \
&& pip install supervisor \
&& pip install ipython==5.4.1 \
&& git clone https://github.com/p4lang/scapy-vxlan.git \
&& cd scapy-vxlan \
&& python setup.py install \
&& cd .. \
&& rm -fr scapy-vxlan \
&& git clone https://github.com/sflow/sflowtool \
&& cd sflowtool \
&& ./boot.sh \
&& ./configure \
&& make \
&& make install \
&& cd .. \
&& rm -fr sflowtool \
&& git clone https://github.com/dyninc/OpenBFDD.git \
&& cd OpenBFDD \
&& ./autogen.sh \
&& ./configure \
&& make \
&& make install \
&& cd .. \
&& rm -fr OpenBFDD \
&& wget https://github.com/nanomsg/nanomsg/archive/1.0.0.tar.gz \
&& tar xvfz 1.0.0.tar.gz \
&& cd nanomsg-1.0.0 \
&& mkdir -p build \
&& cd build \
&& cmake .. \
&& make install \
&& ldconfig \
&& cd ../.. \
&& rm -fr nanomsg-1.0.0 \
&& rm -f 1.0.0.tar.gz \
&& pip install cffi \
&& pip install nnpy \
&& pip install dpkt \
&& pip install ipaddress \
&& pip install pysubnettree \
&& pip install paramiko \
&& pip install flask \
&& pip install exabgp==3.4.17\
&& pip install pyaml \
&& pip install pybrctl pyro4 rpyc yabgp \
&& pip install unittest-xml-reporting \
&& pip install pyrasite \
&& pip install retrying \
&& mkdir -p /opt \
&& cd /opt \
&& wget https://raw.githubusercontent.com/p4lang/ptf/master/ptf_nn/ptf_nn_agent.py
RUN python3 -m venv --system-site-packages env-python3
# Activating a virtualenv. The virtualenv automatically works for RUN, ENV and CMD.
ENV VIRTUAL_ENV=/root/env-python3
ARG BACKUP_OF_PATH="$PATH"
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 PYTHONIOENCODING=UTF-8
RUN python3 -m pip install --upgrade --ignore-installed pip
# Install all python modules from pypi. python3-scapy is exception, ptf debian package requires python3-scapy
RUN python3 -m pip install setuptools \
&& pip3 install supervisor \
&& pip3 install ipython==5.4.1 \
&& pip3 install Cython \
&& pip3 install cffi \
&& pip3 install nnpy \
&& pip3 install dpkt \
&& pip3 install ipaddress \
&& pip3 install pysubnettree \
&& pip3 install paramiko \
&& pip3 install Flask \
&& pip3 install exabgp \
&& pip3 install pyaml \
&& pip3 install pybrctl pyro4 rpyc yabgp \
&& pip3 install unittest-xml-reporting \
&& pip3 install pyrasite \
&& pip3 install python-libpcap \
&& pip3 install enum34 \
&& pip3 install grpcio \
&& pip3 install grpcio-tools \
&& pip3 install protobuf \
&& pip3 install six==1.16.0 \
&& pip3 install itsdangerous \
&& pip3 install retrying \
&& pip3 install jinja2 \
&& pip3 install scapy==2.4.5 \
&& pip3 install thrift
{% if docker_ptf_whls.strip() -%}
# Copy locally-built Python wheel dependencies
{{ copy_files("python-wheels/", docker_ptf_whls.split(' '), "/python-wheels/") }}
# Install locally-built Python wheel dependencies
{{ install_python_wheels(docker_ptf_whls.split(' ')) }}
{% endif %}
# Deactivating a virtualenv.
ENV PATH="$BACKUP_OF_PATH"
## Adjust sshd settings
RUN mkdir /var/run/sshd \
&& echo 'root:root' | chpasswd \
&& sed -ri '/^#?PermitRootLogin/c\PermitRootLogin yes' /etc/ssh/sshd_config \
&& sed -ri '/^#?UsePAM/c\UsePAM no' /etc/ssh/sshd_config \
&& sed -ri '/^#?UseDNS/c\UseDNS no' /etc/ssh/sshd_config
COPY supervisord.conf /etc/supervisor/
COPY conf.d/ /etc/supervisor/conf.d/
COPY ptf_tgen.sh /ptf_tgen/
# Move tcpdump into /usr/bin Otherwise it's impossible to run tcpdump due to a docker bug
RUN mv /usr/sbin/tcpdump /usr/bin/tcpdump
RUN ln -s /usr/bin/tcpdump /usr/sbin/tcpdump
RUN mkdir -p /var/log/supervisor
# Install Python-based GNMI client
RUN git clone https://github.com/lguohan/gnxi.git \
&& cd gnxi \
&& git checkout f2b11e4 \
&& cd gnmi_cli_py \
&& pip install -r requirements.txt
COPY \
{% for deb in docker_ptf_debs.split(' ') -%}
debs/{{ deb }}{{' '}}
{%- endfor -%}
debs/
RUN dpkg -i \
{% for deb in docker_ptf_debs.split(' ') -%}
debs/{{ deb }}{{' '}}
{%- endfor %}
COPY ["*.ini", "/etc/ptf/"]
EXPOSE 22 8009
ENTRYPOINT ["/usr/local/bin/supervisord", "-c", "/etc/supervisor/supervisord.conf"]