SONiC Management Framework Release 1.0 (#3488)

* Added sonic-mgmt-framework as submodule / docker

* fix build issues

* update sonic-mgmt-framework submodule branch to master

* Merged changes 70007e6d2ba3a4c0b371cd693ccc63e0a8906e77..00d4fcfed6a759e40d7b92120ea0ee1f08300fc6

00d4fcfed6a759e40d7b92120ea0ee1f08300fc6 Modified environemnt variables

* Changes to build sonic-mgmt-framework docker

* bumped up sonic-mgmt-framework commit-id

* version bump for sonic-mgmt-framework commit-it

* bumped up sonic-mgmt-framework commit-id

* Add python packages to docker

* Build fix for docker with python packages

* added libyang as dependent package

* Allow building images on NFS-mounted clones

Prior to this change, `build_debian.sh` would generate a Debian
filesystem in `./fsroot`. This needs root permissions, and one of the
tests that is performed is whether the user can create a character
special file in the filesystem (using mknod).

On most NFS deployments, `root` is the least privileged user, and cannot
run mknod. Also, attempting to run commands like rm or mv as root would
fail due to permission errors, since the root user gets mapped to an
unprivileged user like `nobody`.

This commit changes the location of the Debian filesystem to `/fsroot`,
which is a tmpfs mount within the slave Docker. The default squashfs,
docker tarball and zip files are also created within /tmp, before being
copied back to /sonic as the regular user.

The side effect of this change is that the contents of `/fsroot` are no
longer available once the slave container exits, however they are
available within the squashfs image.

Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com>

* bumped up sonc-mgmt-framework commit to include PR #18

*     REST Server startup script is enahnced to read the settings from
    ConfigDB. Below table provides mapping of db field to command line
    argument name.

    ============================================================
    ConfigDB entry key      Field name      REST Server argument
    ============================================================
    REST_SERVER|default     port            -port
    REST_SERVER|default     client_auth     -client_auth
    REST_SERVER|default     log_level       -v
    DEVICE_METADATA|x509    server_crt      -cert
    DEVICE_METADATA|x509    server_key      -key
    DEVICE_METADATA|x509    ca_crt          -cacert
    ============================================================

* Replace src/telemetry as submodule to sonic-telemetry

* Update telemetry commit HEAD

* Update sonic-telemetry commit HEAD

* libyang env path update

* Add libyang dependency to telemetry

* Add scripts to create JSON files for CLI backend

Scripts to create /var/platform/syseeprom and /var/platform/system, which are back-end
files for CLI, for system EEPROM and system information.

Signed-off-by: Howard Persh <Howard_Persh@dell.com>

* In startup script, create directory where CLI back-end files live

Signed-off-by: Howard Persh <Howard_Persh@dell.com>

* build dependency pkgs added to docker for build failure fix

* Changes to fix build issue for mgmt framework

* Fix exec path issue with telemetry

* s5232[device] PSU detecttion and default led state support

* Processing of first boot in rc.local should not have premature exit

Signed-off-by: Howard Persh <Howard_Persh@dell.com>

*  docker mount options added for platform, system features

* bumped up sonic-mgmt-framework commit id to pick 23rd July 2019 changes

* Added mount options for telemetry docker to get access for system and platform info.

* Update commit for sonic-utilities

* [dell]: Corrected dport map and renamed config files for S5232F

* Fix telemetry submodule commit

* added support for sonic-cli console

* [Dell S5232F, Z9264F] Harden FPGA driver kernel module

For Dell S5232F and Z9264F platforms, be more strict when checking state
in ISR of FPGA driver, to harden against spurious interrupts.

Signed-off-by: Howard Persh <Howard_Persh@dell.com>

* update mgmt-framework submodule to 27th Aug commit.

* remove changes not related to mgmt-framework and sonic-telemetry

* Revert "Replace src/telemetry as submodule to sonic-telemetry"

This reverts commit 11c3192975.

* Revert "Replace src/telemetry as submodule to sonic-telemetry"

This reverts commit 11c3192975.

* make submodule changes and remove a change not related to PR

* more changes

* Update .gitmodules

* Update Dockerfile.j2

* Update .gitmodules

* Update .gitmodules

* Update .gitmodules

reverting experimental change

* Removed syspoll for release_1.0

Signed-off-by: Jeff Yin <29264773+jeff-yin@users.noreply.github.com>

* Update docker-sonic-mgmt-framework.mk

* Update sonic-mgmt-framework.mk

* Update sonic-mgmt-framework.mk

* Update docker-sonic-mgmt-framework.mk

* Update docker-sonic-mgmt-framework.mk

* Revert "Processing of first boot in rc.local should not have premature exit"

This reverts commit e99a91ffc2.

* Remove old telemetry directory

* Update docker-sonic-mgmt-framework.mk

* Resolving merge conflict with Azure

* Reverting the wrong merge

* Use CVL_SCHEMA_PATH instead of changing directory for telemetry startup

* Add missing export

* Add python mmh3 to slave dockerfile

* Remove sonic-mgmt-framework build dep for telemetry, fix dialout startup issues

* Provided flag to disable compiling mgmt-framework

* Update sonic-utilites point latest commit id

* Point sonic-utilities to Azure accepted SHA

* Updating mgmt framework to right sha

* Add sonic-telemetry submodule

* Update the mgmt-framework commit id

Co-authored-by: jghalam <joe.ghalam@gmail.com>
Co-authored-by: Partha Dutta <51353699+dutta-partha@users.noreply.github.com>
Co-authored-by: srideepDell <srideep_devireddy@dell.com>
Co-authored-by: nirenjan <nirenjan@users.noreply.github.com>
Co-authored-by: Sachin Holla <51310506+sachinholla@users.noreply.github.com>
Co-authored-by: Eric Seifert <seiferteric@gmail.com>
Co-authored-by: Howard Persh <hpersh@yahoo.com>
Co-authored-by: Jeff Yin <29264773+jeff-yin@users.noreply.github.com>
Co-authored-by: Arunsundar Kannan <31632515+arunsundark@users.noreply.github.com>
Co-authored-by: rvasanthm <51932293+rvasanthm@users.noreply.github.com>
Co-authored-by: Ashok Daparthi-Dell <Ashok_Daparthi@Dell.com>
Co-authored-by: anand-kumar-subramanian <51383315+anand-kumar-subramanian@users.noreply.github.com>
This commit is contained in:
Prabhu Sreenivasan 2019-12-24 11:17:16 +05:30 committed by Abhishek
parent 6045e34650
commit 7ec2732387
25 changed files with 225 additions and 64 deletions

6
.gitmodules vendored
View File

@ -66,6 +66,12 @@
[submodule "platform/mellanox/mlnx-sai/SAI-Implementation"]
path = platform/mellanox/mlnx-sai/SAI-Implementation
url = https://github.com/Mellanox/SAI-Implementation
[submodule "src/sonic-mgmt-framework"]
path = src/sonic-mgmt-framework
url = https://github.com/Azure/sonic-mgmt-framework
[submodule "src/sonic-telemetry"]
path = src/sonic-telemetry
url = https://github.com/Azure/sonic-telemetry
[submodule "Switch-SDK-drivers"]
path = platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers
url = https://github.com/Mellanox/Switch-SDK-drivers

View File

@ -0,0 +1,42 @@
FROM docker-config-engine-stretch
ARG docker_container_name
RUN [ -f /etc/rsyslog.conf ] && sed -ri "s/%syslogtag%/$docker_container_name#%syslogtag%/;" /etc/rsyslog.conf
## Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN pip install connexion==1.1.15 \
setuptools==21.0.0 \
grpcio-tools==1.20.0 \
pyangbind==0.6.0 \
certifi==2017.4.17 \
python-dateutil==2.6.0 \
six==1.11.0 \
urllib3==1.21.1
## Install redis-tools dependencies
## TODO: implicitly install dependencies
RUN apt-get -y install libjemalloc1 libatomic1 liblua5.1-0 lua-bitop lua-cjson
COPY \
{% for deb in docker_sonic_mgmt_framework_debs.split(' ') -%}
debs/{{ deb }}{{' '}}
{%- endfor -%}
debs/
RUN dpkg -i \
{% for deb in docker_sonic_mgmt_framework_debs.split(' ') -%}
debs/{{ deb }}{{' '}}
{%- endfor %}
COPY ["start.sh", "rest-server.sh", "/usr/bin/"]
COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]
RUN apt-get clean -y; apt-get autoclean -y; apt-get autoremove -y
RUN rm -rf /debs
ENTRYPOINT ["/usr/bin/supervisord"]

View File

@ -0,0 +1,4 @@
#!/bin/bash
docker exec -it mgmt-framework /usr/sbin/cli/clish_start "$@"

View File

@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Startup script for SONiC Management REST Server
SERVER_PORT=
LOG_LEVEL=
CLIENT_AUTH=
SERVER_CRT=
SERVER_KEY=
CA_CERT=
# Read basic server settings from REST_SERVER|default entry
HAS_REST_CONFIG=$(sonic-cfggen -d -v "1 if REST_SERVER and REST_SERVER['default']")
if [ "$HAS_REST_CONFIG" == "1" ]; then
SERVER_PORT=$(sonic-cfggen -d -v "REST_SERVER['default']['port']")
CLIENT_AUTH=$(sonic-cfggen -d -v "REST_SERVER['default']['client_auth']")
LOG_LEVEL=$(sonic-cfggen -d -v "REST_SERVER['default']['log_level']")
fi
# Read certificate file paths from DEVICE_METADATA|x509 entry.
HAS_X509_CONFIG=$(sonic-cfggen -d -v "1 if DEVICE_METADATA and DEVICE_METADATA['x509']")
if [ "$HAS_X509_CONFIG" == "1" ]; then
SERVER_CRT=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_crt']")
SERVER_KEY=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_key']")
CA_CRT=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['ca_crt']")
fi
# Create temporary server certificate if they not configured in ConfigDB
if [ -z $SERVER_CRT ] && [ -z $SERVER_KEY ]; then
echo "Generating temporary TLS server certificate ..."
(cd /tmp && /usr/sbin/generate_cert --host="localhost,127.0.0.1")
SERVER_CRT=/tmp/cert.pem
SERVER_KEY=/tmp/key.pem
fi
REST_SERVER_ARGS="-ui /rest_ui -logtostderr"
[ ! -z $SERVER_PORT ] && REST_SERVER_ARGS+=" -port $SERVER_PORT"
[ ! -z $LOG_LEVEL ] && REST_SERVER_ARGS+=" -v $LOG_LEVEL"
[ ! -z $CLIENT_AUTH ] && REST_SERVER_ARGS+=" -client_auth $CLIENT_AUTH"
[ ! -z $SERVER_CRT ] && REST_SERVER_ARGS+=" -cert $SERVER_CRT"
[ ! -z $SERVER_KEY ] && REST_SERVER_ARGS+=" -key $SERVER_KEY"
[ ! -z $CA_CRT ] && REST_SERVER_ARGS+=" -cacert $CA_CRT"
echo "REST_SERVER_ARGS = $REST_SERVER_ARGS"
export CVL_SCHEMA_PATH=/usr/sbin/schema
export LIBYANG_EXTENSIONS_PLUGINS_DIR=/usr/lib/x86_64-linux-gnu/libyang/extensions
export LIBYANG_USER_TYPES_PLUGINS_DIR=/usr/lib/x86_64-linux-gnu/libyang/user_types
exec /usr/sbin/rest_server ${REST_SERVER_ARGS}

View File

@ -0,0 +1,10 @@
#!/usr/bin/env bash
mkdir -p /var/sonic
echo "# Config files managed by sonic-config-engine" > /var/sonic/config_status
rm -f /var/run/rsyslogd.pid
supervisorctl start rsyslogd
supervisorctl start rest-server

View File

@ -0,0 +1,28 @@
[supervisord]
logfile_maxbytes=1MB
logfile_backups=2
nodaemon=true
[program:start.sh]
command=/usr/bin/start.sh
priority=1
autostart=true
autorestart=false
stdout_logfile=syslog
stderr_logfile=syslog
[program:rsyslogd]
command=/usr/sbin/rsyslogd -n
priority=2
autostart=false
autorestart=false
stdout_logfile=syslog
stderr_logfile=syslog
[program:rest-server]
command=/usr/bin/rest-server.sh
priority=3
autostart=false
autorestart=true
stdout_logfile=syslog
stderr_logfile=syslog

View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Start with default config
export CVL_SCHEMA_PATH=/usr/sbin/schema
exec /usr/sbin/dialout_client_cli -insecure -logtostderr -v 2

View File

@ -6,8 +6,8 @@ nodaemon=true
[eventlistener:supervisor-proc-exit-listener]
command=/usr/bin/supervisor-proc-exit-listener
events=PROCESS_STATE_EXITED
autostart=always
autorestart=unexpected
autostart=true
autorestart=false
[program:start.sh]
command=/usr/bin/start.sh

View File

@ -6,6 +6,7 @@ X509=`sonic-cfggen -d -v "DEVICE_METADATA['x509']"`
TELEMETRY=`sonic-cfggen -d -v 'TELEMETRY.keys() | join(" ") if TELEMETRY'`
TELEMETRY_ARGS=" -logtostderr"
export CVL_SCHEMA_PATH=/usr/sbin/schema
if [ -n "$X509" ]; then
SERVER_CRT=`sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_crt']"`

View File

@ -0,0 +1,14 @@
[Unit]
Description=Management Framework container
Requires=swss.service
After=swss.service
Before=ntp-config.service
[Service]
User={{ sonicadmin_user }}
ExecStartPre=/usr/bin/{{docker_container_name}}.sh start
ExecStart=/usr/bin/{{docker_container_name}}.sh wait
ExecStop=/usr/bin/{{docker_container_name}}.sh stop
[Install]
WantedBy=multi-user.target

View File

@ -331,4 +331,7 @@ if [ -f $FIRST_BOOT_FILE ]; then
firsttime_exit
fi
# Create dir where following scripts put their output files
mkdir -p /var/platform
exit 0

View File

@ -98,3 +98,6 @@ DEFAULT_VS_PREPARE_MEM = yes
# ENABLE_SYSTEM_SFLOW - build docker-sonic-sflow for sFlow support
ENABLE_SFLOW = y
# ENABLE_MGMT_FRAMEWORK - build docker-sonic-mgt-framework for CLI and REST server support
ENABLE_MGMT_FRAMEWORK = y

View File

@ -0,0 +1,34 @@
# docker image for mgmt-framework
DOCKER_MGMT_FRAMEWORK_STEM = docker-sonic-mgmt-framework
DOCKER_MGMT_FRAMEWORK = $(DOCKER_MGMT_FRAMEWORK_STEM).gz
DOCKER_MGMT_FRAMEWORK_DBG = $(DOCKER_MGMT_FRAMEWORK_STEM)-$(DBG_IMAGE_MARK).gz
$(DOCKER_MGMT_FRAMEWORK)_PATH = $(DOCKERS_PATH)/$(DOCKER_MGMT_FRAMEWORK_STEM)
$(DOCKER_MGMT_FRAMEWORK)_DEPENDS += $(REDIS_TOOLS) $(SONIC_MGMT_FRAMEWORK)
$(DOCKER_MGMT_FRAMEWORK)_DBG_DEPENDS = $($(DOCKER_CONFIG_ENGINE_STRETCH)_DBG_DEPENDS)
$(DOCKER_MGMT_FRAMEWORK)_DBG_DEPENDS += $(REDIS_TOOLS) $(SONIC_MGMT_FRAMEWORK_DBG)
SONIC_DOCKER_IMAGES += $(DOCKER_MGMT_FRAMEWORK)
$(DOCKER_MGMT_FRAMEWORK)_LOAD_DOCKERS += $(DOCKER_CONFIG_ENGINE_STRETCH)
$(DOCKER_MGMT_FRAMEWORK)_DBG_IMAGE_PACKAGES = $($(DOCKER_CONFIG_ENGINE_STRETCH)_DBG_IMAGE_PACKAGES)
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
SONIC_INSTALL_DOCKER_IMAGES += $(DOCKER_MGMT_FRAMEWORK)
SONIC_STRETCH_DOCKERS += $(DOCKER_MGMT_FRAMEWORK)
endif
SONIC_DOCKER_DBG_IMAGES += $(DOCKER_MGMT_FRAMEWORK_DBG)
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
SONIC_INSTALL_DOCKER_DBG_IMAGES += $(DOCKER_MGMT_FRAMEWORK_DBG)
SONIC_STRETCH_DBG_DOCKERS += $(DOCKER_MGMT_FRAMEWORK_DBG)
endif
$(DOCKER_MGMT_FRAMEWORK)_CONTAINER_NAME = mgmt-framework
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += --net=host --privileged -t
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += -v /etc/sonic:/etc/sonic:ro
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += -v /etc:/host_etc:ro
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += --mount type=bind,source="/var/platform/",target="/mnt/platform/"
$(DOCKER_MGMT_FRAMEWORK)_BASE_IMAGE_FILES += sonic-cli:/usr/bin/sonic-cli

View File

@ -27,5 +27,6 @@ endif
$(DOCKER_TELEMETRY)_CONTAINER_NAME = telemetry
$(DOCKER_TELEMETRY)_RUN_OPT += --net=host --privileged -t
$(DOCKER_TELEMETRY)_RUN_OPT += -v /etc/sonic:/etc/sonic:ro
$(DOCKER_TELEMETRY)_RUN_OPT += --mount type=bind,source="/var/platform/",target="/mnt/platform/"
$(DOCKER_TELEMETRY)_FILES += $(SUPERVISOR_PROC_EXIT_LISTENER_SCRIPT)

View File

@ -0,0 +1,16 @@
# SONiC mgmt-framework package
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
SONIC_MGMT_FRAMEWORK = sonic-mgmt-framework_1.0-01_amd64.deb
$(SONIC_MGMT_FRAMEWORK)_SRC_PATH = $(SRC_PATH)/sonic-mgmt-framework
$(SONIC_MGMT_FRAMEWORK)_DEPENDS = $(LIBYANG_DEV) $(LIBYANG)
$(SONIC_MGMT_FRAMEWORK)_RDEPENDS = $(LIBYANG)
SONIC_DPKG_DEBS += $(SONIC_MGMT_FRAMEWORK)
SONIC_MGMT_FRAMEWORK_DBG = sonic-mgmt-framework-dbg_1.0-01_amd64.deb
$(SONIC_MGMT_FRAMEWORK_DBG)_DEPENDS += $(SONIC_MGMT_FRAMEWORK)
$(SONIC_MGMT_FRAMEWORK_DBG)_RDEPENDS += $(SONIC_MGMT_FRAMEWORK)
$(eval $(call add_derived_package,$(SONIC_MGMT_FRAMEWORK),$(SONIC_MGMT_FRAMEWORK_DBG)))
endif

View File

@ -1,5 +1,7 @@
# SONiC telemetry package
SONIC_TELEMETRY = sonic-telemetry_0.1_$(CONFIGURED_ARCH).deb
$(SONIC_TELEMETRY)_SRC_PATH = $(SRC_PATH)/telemetry
$(SONIC_TELEMETRY)_SRC_PATH = $(SRC_PATH)/sonic-telemetry
$(SONIC_TELEMETRY)_DEPENDS = $(LIBYANG_DEV) $(LIBYANG)
$(SONIC_TELEMETRY)_RDEPENDS = $(LIBYANG)
SONIC_DPKG_DEBS += $(SONIC_TELEMETRY)

View File

@ -354,6 +354,9 @@ RUN pip install mockredispy==2.9.3
RUN pip install pytest-runner==4.4
RUN pip install setuptools==40.8.0
# For mgmt-framework build
RUN pip install mmh3
# Install dependencies for isc-dhcp-relay build
RUN apt-get -y build-dep isc-dhcp

@ -0,0 +1 @@
Subproject commit 8b199a9f822ca42564a7a89da0cab3133684bd12

1
src/sonic-telemetry Submodule

@ -0,0 +1 @@
Subproject commit aaa9188fda4b6f36dd5da00cdb240933fedae5ce

View File

@ -1,20 +0,0 @@
export GOPATH=/tmp/go
INSTALL := /usr/bin/install
all: sonic-telemetry
sonic-telemetry:
/usr/local/go/bin/go get -v github.com/Azure/sonic-telemetry/telemetry
/usr/local/go/bin/go get -v github.com/Azure/sonic-telemetry/dialout/dialout_client_cli
install:
$(INSTALL) -D ${GOPATH}/bin/telemetry $(DESTDIR)/usr/sbin/telemetry
$(INSTALL) -D ${GOPATH}/bin/dialout_client_cli $(DESTDIR)/usr/sbin/dialout_client_cli
deinstall:
rm $(DESTDIR)/usr/sbin/telemetry
rm $(DESTDIR)/usr/sbin/dialout_client_cli
clean:
rm -fr ${GOPATH}

View File

@ -1,5 +0,0 @@
sonic-telemetry (0.1) UNRELEASED; urgency=medium
* Initial release.
-- Jipan Yang <jipan.yang@alibaba-inc.com> Sat, 24 Mar 2018 12:48:22 -0700

View File

@ -1 +0,0 @@
9

View File

@ -1,17 +0,0 @@
Source: sonic-telemetry
Section: devel
Priority: optional
Maintainer: Jipan Yang <jipan.yang@alibaba-inc.com>
Build-Depends: debhelper (>= 8.0.0),
dh-systemd
Standards-Version: 3.9.3
Homepage: https://github.com/Azure/sonic-telemetry
XS-Go-Import-Path: github.com/Azure/sonic-telemetry
Package: sonic-telemetry
Architecture: any
Built-Using: ${misc:Built-Using}
Depends: ${misc:Depends},
${shlibs:Depends}
Description: SONiC telemetry
sonic-telemetry

View File

@ -1,3 +0,0 @@
#!/usr/bin/make -f
%:
dh $@ --with systemd

View File

@ -1,14 +0,0 @@
#!/bin/bash
#
### BEGIN INIT INFO
# Provides: sonic-telemetry
# Required-Start: $local_fs $network $remote_fs $syslog
# Required-Stop: $local_fs $network $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop the telemetry
# Description: sonic-telemetry is an implementation of sonic telemetry daemon in Go
### END INIT INFO
#
exit 0