sonic-buildimage/dockers/docker-database/docker-database-init.sh

123 lines
4.6 KiB
Bash
Raw Normal View History

create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json (#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at Azure/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue 
Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2019-08-28 13:15:10 -05:00
#!/usr/bin/env bash
# For linux host namespace, in both single and multi ASIC platform use the loopback interface
# For other namespaces, use eth0 interface which is connected to the docker0 bridge in the host.
if [[ $NAMESPACE_ID == "" ]]
then
INTFC=lo
else
INTFC=eth0
fi
# Get the ip address of the interface
# if the ip address was not retrieved correctly, put localhost(127.0.0.1) as the default.
host_ip=$(ip -4 -o addr show $INTFC | awk '{print $4}' | cut -d'/' -f1 | head -1)
if [[ $host_ip == "" ]]
then
host_ip=127.0.0.1
fi
redis_port=6379
if [[ $DATABASE_TYPE == "dpudb" ]]; then
host_ip="169.254.200.254"
if ! ip -4 -o addr | awk '{print $4}' | grep $host_ip; then
host_ip=127.0.0.1
fi
DPU_ID=`echo $DEV | tr -dc '0-9'`
redis_port=`expr 6381 + $DPU_ID`
fi
REDIS_DIR=/var/run/redis$NAMESPACE_ID
mkdir -p $REDIS_DIR/sonic-db
mkdir -p /etc/supervisor/conf.d/
if [ -f /etc/sonic/database_config$NAMESPACE_ID.json ]; then
cp /etc/sonic/database_config$NAMESPACE_ID.json $REDIS_DIR/sonic-db/database_config.json
create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json (#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at Azure/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue 
Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2019-08-28 13:15:10 -05:00
else
HOST_IP=$host_ip REDIS_PORT=$redis_port DATABASE_TYPE=$DATABASE_TYPE j2 /usr/share/sonic/templates/database_config.json.j2 > $REDIS_DIR/sonic-db/database_config.json
create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json (#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at Azure/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue 
Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2019-08-28 13:15:10 -05:00
fi
# on VoQ system, we only publish redis_chassis instance and CHASSIS_APP_DB when
# either chassisdb.conf indicates starts chassis_db or connect to chassis_db,
# and redis_chassis instance is started in different container.
# in order to do that, first we save original database config file, then
# call update_chasissdb_config to remove chassis_db config from
# the original database config file and use the modified config file to generate
# supervisord config, so that we won't start redis_chassis service.
# then we will decide to publish modified or original database config file based
# on the setting in chassisdb.conf
start_chassis_db=0
chassis_db_address=""
chassis_db_port=""
chassisdb_config="/usr/share/sonic/platform/chassisdb.conf"
[ -f $chassisdb_config ] && source $chassisdb_config
db_cfg_file="/var/run/redis/sonic-db/database_config.json"
db_cfg_file_tmp="/var/run/redis/sonic-db/database_config.json.tmp"
cp $db_cfg_file $db_cfg_file_tmp
if [[ $DATABASE_TYPE == "chassisdb" ]]; then
# Docker init for database-chassis
echo "Init docker-database-chassis..."
[Supervisor][datbase-chassis] Fixthe failure of starting redis_chassis in database-chassis on Supervisor (#18242) Why I did it redis_chassis process fails to start in the database-chassis on the supervisor card. How I did it The redis and redis_chassis processes in supervisor.conf has been modified to start with 'redis' as user. The modification only added redis and redis group for database containers except database-chassis container. This commit adds code to the docker-database-init.sh to add redis and redis group for the database-chassis to address this issue. How to verify it Booting the image on Supervisor. And verify the database-chassis container should be in UP state, and the database-chassis.service is in active(running) state admin@ixre-cpm-chassis15:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9691c6111e0f docker-fpm-frr:latest "/usr/bin/docker_ini?" 16 hours ago Up 16 hours bgp5 ... c7c7159d4e0c docker-database:latest "/usr/local/bin/dock?" 31 hours ago Up 16 hours database-chassis admin@ixre-cpm-chassis15:~$ sudo systemctl status database-chassis ? database-chassis.service - database-chassis container Loaded: loaded (/lib/systemd/system/database-chassis.service; enabled-runtime; preset: enabled) Active: active (running) since Sun 2024-03-03 04:43:12 UTC; 16h ago Process: 1640 ExecStartPre=/usr/bin/database.sh start chassisdb (code=exited, status=0/SUCCESS) Main PID: 1938 (database.sh) Tasks: 10 (limit: 38307) Memory: 49.7M CGroup: /system.slice/database-chassis.service ??1938 /bin/bash /usr/bin/database.sh wait chassisdb ??1940 docker wait database-chassis Notice: journal has been rotated since unit was started, output may be incomplete. admin@ixre-cpm-chassis15:~$ Signed-off-by: mlok <marty.lok@nokia.com>
2024-03-22 16:20:12 -05:00
VAR_LIB_REDIS_CHASSIS_DIR="/var/lib/redis_chassis"
mkdir -p $VAR_LIB_REDIS_CHASSIS_DIR
update_chassisdb_config -j $db_cfg_file_tmp -k -p $chassis_db_port
# generate all redis server supervisord configuration file
sonic-cfggen -j $db_cfg_file_tmp \
-t /usr/share/sonic/templates/supervisord.conf.j2,/etc/supervisor/conf.d/supervisord.conf \
-t /usr/share/sonic/templates/critical_processes.j2,/etc/supervisor/critical_processes
rm $db_cfg_file_tmp
[Supervisor][datbase-chassis] Fixthe failure of starting redis_chassis in database-chassis on Supervisor (#18242) Why I did it redis_chassis process fails to start in the database-chassis on the supervisor card. How I did it The redis and redis_chassis processes in supervisor.conf has been modified to start with 'redis' as user. The modification only added redis and redis group for database containers except database-chassis container. This commit adds code to the docker-database-init.sh to add redis and redis group for the database-chassis to address this issue. How to verify it Booting the image on Supervisor. And verify the database-chassis container should be in UP state, and the database-chassis.service is in active(running) state admin@ixre-cpm-chassis15:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9691c6111e0f docker-fpm-frr:latest "/usr/bin/docker_ini?" 16 hours ago Up 16 hours bgp5 ... c7c7159d4e0c docker-database:latest "/usr/local/bin/dock?" 31 hours ago Up 16 hours database-chassis admin@ixre-cpm-chassis15:~$ sudo systemctl status database-chassis ? database-chassis.service - database-chassis container Loaded: loaded (/lib/systemd/system/database-chassis.service; enabled-runtime; preset: enabled) Active: active (running) since Sun 2024-03-03 04:43:12 UTC; 16h ago Process: 1640 ExecStartPre=/usr/bin/database.sh start chassisdb (code=exited, status=0/SUCCESS) Main PID: 1938 (database.sh) Tasks: 10 (limit: 38307) Memory: 49.7M CGroup: /system.slice/database-chassis.service ??1938 /bin/bash /usr/bin/database.sh wait chassisdb ??1940 docker wait database-chassis Notice: journal has been rotated since unit was started, output may be incomplete. admin@ixre-cpm-chassis15:~$ Signed-off-by: mlok <marty.lok@nokia.com>
2024-03-22 16:20:12 -05:00
chown -R redis:redis $VAR_LIB_REDIS_CHASSIS_DIR
chown -R redis:redis $REDIS_DIR
exec /usr/local/bin/supervisord
exit 0
fi
# copy/generate the database_global.json file if this is global database service in multi asic/smart switch platform.
if [[ $NAMESPACE_ID == "" && $DATABASE_TYPE == "" && ( $NAMESPACE_COUNT -gt 1 || $NUM_DPU -gt 1) ]]
then
if [ -f /etc/sonic/database_global.json ]; then
cp /etc/sonic/database_global.json $REDIS_DIR/sonic-db/database_global.json
else
j2 /usr/share/sonic/templates/database_global.json.j2 > $REDIS_DIR/sonic-db/database_global.json
fi
fi
# delete chassisdb config to generate supervisord config
update_chassisdb_config -j $db_cfg_file_tmp -d
sonic-cfggen -j $db_cfg_file_tmp \
-t /usr/share/sonic/templates/supervisord.conf.j2,/etc/supervisor/conf.d/supervisord.conf \
-t /usr/share/sonic/templates/critical_processes.j2,/etc/supervisor/critical_processes
if [[ "$start_chassis_db" != "1" ]] && [[ -z "$chassis_db_address" ]]; then
cp $db_cfg_file_tmp $db_cfg_file
else
update_chassisdb_config -j $db_cfg_file -p $chassis_db_port
fi
rm $db_cfg_file_tmp
create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json (#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at Azure/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue 
Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2019-08-28 13:15:10 -05:00
# copy dump.rdb file to each instance for restoration
DUMPFILE=/var/lib/redis/dump.rdb
Fix docker database flush_unused_database failed issue (#11600) #### Why I did it Fix docker-database flush_unused_database failed issue: https://github.com/Azure/sonic-buildimage/issues/11597 When change flush_unused_database from use swsssdk to use swsscommon, get_instancelist() and get_dblist() name changed but not update. #### How I did it Change flush_unused_database code to use swsscommon API: Change get_instancelist to getInstanceList. Change get_dblist to getDbList. #### How to verify it Pass all E2E test. Manually check syslog make sure error log not exist and swss, syncd, bgp service started. Search code in Azure make sure there all similer case are fixed in this PR. #### Which release branch to backport (provide reason below if selected) <!-- - Note we only backport fixes to a release branch, *not* features! - Please also provide a reason for the backporting below. - e.g. - [x] 202006 --> - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 #### Description for the changelog Fix docker-database flush_unused_database failed issue: https://github.com/Azure/sonic-buildimage/issues/11597 When change flush_unused_database from use swsssdk to use swsscommon, get_instancelist() and get_dblist() name changed but not update. #### Link to config_db schema for YANG module changes <!-- Provide a link to config_db schema for the table for which YANG model is defined Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md --> #### A picture of a cute animal (not mandatory but encouraged) Co-authored-by: liuh-80 <azureuser@liuh-dev-vm-02.5fg3zjdzj2xezlx1yazx5oxkzd.hx.internal.cloudapp.net>
2022-08-02 21:18:00 -05:00
redis_inst_list=`/usr/bin/python3 -c "from swsscommon import swsscommon; print(' '.join(swsscommon.SonicDBConfig.getInstanceList().keys()))"`
for inst in $redis_inst_list
do
mkdir -p /var/lib/$inst
if [[ -f $DUMPFILE ]]; then
# copy warmboot rdb file into each new instance location
if [[ "$DUMPFILE" != "/var/lib/$inst/dump.rdb" ]]; then
cp $DUMPFILE /var/lib/$inst/dump.rdb
fi
else
echo -n > /var/lib/$inst/dump.rdb
fi
done
TZ=$(cat /etc/timezone)
rm -rf /etc/localtime
ln -sf /usr/share/zoneinfo/$TZ /etc/localtime
chown -R redis:redis $REDIS_DIR
exec /usr/local/bin/supervisord