8e27bb9bae
# Changes A big shoutout to @luhahn for all his work in #205 which served as the base for this PR. ## Documentation - [x] After thinking for some time about it, I still prefer the distinct option (as started in #350), i.e. having a standalone "HA" doc under `docs/ha-setup.md` to not have a very long README (which is already quite long). Most of the information below should go into it with more details and explanations behind all of the individual components. ## Chart deps ~~- Adds `meilisearch` as a chart dependency for a HA-ready issue indexer. Only works with >= Gitea 1.20~~ ~~- Adds `redis` as a chart dependency for a HA-ready session and queue store.~~ - Adds `redis-cluster` as a chart dependency for a HA-ready session and queue store (alternative to `redis`). Only works with >= Gitea 1.19.2. - Removes `memcached` instead of `redis-cluster` - Add `postgresql-ha` as default DB dep in favor of `postgres` ## Adds smart HA chart logic The goal is to set smart config values that result in a HA-ready Gitea deployment if `replicaCount` > 1. - If `replicaCount` > 1, - `gitea.config.session.PROVIDER` is automatically set to `redis-cluster` - `gitea.config.indexer.REPO_INDEXER_ENABLED` is automatically set to `false` unless the value is `elasticsearch` or `meilisearch` - `redis-cluster` is used for `[queue]` and `[cache]` and `[session]`mode or not Configuration of external instances of `meilisearch` and `minio` are documented in a new markdown doc. ## Deployment vs Statefulset Given all the discussions about this lately (#428), I think we could use both. In the end, we do not have the requirement for a sequential pod scale up/scale down as it would happen in statefulsets. On the other side, we do not have actual stateless pods as we are attaching a RWX to the deployment. Yet I think because we do not have a leader-election requirement, spawning the pods as a deployment makes "Rolling Updates" easier and also signals users that there is no "leader election" logic and each pod can just be "destroyed" at anytime without causing interruption. Hence I think we should be able to switch from a statefulset to a deployment, even in the single-replica case. This change also brought up a templating/linting issue: the definition of `.Values.gitea.config.server.SSH_LISTEN_PORT` in `ssh-svc.yaml` just "luckily" worked so far due to naming-related lint processing. Due to the change from "statefulset" to "deployment", the processing queue changed and caused a failure complaining about `config.server.SSH_LISTEN_PORT` not being defined yet. The only way I could see to fix this was to "properly" define the value in `values.yaml` instead of conditionally definining it in `helpers.tpl`. Maybe there's a better way? ## Chart PVC Creation I've adapted the automated PVC creation from another chart to be able to provide the `storageClassName` as I couldn't get dynamic provisioning for EFS going with the current implementation. In addition the naming and approach within the Gitea chart for PV creation is a bit unusual and aligning it might be beneficial. A semi-unrelated change which will result in a breaking change for existing users but this PR includes a lot of breaking changes already, so including another one might not make it much worse... - New `persistence.mount`: whether to mount an existing PVC (via `persistence.existingClaim` - New `persistence.create`: whether to create a new PVC ## Testing As this PR does a lot of things, we need proper testing. The helm chart can be installed from the Git branch via `helm-git` as follows: ``` helm repo add gitea-charts git+https://gitea.com/gitea/helm-chart@/?ref=deployment helm install gitea --version 0.0.0 ``` It is **highly recommended** to test the chart in a dedicated namespace. I've tested this myself with both `redis` and `redis-cluster` and it seemed to work fine. I just did some basic operations though and we should do more niche testing before merging. Examplary `values.yml` for testing (only needs a valid RWX storage class): <details> <summary>values.yaml</summary> ```yml image: tag: "dev" PullPolicy: "Always" rootless: true replicaCount: 2 persistence: enabled: true accessModes: - ReadWriteMany storageClass: FIXME redis-cluster: enabled: false global: redis: password: gitea gitea: config: indexer: ISSUE_INDEXER_ENABLED: true REPO_INDEXER_ENABLED: false ``` </details> ## Preferred setup The preferred HA setup with respect to performance and stability might currently be as follows: - Repos: RWX (e.g. EFS or Azurefiles NFS) - Issue indexer: Meilisearch (HA) - Session and cache: Redis Cluster (HA) - Attachments/Avatars: Minio (HA) This will result in a ~ 10-pod HA setup overall. All pods have very low resource requests. fix #98 Co-authored-by: pat-s <pat-s@noreply.gitea.io> Reviewed-on: https://gitea.com/gitea/helm-chart/pulls/437 Co-authored-by: pat-s <patrick.schratz@gmail.com> Co-committed-by: pat-s <patrick.schratz@gmail.com>
200 lines
6.8 KiB
YAML
200 lines
6.8 KiB
YAML
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: {{ include "gitea.fullname" . }}-inline-config
|
|
labels:
|
|
{{- include "gitea.labels" . | nindent 4 }}
|
|
type: Opaque
|
|
stringData:
|
|
{{- include "gitea.inline_configuration" . | nindent 2 }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: {{ include "gitea.fullname" . }}
|
|
labels:
|
|
{{- include "gitea.labels" . | nindent 4 }}
|
|
type: Opaque
|
|
stringData:
|
|
assertions: |
|
|
{{- /* multiple replicas assertions */ -}}
|
|
{{- if gt .Values.replicaCount 1.0 -}}
|
|
{{- if .Values.gitea.config.cron.GIT_GC_REPOS -}}
|
|
{{- if .Values.gitea.config.cron.GIT_GC_REPOS.enabled -}}
|
|
{{- fail "Invoking the garbage collector via CRON is not yet supported when running with multiple replicas. Please set 'GIT_GC_REPOS.enabled = false'." -}}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- if eq (first .Values.persistence.accessModes) "ReadWriteOnce" -}}
|
|
{{- fail "When using multiple replicas, a RWX file system is required and gitea.persistence.accessModes[0] must be set to ReadWriteMany." -}}
|
|
{{- end }}
|
|
|
|
{{- if eq (get .Values.gitea.config.indexer "ISSUE_INDEXER_TYPE") "bleve" -}}
|
|
{{- fail "When using multiple replicas, the issue indexer (gitea.config.indexer.ISSUE_INDEXER_TYPE) must be set to a HA-ready provider such as 'meilisearch', 'elasticsearch' or 'db' (if the DB is HA-ready)." -}}
|
|
{{- end }}
|
|
{{- if .Values.gitea.config.indexer.REPO_INDEXER_TYPE -}}
|
|
{{- if eq (get .Values.gitea.config.indexer "REPO_INDEXER_TYPE") "bleve" -}}
|
|
{{- if .Values.gitea.config.indexer.REPO_INDEXER_ENABLED -}}
|
|
{{- if eq (get .Values.gitea.config.indexer "REPO_INDEXER_ENABLED") "true" -}}
|
|
{{- fail "When using multiple replicas, the repo indexer (gitea.config.indexer.REPO_INDEXER_TYPE) must be set to 'meilisearch' or 'elasticsearch' or disabled." -}}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
|
|
{{- end }}
|
|
config_environment.sh: |-
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
function env2ini::log() {
|
|
printf "${1}\n"
|
|
}
|
|
|
|
function env2ini::read_config_to_env() {
|
|
local section="${1}"
|
|
local line="${2}"
|
|
|
|
if [[ -z "${line}" ]]; then
|
|
# skip empty line
|
|
return
|
|
fi
|
|
|
|
# 'xargs echo -n' trims all leading/trailing whitespaces and a trailing new line
|
|
local setting="$(awk -F '=' '{print $1}' <<< "${line}" | xargs echo -n)"
|
|
|
|
if [[ -z "${setting}" ]]; then
|
|
env2ini::log ' ! invalid setting'
|
|
exit 1
|
|
fi
|
|
|
|
local value=''
|
|
local regex="^${setting}(\s*)=(\s*)(.*)"
|
|
if [[ $line =~ $regex ]]; then
|
|
value="${BASH_REMATCH[3]}"
|
|
else
|
|
env2ini::log ' ! invalid setting'
|
|
exit 1
|
|
fi
|
|
|
|
env2ini::log " + '${setting}'"
|
|
|
|
if [[ -z "${section}" ]]; then
|
|
export "GITEA____${setting^^}=${value}" # '^^' makes the variable content uppercase
|
|
return
|
|
fi
|
|
|
|
local masked_section="${section//./_0X2E_}" # '//' instructs to replace all matches
|
|
masked_section="${masked_section//-/_0X2D_}"
|
|
|
|
export "GITEA__${masked_section^^}__${setting^^}=${value}" # '^^' makes the variable content uppercase
|
|
}
|
|
|
|
function env2ini::reload_preset_envs() {
|
|
env2ini::log "Reloading preset envs..."
|
|
|
|
while read -r line; do
|
|
if [[ -z "${line}" ]]; then
|
|
# skip empty line
|
|
return
|
|
fi
|
|
|
|
# 'xargs echo -n' trims all leading/trailing whitespaces and a trailing new line
|
|
local setting="$(awk -F '=' '{print $1}' <<< "${line}" | xargs echo -n)"
|
|
|
|
if [[ -z "${setting}" ]]; then
|
|
env2ini::log ' ! invalid setting'
|
|
exit 1
|
|
fi
|
|
|
|
local value=''
|
|
local regex="^${setting}(\s*)=(\s*)(.*)"
|
|
if [[ $line =~ $regex ]]; then
|
|
value="${BASH_REMATCH[3]}"
|
|
else
|
|
env2ini::log ' ! invalid setting'
|
|
exit 1
|
|
fi
|
|
|
|
env2ini::log " + '${setting}'"
|
|
|
|
export "${setting^^}=${value}" # '^^' makes the variable content uppercase
|
|
done < "/tmp/existing-envs"
|
|
|
|
rm /tmp/existing-envs
|
|
}
|
|
|
|
|
|
function env2ini::process_config_file() {
|
|
local config_file="${1}"
|
|
local section="$(basename "${config_file}")"
|
|
|
|
if [[ $section == '_generals_' ]]; then
|
|
env2ini::log " [ini root]"
|
|
section=''
|
|
else
|
|
env2ini::log " ${section}"
|
|
fi
|
|
|
|
while read -r line; do
|
|
env2ini::read_config_to_env "${section}" "${line}"
|
|
done < <(awk 1 "${config_file}") # Helm .toYaml trims the trailing new line which breaks line processing; awk 1 ... adds it back while reading
|
|
}
|
|
|
|
function env2ini::load_config_sources() {
|
|
local path="${1}"
|
|
|
|
if [[ -d "${path}" ]]; then
|
|
env2ini::log "Processing $(basename "${path}")..."
|
|
|
|
while read -d '' configFile; do
|
|
env2ini::process_config_file "${configFile}"
|
|
done < <(find "${path}" -type l -not -name '..data' -print0)
|
|
|
|
env2ini::log "\n"
|
|
fi
|
|
}
|
|
|
|
function env2ini::generate_initial_secrets() {
|
|
# These environment variables will either be
|
|
# - overwritten with user defined values,
|
|
# - initially used to set up Gitea
|
|
# Anyway, they won't harm existing app.ini files
|
|
|
|
export GITEA__SECURITY__INTERNAL_TOKEN=$(gitea generate secret INTERNAL_TOKEN)
|
|
export GITEA__SECURITY__SECRET_KEY=$(gitea generate secret SECRET_KEY)
|
|
export GITEA__OAUTH2__JWT_SECRET=$(gitea generate secret JWT_SECRET)
|
|
export GITEA__SERVER__LFS_JWT_SECRET=$(gitea generate secret LFS_JWT_SECRET)
|
|
|
|
env2ini::log "...Initial secrets generated\n"
|
|
}
|
|
|
|
# save existing envs prior to script execution. Necessary to keep order of preexisting and custom envs
|
|
env | (grep GITEA || [[ $? == 1 ]]) > /tmp/existing-envs
|
|
|
|
# MUST BE CALLED BEFORE OTHER CONFIGURATION
|
|
env2ini::generate_initial_secrets
|
|
|
|
env2ini::load_config_sources '/env-to-ini-mounts/inlines/'
|
|
env2ini::load_config_sources '/env-to-ini-mounts/additionals/'
|
|
|
|
# load existing envs to override auto generated envs
|
|
env2ini::reload_preset_envs
|
|
|
|
env2ini::log "=== All configuration sources loaded ===\n"
|
|
|
|
# safety to prevent rewrite of secret keys if an app.ini already exists
|
|
if [ -f ${GITEA_APP_INI} ]; then
|
|
env2ini::log 'An app.ini file already exists. To prevent overwriting secret keys, these settings are dropped and remain unchanged:'
|
|
env2ini::log ' - security.INTERNAL_TOKEN'
|
|
env2ini::log ' - security.SECRET_KEY'
|
|
env2ini::log ' - oauth2.JWT_SECRET'
|
|
env2ini::log ' - server.LFS_JWT_SECRET'
|
|
|
|
unset GITEA__SECURITY__INTERNAL_TOKEN
|
|
unset GITEA__SECURITY__SECRET_KEY
|
|
unset GITEA__OAUTH2__JWT_SECRET
|
|
unset GITEA__SERVER__LFS_JWT_SECRET
|
|
fi
|
|
|
|
environment-to-ini -o $GITEA_APP_INI
|