Replace/interpolate variables in files sync'ed from mounts (#1039)

This commit is contained in:
Geoff Bourne 2021-09-15 21:27:48 -05:00 committed by GitHub
parent 9f66622472
commit 4c52d0ec42
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
28 changed files with 157 additions and 178 deletions

View file

@ -8,7 +8,7 @@ on:
- java8-openj9
- java11*
- java16*
- test/multiarch/*
- test/*
tags:
- "[0-9]+.[0-9]+.[0-9]+"
- "[0-9]+.[0-9]+.[0-9]+-multiarch*"

View file

@ -7,7 +7,7 @@ on:
- openj9
- openj9-11
- adopt11
- test/*
- test/alpine/*
tags:
- "[0-9]+.[0-9]+.[0-9]+-java8"
- "[0-9]+.[0-9]+.[0-9]+-openj9"

View file

@ -3,7 +3,7 @@
Adding a new server `TYPE` can vary due to the complexity of obtaining and configuring each type; however, the addition of any server type includes at least the following steps:
1. Copy an existing "start-deploy*" script, such as [start-deployMohist](start-deployMohist) and rename it accordingly making sure to retain the "start-deploy" prefix
2. Modify the type-specific behavior between the "start-utils" preamble and the hand-off to `start-finalSetupWorld` at the end of the script
2. Modify the type-specific behavior between the "start-utils" preamble and the hand-off to `start-setupWorld` at the end of the script
3. Develop and test the changes using the [iterative process described below](#iterative-script-development)
4. Add a case-entry to the `case "${TYPE^^}"` in [start-configuration](start-configuration)
5. Add a section to the [README](README.md). It is recommended to copy-modify an existing section to retain a similar wording and level of detail

View file

@ -60,9 +60,10 @@ RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT} \
--var version=0.1.1 --var app=maven-metadata-release --file {{.app}} \
--from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
ARG ASCIIFY_VERSION=1.0.1
ADD https://jitpack.io/com/github/itzg/asciify/${ASCIIFY_VERSION}/asciify-${ASCIIFY_VERSION}.jar /usr/share/asciify/asciify.jar
RUN chmod -R a+r /usr/share/asciify
ARG MC_HELPER_VERSION=1.2.0
RUN curl -fsSL https://github.com/itzg/mc-image-helper/releases/download/v${MC_HELPER_VERSION}/mc-image-helper-${MC_HELPER_VERSION}.tgz \
| tar -C /usr/share -zxf - \
&& ln -s /usr/share/mc-image-helper-${MC_HELPER_VERSION}/bin/mc-image-helper /usr/bin
COPY mcstatus /usr/local/bin

View file

@ -656,20 +656,20 @@ then you apply a workaround by adding this to the run invocation:
There are optional volume paths that can be attached to supply content to be copied into the data area:
`/plugins`
: contents are copied into `/data/plugins` for Bukkit related server types. Set `PLUGINS_SYNC_UPDATE=false` if you want files from `/plugins` to take precedence over newer files in `/data/plugins`.
: contents are synchronized into `/data/plugins` for Bukkit related server types. Set `SYNC_SKIP_NEWER_IN_DESTINATION=false` if you want files from `/plugins` to take precedence over newer files in `/data/plugins`.
`/mods`
: contents are copied into `/data/mods` for Forge related server types
: contents are synchronized into `/data/mods` for Forge related server types. The destination can be changed by setting `COPY_MODS_DEST`.
`/config`
: contents are copied into `/data/config` by default, but can be changed with `COPY_CONFIG_DEST`
: contents are synchronized into `/data/config` by default, but can be changed with `COPY_CONFIG_DEST`. For example, `-v ./config:/config -e COPY_CONFIG_DEST=/data` will allow you to copy over files like `bukkit.yml` and so on directly into the server directory. Set `SYNC_SKIP_NEWER_IN_DESTINATION=false` if you want files from `/config` to take precedence over newer files in `/data/config`.
By default, the [environment variable processing](#replacing-variables-inside-configs) is performed on synchronized files that match the expected suffixes in `REPLACE_ENV_SUFFIXES` (by default "yml,yaml,txt,cfg,conf,properties,hjson,json,tml,toml") and are not excluded by `REPLACE_ENV_VARIABLES_EXCLUDES` and `REPLACE_ENV_VARIABLES_EXCLUDE_PATHS`. This processing can be disabled by setting `REPLACE_ENV_DURING_SYNC` to `false`.
If you want old mods/plugins to be removed before the content is brought over from those attach points, then add `-e REMOVE_OLD_MODS=TRUE`. You can fine tune the removal process by specifying the `REMOVE_OLD_MODS_INCLUDE` and `REMOVE_OLD_MODS_EXCLUDE` variables. By default, everything will be removed. You can also specify the `REMOVE_OLD_MODS_DEPTH` (default is 16) variable to only delete files up to a certain level.
For example: `-e REMOVE_OLD_MODS=TRUE -e REMOVE_OLD_MODS_INCLUDE="*.jar" -e REMOVE_OLD_MODS_DEPTH=1` will remove all old jar files that are directly inside the `plugins/` or `mods/` directory.
You can specify the destination of the files that are copied from `/mods` and `/config` by setting the `COPY_MODS_DEST` and `COPY_CONFIG_DEST`, where the default is `/data/mods` and `/data/config`. For example, `-v ./config:/config -e COPY_CONFIG_DEST=/data` will allow you to copy over files like `bukkit.yml` and so on directly into the server directory.
These paths work well if you want to have a common set of modules in a separate location, but still have multiple worlds with different server requirements in either persistent volumes or a downloadable archive.
> For more flexibility with mods/plugins preparation, you can declare directories to use in [the `MODS` variable](#downloadable-modplugin-pack-for-forge-bukkit-and-spigot-servers)
@ -768,21 +768,25 @@ This will reset any manual configuration of the `server.properties` file, so if
you want to make any persistent configuration changes you will need to make sure
you have properly set the proper environment variables in your docker run command (described below).
### Server name
### Message of the Day
The server name (e.g. for bungeecord) can be set like:
The message of the day, shown below each server entry in the client UI, can be changed with the `MOTD` environment variable, such as
docker run -d -e SERVER_NAME=MyServer ...
-e MOTD="My Server"
### Server port
If you leave it off, a default is computed from the server type and version, such as
> **WARNING:** only change this value if you know what you're doing. It is only needed when using host networking and it is rare that host networking should be used. Use `-p` port mappings instead.
A Paper Minecraft Server powered by Docker
If you must, the server port can be set like:
That way you can easily differentiate between several servers you may have started.
docker run -d -e SERVER_PORT=25566 ...
The section symbol (§) and other unicode characters are automatically converted to allow [formatting codes](https://minecraft.fandom.com/wiki/Formatting_codes) to be used consistently with all server versions. For example,
**however**, be sure to change your port mapping accordingly and be prepared for some features to break.
-e MOTD="A §l§cMinecraft§r §nserver"
renders
![](docs/motd-example.png)
### Difficulty
@ -971,26 +975,6 @@ For example:
docker run -d -e MODE=creative ...
### Message of the Day
The message of the day, shown below each server entry in the client UI, can be changed with the `MOTD` environment variable, such as
-e MOTD="My Server"
If you leave it off, a default is computed from the server type and version, such as
A Paper Minecraft Server powered by Docker
That way you can easily differentiate between several servers you may have started.
The section symbol (§) and other unicode characters are automatically converted to allow [formatting codes](https://minecraft.fandom.com/wiki/Formatting_codes) to be used consistently with all server versions. For example,
-e MOTD="A §l§cMinecraft§r §nserver"
renders
![](docs/motd-example.png)
### PVP Mode
By default, servers are created with player-vs-player (PVP) mode enabled. You can disable this with the `PVP`
@ -1058,6 +1042,22 @@ Allows users to use flight on your server while in Survival mode, if they have a
-e ALLOW_FLIGHT=TRUE|FALSE
### Server name
The server name (e.g. for bungeecord) can be set like:
docker run -d -e SERVER_NAME=MyServer ...
### Server port
> **WARNING:** only change this value if you know what you're doing. It is only needed when using host networking and it is rare that host networking should be used. Use `-p` port mappings instead.
If you must, the server port can be set like:
docker run -d -e SERVER_PORT=25566 ...
**however**, be sure to change your port mapping accordingly and be prepared for some features to break.
### Other server property mappings
| Environment Variable | Server Property |
@ -1090,24 +1090,13 @@ in your config files after the container starts.
For those cases there is the option to replace defined variables inside your configs
with environment variables defined at container runtime.
If you set the enviroment variable `REPLACE_ENV_VARIABLES` to `TRUE` the startup script
will go thru all files inside your `/data` volume and replace variables that match your
defined environment variables. Variables that you want to replace need to be wrapped
inside `${YOUR_VARIABLE}` curly brackets and prefixed with a dollar sign. This is the regular
syntax for enviromment variables inside strings or config files.
When the environment variable `REPLACE_ENV_IN_PLACE` is set to `true` (the default), the startup script will go through all files inside the container's `/data` path and replace variables that match the container's environment variables. Variables can instead (or in addition to) be replaced in files sync'ed from `/plugins`, `/mods`, and `/config` by setting `REPLACE_ENV_DURING_SYNC` to `true` (defaults to `false`).
Optionally you can also define a prefix to only match predefined environment variables.
Variables that you want to replace need to be declared inside curly brackets and prefixed with a dollar sign, such as `${CFG_YOUR_VARIABLE}`, which is same as many scripting languages.
`ENV_VARIABLE_PREFIX="CFG_"` <-- this is the default prefix
You can also change `REPLACE_ENV_VARIABLE_PREFIX`, which defaults to "CFG_", to limit which environment variables are allowed to be used. For example, with "CFG_" as the prefix, the variable `${CFG_DB_HOST}` would be subsituted, but not `${DB_HOST}`.
If you want use file for value (like when use secrets) you can add suffix `_FILE` to your variable name (in run command).
There are some limitations to what characters you can use.
| Type | Allowed Characters |
| ----- | ------------------- |
| Name | `0-9a-zA-Z_-` |
| Value | `0-9a-zA-Z_-:/=?.+` |
If you want to use a file for value (like when use secrets) you can add suffix `_FILE` to your variable name.
Variables will be replaced in files with the following extensions:
`.yml`, `.yaml`, `.txt`, `.cfg`, `.conf`, `.properties`.

View file

@ -101,7 +101,7 @@ if ! isTrue ${USE_MODPACK_START_SCRIPT:-true}; then
export FTB_DIR=$(dirname "${SERVER}")
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@
fi
entryScriptExpr="
@ -236,5 +236,4 @@ elif [ -e "${FTB_DIR}/Install.sh" ]; then
popd
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -29,4 +29,4 @@ fi
export SKIP_LOG4J_CONFIG=true
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld "$@"
exec ${SCRIPTS:-/}start-setupWorld "$@"

View file

@ -32,5 +32,4 @@ fi
export SKIP_LOG4J_CONFIG=true
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -79,5 +79,4 @@ if ! [ -v SERVER ]; then
exit 2
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -60,5 +60,4 @@ if [[ ! -e ${SERVER} ]]; then
mv fabric-server-launch.jar ${SERVER}
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -118,5 +118,4 @@ else
export SERVER=$(cat $installMarker)
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -60,5 +60,4 @@ export LEVEL
export SKIP_LOG4J_CONFIG=true
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -89,5 +89,4 @@ else
fi
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -40,5 +40,4 @@ fi
export SKIP_LOG4J_CONFIG=true
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld "$@"
exec ${SCRIPTS:-/}start-setupWorld "$@"

View file

@ -37,5 +37,4 @@ if [ ! -e $SERVER ] || [ -n "$FORCE_REDOWNLOAD" ]; then
curl -sSL -o $SERVER https://repo.spongepowered.org/maven/org/spongepowered/$TYPE/$SPONGEVERSION/$SERVER
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -45,5 +45,4 @@ fi
isDebugging && ls -l
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -26,5 +26,4 @@ fi
export TYPE=SPIGOT
export SKIP_LOG4J_CONFIG=true
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -1,51 +0,0 @@
#!/bin/bash
. ${SCRIPTS:-/}start-utils
: ${ENV_VARIABLE_PREFIX:=CFG_}
if isTrue "${REPLACE_ENV_VARIABLES}"; then
log "Replacing env variables in configs that match the prefix $ENV_VARIABLE_PREFIX ..."
# File excludes
fileExcludes=
for f in ${REPLACE_ENV_VARIABLES_EXCLUDES}; do
fileExcludes="${fileExcludes} -not -name $f"
done
# Directory excludes (recursive)
dirExcludes=$(join_by " -o -path " ${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS})
if [[ $dirExcludes ]]; then
dirExcludes=" -type d ( -path ${dirExcludes} ) -prune -o"
fi
isDebugging && echo "Using find file exclusions: $fileExcludes"
isDebugging && echo "Using find directory exclusions: $dirExcludes"
for name in $(compgen -v $ENV_VARIABLE_PREFIX); do
# check if name of env variable matches the prefix
# sanity check environment variables to avoid code injections
# Read content from file environment
if [[ $name = *"_FILE" ]]; then
value=$(<${!name})
name="${name%_FILE}"
else
value=${!name}
fi
log "Replacing $name with $value ..."
value=${value//\\/\\\\}
value=${value//#/\\#}
find /data/ \
$dirExcludes \
-type f \
\( -name "*.yml" -or -name "*.yaml" -or -name "*.txt" -or -name "*.cfg" \
-or -name "*.conf" -or -name "*.properties" -or -name "*.hjson" -or -name "*.json" \) \
$fileExcludes \
-exec sed -i 's#${'"$name"'}#'"$value"'#g' {} \;
done
fi
exec ${SCRIPTS:-/}start-minecraftFinalSetup $@

View file

@ -1,40 +0,0 @@
#!/bin/bash
. ${SCRIPTS:-/}start-utils
: ${PLUGINS_SYNC_UPDATE:=true}
isDebugging && set -x
if [ -d /plugins ]; then
case ${TYPE} in
SPIGOT|BUKKIT|PAPER|MAGMA)
mkdir -p /data/plugins
log "Copying plugins over..."
if isTrue ${PLUGINS_SYNC_UPDATE}; then
updateArg="--update"
fi
# Copy plugins over using rsync to allow deeply nested updates of plugins
rsync -a --out-format="update:%f:Last Modified %M" --prune-empty-dirs $updateArg /plugins /data
;;
esac
fi
# If any modules have been provided, copy them over
: ${COPY_MODS_DEST:="/data/mods"}
if [ -d /mods ]; then
log "Copying any mods over..."
mkdir -p "${COPY_MODS_DEST}"
rsync -a --out-format="update:%f:Last Modified %M" "${rsyncArgs[@]}" --prune-empty-dirs --update /mods/ "${COPY_MODS_DEST}"
fi
: ${COPY_CONFIG_DEST:="/data/config"}
if [ -d /config ]; then
log "Copying any configs from /config to ${COPY_CONFIG_DEST}"
mkdir -p "${COPY_CONFIG_DEST}"
rsync -a --out-format="update:%f:Last Modified %M" "${rsyncArgs[@]}" --prune-empty-dirs --update /config/ "${COPY_CONFIG_DEST}"
fi
exec ${SCRIPTS:-/}start-finalSetupServerProperties $@

24
start-setupEnvVariables Normal file
View file

@ -0,0 +1,24 @@
#!/bin/bash
. ${SCRIPTS:-/}start-utils
: ${REPLACE_ENV_IN_PLACE:=${REPLACE_ENV_VARIABLES:-false}}
: ${REPLACE_ENV_PATHS:=/data}
: ${REPLACE_ENV_SUFFIXES:=yml,yaml,txt,cfg,conf,properties,hjson,json,tml,toml}
: ${REPLACE_ENV_VARIABLE_PREFIX:=${ENV_VARIABLE_PREFIX:-CFG_}}
: ${REPLACE_ENV_VARIABLES_EXCLUDES:=}
: ${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS:=}
: ${DEBUG:=false}
if isTrue "${REPLACE_ENV_IN_PLACE}"; then
log "Replacing env variables in ${REPLACE_ENV_PATHS} that match the prefix $REPLACE_ENV_VARIABLE_PREFIX ..."
mc-image-helper --debug=${DEBUG} interpolate \
--replace-env-file-suffixes="${REPLACE_ENV_SUFFIXES}" \
--replace-env-excludes="${REPLACE_ENV_VARIABLES_EXCLUDES}" \
--replace-env-exclude-paths="${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS}" \
--replace-env-prefix=${REPLACE_ENV_VARIABLE_PREFIX} \
"${REPLACE_ENV_PATHS[@]}"
fi
exec ${SCRIPTS:-/}start-finalExec $@

View file

@ -24,4 +24,4 @@ case "X$MODCONFIG" in
esac
fi
exec ${SCRIPTS:-/}start-finalSetupMounts $@
exec ${SCRIPTS:-/}start-setupMounts $@

View file

@ -188,4 +188,4 @@ if [[ "${GENERIC_PACK}" ]]; then
fi
fi
exec ${SCRIPTS:-/}start-finalSetupModconfig $@
exec ${SCRIPTS:-/}start-setupModconfig $@

69
start-setupMounts Normal file
View file

@ -0,0 +1,69 @@
#!/bin/bash
. ${SCRIPTS:-/}start-utils
: ${SYNC_SKIP_NEWER_IN_DESTINATION:=${PLUGINS_SYNC_UPDATE:-true}}
: ${REPLACE_ENV_DURING_SYNC:=true}
: ${REPLACE_ENV_SUFFIXES:=yml,yaml,txt,cfg,conf,properties,hjson,json,tml,toml}
: ${REPLACE_ENV_VARIABLE_PREFIX:=${ENV_VARIABLE_PREFIX:-CFG_}}
: ${REPLACE_ENV_VARIABLES_EXCLUDES:=}
: ${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS:=}
: ${DEBUG:=false}
set -e
isDebugging && set -x
if isTrue ${SYNC_SKIP_NEWER_IN_DESTINATION}; then
updateArg="--skip-newer-in-destination"
fi
if isTrue ${REPLACE_ENV_DURING_SYNC}; then
subcommand=sync-and-interpolate
else
subcommand=sync
fi
if [ -d /plugins ]; then
case ${TYPE} in
SPIGOT|BUKKIT|PAPER|MAGMA)
mkdir -p /data/plugins
log "Copying plugins over..."
mc-image-helper \
--debug=${DEBUG} ${subcommand} $updateArg \
--replace-env-file-suffixes="${REPLACE_ENV_SUFFIXES}" \
--replace-env-excludes="${REPLACE_ENV_VARIABLES_EXCLUDES}" \
--replace-env-exclude-paths="${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS}" \
--replace-env-prefix=${REPLACE_ENV_VARIABLE_PREFIX} \
/plugins /data/plugins
;;
esac
fi
# If any modules have been provided, copy them over
: ${COPY_MODS_DEST:="/data/mods"}
if [ -d /mods ]; then
log "Copying any mods over..."
mc-image-helper \
--debug=${DEBUG} ${subcommand} $updateArg \
--replace-env-file-suffixes="${REPLACE_ENV_SUFFIXES}" \
--replace-env-excludes="${REPLACE_ENV_VARIABLES_EXCLUDES}" \
--replace-env-exclude-paths="${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS}" \
--replace-env-prefix=${REPLACE_ENV_VARIABLE_PREFIX} \
/mods "${COPY_MODS_DEST}"
fi
: ${COPY_CONFIG_DEST:="/data/config"}
if [ -d /config ]; then
log "Copying any configs from /config to ${COPY_CONFIG_DEST}"
mc-image-helper \
--debug=${DEBUG} ${subcommand} $updateArg \
--replace-env-file-suffixes="${REPLACE_ENV_SUFFIXES}" \
--replace-env-excludes="${REPLACE_ENV_VARIABLES_EXCLUDES}" \
--replace-env-exclude-paths="${REPLACE_ENV_VARIABLES_EXCLUDE_PATHS}" \
--replace-env-prefix=${REPLACE_ENV_VARIABLE_PREFIX} \
/config "${COPY_CONFIG_DEST}"
fi
exec ${SCRIPTS:-/}start-setupServerProperties $@

View file

@ -56,7 +56,7 @@ function customizeServerProps {
setServerProp "server-name" "$SERVER_NAME"
setServerProp "server-ip" "$SERVER_IP"
setServerProp "server-port" "$SERVER_PORT"
setServerProp "motd" "$(echo $MOTD | java -jar /usr/share/asciify/asciify.jar)"
setServerProp "motd" "$(echo $MOTD | mc-image-helper asciify)"
setServerProp "allow-nether" "$ALLOW_NETHER"
setServerProp "announce-player-achievements" "$ANNOUNCE_PLAYER_ACHIEVEMENTS"
setServerProp "enable-command-block" "$ENABLE_COMMAND_BLOCK"
@ -217,4 +217,4 @@ if isDebugging; then
cat "${SERVER_PROPERTIES}"
fi
exec ${SCRIPTS:-/}start-finalSetupEnvVariables $@
exec ${SCRIPTS:-/}start-setupEnvVariables $@

View file

@ -71,4 +71,4 @@ if [[ "$WORLD" ]] && ( isTrue "${FORCE_WORLD_COPY}" || [ ! -d "$worldDest" ] );
fi
fi
exec ${SCRIPTS:-/}start-finalSetupModpack $@
exec ${SCRIPTS:-/}start-setupModpack $@

View file

@ -101,5 +101,4 @@ if [[ ${SPIGET_RESOURCES} ]]; then
done
fi
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld $@
exec ${SCRIPTS:-/}start-setupWorld $@

View file

@ -14,5 +14,4 @@ fi
log "Resolved CURSE_INSTANCE_JSON as ${CURSE_INSTANCE_JSON}"
# Continue to Final Setup
exec ${SCRIPTS:-/}start-finalSetupWorld "$@"
exec ${SCRIPTS:-/}start-setupWorld "$@"