Update release notes

Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes #284.

.dockerignore

@@ -1,3 +1,5 @@
 output/
 work/
 deploy/
+apt-cacher-ng/
+.git/objects/*

.gitignore

@@ -3,6 +3,7 @@ work/*
 config
 postrun.sh
 SKIP
+SKIP_IMAGES
 .pc
 *-pc
 apt-cacher-ng/

Dockerfile

@@ -1,10 +1,12 @@
-FROM debian:stretch
+FROM debian:buster
+
+ENV DEBIAN_FRONTEND noninteractive
 
 RUN apt-get -y update && \
     apt-get -y install \
         git vim parted \
-        quilt realpath qemu-user-static debootstrap zerofree pxz zip dosfstools \
+        quilt coreutils qemu-user-static debootstrap zerofree zip dosfstools \
-        bsdtar libcap2-bin rsync grep udev xz-utils \
+        bsdtar libcap2-bin rsync grep udev xz-utils curl xxd file kmod\
     && rm -rf /var/lib/apt/lists/*
 
 COPY . /pi-gen/

README.md

@@ -3,17 +3,18 @@
 _Tool used to create the raspberrypi.org Raspbian images_
 
 
-### TODO
-1. Documentation
-
-
 ## Dependencies
 
-On Debian-based systems:
+pi-gen runs on Debian based operating systems. Currently it is only supported on
+either Debian Buster or Ubuntu Xenial and is known to have issues building on
+earlier releases of these systems. On other Linux distributions it may be possible
+to use the Docker build described below.
+
+To install the required dependencies for pi-gen you should run:
 
 ```bash
-apt-get install quilt parted realpath qemu-user-static debootstrap zerofree pxz zip \
+apt-get install coreutils quilt parted qemu-user-static debootstrap zerofree zip \
-dosfstools bsdtar libcap2-bin grep rsync xz-utils
+dosfstools bsdtar libcap2-bin grep rsync xz-utils file git curl
 ```
 
 The file `depends` contains a list of tools needed.  The format of this
@@ -62,10 +63,56 @@ The following environment variables are supported:
    system for each build stage, amounting to tens of gigabytes in the case of
    Raspbian.
    
+   **CAUTION**: If your working directory is on an NTFS partition you probably won't be able to build. Make sure this is a proper Linux filesystem.
+
  * `DEPLOY_DIR`  (Default: `"$BASE_DIR/deploy"`)
 
    Output directory for target system images and NOOBS bundles.
 
+ * `DEPLOY_ZIP` (Default: `1`)
+
+   Setting to `0` will deploy the actual image (`.img`) instead of a zipped image (`.zip`).
+
+ * `USE_QEMU` (Default: `"0"`)
+
+   Setting to '1' enables the QEMU mode - creating an image that can be mounted via QEMU for an emulated
+   environment. These images include "-qemu" in the image file name.
+
+ * `LOCALE_DEFAULT` (Default: "en_GB.UTF-8" )
+
+   Default system locale.
+
+ * `KEYBOARD_KEYMAP` (Default: "gb" )
+
+   Default keyboard keymap.
+
+ * `KEYBOARD_LAYOUT` (Default: "English (UK)" )
+
+   Default keyboard layout.
+
+ * `TIMEZONE_DEFAULT` (Default: "Europe/London" )
+
+   Default keyboard layout.
+
+ * `FIRST_USER_NAME` (Default: "pi" )
+
+   Username for the first user
+
+ * `FIRST_USER_PASS` (Default: "raspberry")
+
+   Password for the first user
+
+ * `WPA_ESSID`, `WPA_PASSWORD` and `WPA_COUNTRY` (Default: unset)
+
+   If these are set, they are use to configure `wpa_supplicant.conf`, so that the raspberry pi can automatically connect to a wifi network on first boot.
+
+ * `ENABLE_SSH` (Default: `0`)
+
+   Setting to `1` will enable ssh server for remote log in. Note that if you are using a common password such as the defaults there is a high risk of attackers taking over you RaspberryPi.
+
+ * `STAGE_LIST` (Default: `stage*`)
+
+    If set, then instead of working through the numeric stages in order, this list will be followed. For example setting to `"stage0 stage1 mystage stage2"` will run the contents of `mystage` before stage2. Note that quotes are needed around the list. An absolute or relative path can be given for stages outside the pi-gen directory.
 
 A simple example for building Raspbian:
 
@@ -73,9 +120,68 @@ A simple example for building Raspbian:
 IMG_NAME='Raspbian'
 ```
 
+The config file can also be specified on the command line as an argument the `build.sh` or `build-docker.sh` scripts.
+
+```
+./build.sh -c myconfig
+```
+
+This is parsed after `config` so can be used to override values set there.
+
+## How the build process works
+
+The following process is followed to build images:
+
+ * Loop through all of the stage directories in alphanumeric order
+
+ * Move on to the next directory if this stage directory contains a file called
+   "SKIP"
+
+ * Run the script ```prerun.sh``` which is generally just used to copy the build
+   directory between stages.
+
+ * In each stage directory loop through each subdirectory and then run each of the
+   install scripts it contains, again in alphanumeric order. These need to be named
+   with a two digit padded number at the beginning.
+   There are a number of different files and directories which can be used to
+   control different parts of the build process:
+
+     - **00-run.sh** - A unix shell script. Needs to be made executable for it to run.
+
+     - **00-run-chroot.sh** - A unix shell script which will be run in the chroot
+       of the image build directory. Needs to be made executable for it to run.
+
+     - **00-debconf** - Contents of this file are passed to debconf-set-selections
+       to configure things like locale, etc.
+
+     - **00-packages** - A list of packages to install. Can have more than one, space
+       separated, per line.
+
+     - **00-packages-nr** - As 00-packages, except these will be installed using
+       the ```--no-install-recommends -y``` parameters to apt-get.
+
+     - **00-patches** - A directory containing patch files to be applied, using quilt.
+       If a file named 'EDIT' is present in the directory, the build process will
+       be interrupted with a bash session, allowing an opportunity to create/revise
+       the patches.
+
+  * If the stage directory contains files called "EXPORT_NOOBS" or "EXPORT_IMAGE" then
+    add this stage to a list of images to generate
+
+  * Generate the images for any stages that have specified them
+
+It is recommended to examine build.sh for finer details.
+
 
 ## Docker Build
 
+Docker can be used to perform the build inside a container. This partially isolates
+the build from the host system, and allows using the script on non-debian based
+systems (e.g. Fedora Linux). The isolate is not complete due to the need to use
+some kernel level services for arm emulation (binfmt) and loop devices (losetup).
+
+To build:
+
 ```bash
 vi config         # Edit your config file. See above.
 ./build-docker.sh
@@ -91,6 +197,18 @@ continue:
 CONTINUE=1 ./build-docker.sh
 ```
 
+To examine the container after a failure you can enter a shell within it using:
+
+```bash
+sudo docker run -it --privileged --volumes-from=pigen_work pi-gen /bin/bash
+```
+
+After successful build, the build container is by default removed. This may be undesired when making incremental changes to a customized build. To prevent the build script from remove the container add
+
+```bash
+PRESERVE_CONTAINER=1 ./build-docker.sh
+```
+
 There is a possibility that even when running from a docker container, the
 installation of `qemu-user-static` will silently fail when building the image
 because `binfmt-support` _must be enabled on the underlying kernel_. An easy
@@ -159,17 +277,61 @@ If you wish to build up to a specified stage (such as building up to stage 2
 for a lite system), place an empty file named `SKIP` in each of the `./stage`
 directories you wish not to include.
 
-Then remove the `EXPORT*` files from `./stage4` (if building up to stage 2) or
+Then add an empty file named `SKIP_IMAGES` to `./stage4` (if building up to stage 2) or
-from `./stage2` (if building a minimal system).
+to `./stage2` (if building a minimal system).
 
 ```bash
 # Example for building a lite system
 echo "IMG_NAME='Raspbian'" > config
 touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
-rm stage4/EXPORT*
+touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
 sudo ./build.sh  # or ./build-docker.sh
 ```
 
 If you wish to build further configurations upon (for example) the lite
 system, you can also delete the contents of `./stage3` and `./stage4` and
 replace with your own contents in the same format.
+
+
+## Skipping stages to speed up development
+
+If you're working on a specific stage the recommended development process is as
+follows:
+
+ * Add a file called SKIP_IMAGES into the directories containing EXPORT_* files
+   (currently stage2, stage4 and stage5)
+ * Add SKIP files to the stages you don't want to build. For example, if you're
+   basing your image on the lite image you would add these to stages 3, 4 and 5.
+ * Run build.sh to build all stages
+ * Add SKIP files to the earlier successfully built stages
+ * Modify the last stage
+ * Rebuild just the last stage using ```sudo CLEAN=1 ./build.sh```
+ * Once you're happy with the image you can remove the SKIP_IMAGES files and
+   export your image to test
+
+# Troubleshooting
+
+## `64 Bit Systems`
+Please note there is currently an issue when compiling with a 64 Bit OS. See https://github.com/RPi-Distro/pi-gen/issues/271
+
+## `binfmt_misc`
+
+Linux is able execute binaries from other architectures, meaning that it should be
+possible to make use of `pi-gen` on an x86_64 system, even though it will be running
+ARM binaries. This requires support from the [`binfmt_misc`](https://en.wikipedia.org/wiki/Binfmt_misc)
+kernel module.
+
+You may see the following error:
+
+```
+update-binfmts: warning: Couldn't load the binfmt_misc module.
+```
+
+To resolve this, ensure that the following files are available (install them if necessary):
+
+```
+/lib/modules/$(uname -r)/kernel/fs/binfmt_misc.ko
+/usr/bin/qemu-arm-static
+```
+
+You may also need to load the module by hand - run `modprobe binfmt_misc`.

build-docker.sh

@@ -1,77 +1,106 @@
-#!/bin/bash -e
+#!/bin/bash -eu
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+
+BUILD_OPTS="$*"
 
 DOCKER="docker"
-set +e
+
-$DOCKER ps >/dev/null 2>&1
+if ! ${DOCKER} ps >/dev/null 2>&1; then
-if [ $? != 0 ]; then
 	DOCKER="sudo docker"
 fi
-if ! $DOCKER ps >/dev/null; then
+if ! ${DOCKER} ps >/dev/null; then
 	echo "error connecting to docker:"
-	$DOCKER ps
+	${DOCKER} ps
 	exit 1
 fi
-set -e
 
+CONFIG_FILE=""
+if [ -f "${DIR}/config" ]; then
+	CONFIG_FILE="${DIR}/config"
+fi
 
-config_mount=()
+while getopts "c:" flag
-if [ -f config ]; then
+do
-	config_mount=("-v" "$(pwd)/config:/pi-gen/config:ro")
+	case "${flag}" in
-	source config
+		c)
+			CONFIG_FILE="${OPTARG}"
+			;;
+		*)
+			;;
+	esac
+done
+
+# Ensure that the configuration file is an absolute path
+CONFIG_FILE=$(realpath -s "$CONFIG_FILE")
+
+# Ensure that the confguration file is present
+if test -z "${CONFIG_FILE}"; then
+	echo "Configuration file need to be present in '${DIR}/config' or path passed as parameter"
+	exit 1
+else
+	# shellcheck disable=SC1090
+	source "${CONFIG_FILE}"
 fi
 
 CONTAINER_NAME=${CONTAINER_NAME:-pigen_work}
 CONTINUE=${CONTINUE:-0}
+PRESERVE_CONTAINER=${PRESERVE_CONTAINER:-0}
 
-if [ "$*" != "" ] || [ -z "${IMG_NAME}" ]; then
+if [ -z "${IMG_NAME}" ]; then
-	if [ -z "${IMG_NAME}" ]; then
+	echo "IMG_NAME not set in 'config'" 1>&2
-		echo "IMG_NAME not set in 'build'" 1>&2
 	echo 1>&2
-	fi
+exit 1
-	cat >&2 <<EOF
-Usage:
-    build-docker.sh [options]
-Optional environment arguments: ( =<default> )
-    CONTAINER_NAME=pigen_work  set a name for the build container
-    CONTINUE=0                  continue from a previously started container
-EOF
-	exit 1
 fi
 
-CONTAINER_EXISTS=$($DOCKER ps -a --filter name="$CONTAINER_NAME" -q)
+# Ensure the Git Hash is recorded before entering the docker container
-CONTAINER_RUNNING=$($DOCKER ps --filter name="$CONTAINER_NAME" -q)
+GIT_HASH=${GIT_HASH:-"$(git rev-parse HEAD)"}
-if [ "$CONTAINER_RUNNING" != "" ]; then
+
-	echo "The build is already running in container $CONTAINER_NAME. Aborting."
+CONTAINER_EXISTS=$(${DOCKER} ps -a --filter name="${CONTAINER_NAME}" -q)
+CONTAINER_RUNNING=$(${DOCKER} ps --filter name="${CONTAINER_NAME}" -q)
+if [ "${CONTAINER_RUNNING}" != "" ]; then
+	echo "The build is already running in container ${CONTAINER_NAME}. Aborting."
 	exit 1
 fi
-if [ "$CONTAINER_EXISTS" != "" ] && [ "$CONTINUE" != "1" ]; then
+if [ "${CONTAINER_EXISTS}" != "" ] && [ "${CONTINUE}" != "1" ]; then
-	echo "Container $CONTAINER_NAME already exists and you did not specify CONTINUE=1. Aborting."
+	echo "Container ${CONTAINER_NAME} already exists and you did not specify CONTINUE=1. Aborting."
 	echo "You can delete the existing container like this:"
-	echo "  docker rm -v $CONTAINER_NAME"
+	echo "  ${DOCKER} rm -v ${CONTAINER_NAME}"
 	exit 1
 fi
 
-$DOCKER build -t pi-gen .
+# Modify original build-options to allow config file to be mounted in the docker container
-if [ "$CONTAINER_EXISTS" != "" ]; then
+BUILD_OPTS="$(echo "${BUILD_OPTS:-}" | sed -E 's@\-c\s?([^ ]+)@-c /config@')"
-	trap "echo 'got CTRL+C... please wait 5s';docker stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
+
-	time $DOCKER run --rm --privileged \
+${DOCKER} build -t pi-gen "${DIR}"
+if [ "${CONTAINER_EXISTS}" != "" ]; then
+	trap 'echo "got CTRL+C... please wait 5s" && ${DOCKER} stop -t 5 ${CONTAINER_NAME}_cont' SIGINT SIGTERM
+	time ${DOCKER} run --rm --privileged \
+		--volume "${CONFIG_FILE}":/config:ro \
+		-e "GIT_HASH=${GIT_HASH}" \
 		--volumes-from="${CONTAINER_NAME}" --name "${CONTAINER_NAME}_cont" \
-		-e IMG_NAME=${IMG_NAME}\
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
-	cd /pi-gen; ./build.sh;
+	cd /pi-gen; ./build.sh ${BUILD_OPTS} &&
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 else
-	trap "echo 'got CTRL+C... please wait 5s'; docker stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
+	trap 'echo "got CTRL+C... please wait 5s" && ${DOCKER} stop -t 5 ${CONTAINER_NAME}' SIGINT SIGTERM
-	$DOCKER run --name "${CONTAINER_NAME}" --privileged \
+	time ${DOCKER} run --name "${CONTAINER_NAME}" --privileged \
-		-e IMG_NAME=${IMG_NAME}\
+		--volume "${CONFIG_FILE}":/config:ro \
-		-v "$(pwd)/deploy:/pi-gen/deploy" \
+		-e "GIT_HASH=${GIT_HASH}" \
-		"${config_mount[@]}" \
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
-	cd /pi-gen; ./build.sh &&
+	cd /pi-gen; ./build.sh ${BUILD_OPTS} &&
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 fi
+echo "copying results from deploy/"
+${DOCKER} cp "${CONTAINER_NAME}":/pi-gen/deploy .
+ls -lah deploy
+
+# cleanup
+if [ "${PRESERVE_CONTAINER}" != "1" ]; then
+	${DOCKER} rm -v "${CONTAINER_NAME}"
+fi
+
 echo "Done! Your image(s) should be in deploy/"

build.sh

@@ -1,22 +1,23 @@
 #!/bin/bash -e
-
+# shellcheck disable=SC2119
 run_sub_stage()
 {
 	log "Begin ${SUB_STAGE_DIR}"
-	pushd ${SUB_STAGE_DIR} > /dev/null
+	pushd "${SUB_STAGE_DIR}" > /dev/null
 	for i in {00..99}; do
-		if [ -f ${i}-debconf ]; then
+		if [ -f "${i}-debconf" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-debconf"
 			on_chroot << EOF
 debconf-set-selections <<SELEOF
-`cat ${i}-debconf`
+$(cat "${i}-debconf")
 SELEOF
 EOF
+
 		log "End ${SUB_STAGE_DIR}/${i}-debconf"
 		fi
-		if [ -f ${i}-packages-nr ]; then
+		if [ -f "${i}-packages-nr" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages-nr"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages-nr)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages-nr")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install --no-install-recommends -y $PACKAGES
@@ -24,9 +25,9 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages-nr"
 		fi
-		if [ -f ${i}-packages ]; then
+		if [ -f "${i}-packages" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install -y $PACKAGES
@@ -34,22 +35,22 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages"
 		fi
-		if [ -d ${i}-patches ]; then
+		if [ -d "${i}-patches" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-patches"
-			pushd ${STAGE_WORK_DIR} > /dev/null
+			pushd "${STAGE_WORK_DIR}" > /dev/null
 			if [ "${CLEAN}" = "1" ]; then
 				rm -rf .pc
-				rm -rf *-pc
+				rm -rf ./*-pc
 			fi
-			QUILT_PATCHES=${SUB_STAGE_DIR}/${i}-patches
+			QUILT_PATCHES="${SUB_STAGE_DIR}/${i}-patches"
-			SUB_STAGE_QUILT_PATCH_DIR="$(basename $SUB_STAGE_DIR)-pc"
+			SUB_STAGE_QUILT_PATCH_DIR="$(basename "$SUB_STAGE_DIR")-pc"
-			mkdir -p $SUB_STAGE_QUILT_PATCH_DIR
+			mkdir -p "$SUB_STAGE_QUILT_PATCH_DIR"
-			ln -snf $SUB_STAGE_QUILT_PATCH_DIR .pc
+			ln -snf "$SUB_STAGE_QUILT_PATCH_DIR" .pc
-			if [ -e ${SUB_STAGE_DIR}/${i}-patches/EDIT ]; then
+			quilt upgrade
+			if [ -e "${SUB_STAGE_DIR}/${i}-patches/EDIT" ]; then
 				echo "Dropping into bash to edit patches..."
 				bash
 			fi
-			quilt upgrade
 			RC=0
 			quilt push -a || RC=$?
 			case "$RC" in
@@ -80,18 +81,20 @@ EOF
 
 run_stage(){
 	log "Begin ${STAGE_DIR}"
-	STAGE=$(basename ${STAGE_DIR})
+	STAGE="$(basename "${STAGE_DIR}")"
-	pushd ${STAGE_DIR} > /dev/null
+	pushd "${STAGE_DIR}" > /dev/null
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	STAGE_WORK_DIR=${WORK_DIR}/${STAGE}
+	STAGE_WORK_DIR="${WORK_DIR}/${STAGE}"
-	ROOTFS_DIR=${STAGE_WORK_DIR}/rootfs
+	ROOTFS_DIR="${STAGE_WORK_DIR}"/rootfs
-	if [ -f ${STAGE_DIR}/EXPORT_IMAGE ]; then
+	if [ ! -f SKIP_IMAGES ]; then
+		if [ -f "${STAGE_DIR}/EXPORT_IMAGE" ]; then
 			EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
 		fi
+	fi
 	if [ ! -f SKIP ]; then
 		if [ "${CLEAN}" = "1" ]; then
-			if [ -d ${ROOTFS_DIR} ]; then
+			if [ -d "${ROOTFS_DIR}" ]; then
-				rm -rf ${ROOTFS_DIR}
+				rm -rf "${ROOTFS_DIR}"
 			fi
 		fi
 		if [ -x prerun.sh ]; then
@@ -99,17 +102,17 @@ run_stage(){
 			./prerun.sh
 			log "End ${STAGE_DIR}/prerun.sh"
 		fi
-		for SUB_STAGE_DIR in ${STAGE_DIR}/*; do
+		for SUB_STAGE_DIR in "${STAGE_DIR}"/*; do
-			if [ -d ${SUB_STAGE_DIR} ] &&
+			if [ -d "${SUB_STAGE_DIR}" ] &&
-			   [ ! -f ${SUB_STAGE_DIR}/SKIP ]; then
+			   [ ! -f "${SUB_STAGE_DIR}/SKIP" ]; then
 				run_sub_stage
 			fi
 		done
 	fi
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	PREV_STAGE=${STAGE}
+	PREV_STAGE="${STAGE}"
-	PREV_STAGE_DIR=${STAGE_DIR}
+	PREV_STAGE_DIR="${STAGE_DIR}"
-	PREV_ROOTFS_DIR=${ROOTFS_DIR}
+	PREV_ROOTFS_DIR="${ROOTFS_DIR}"
 	popd > /dev/null
 	log "End ${STAGE_DIR}"
 }
@@ -119,23 +122,63 @@ if [ "$(id -u)" != "0" ]; then
 	exit 1
 fi
 
+
 if [ -f config ]; then
+	# shellcheck disable=SC1091
 	source config
 fi
 
+while getopts "c:" flag
+do
+	case "$flag" in
+		c)
+			EXTRA_CONFIG="$OPTARG"
+			# shellcheck disable=SC1090
+			source "$EXTRA_CONFIG"
+			;;
+		*)
+			;;
+	esac
+done
+
+export PI_GEN=${PI_GEN:-pi-gen}
+export PI_GEN_REPO=${PI_GEN_REPO:-https://github.com/RPi-Distro/pi-gen}
+
 if [ -z "${IMG_NAME}" ]; then
 	echo "IMG_NAME not set" 1>&2
 	exit 1
 fi
 
-export IMG_DATE=${IMG_DATE:-"$(date +%Y-%m-%d)"}
+export USE_QEMU="${USE_QEMU:-0}"
+export IMG_DATE="${IMG_DATE:-"$(date +%Y-%m-%d)"}"
+export IMG_FILENAME="${IMG_FILENAME:-"${IMG_DATE}-${IMG_NAME}"}"
+export ZIP_FILENAME="${ZIP_FILENAME:-"image_${IMG_DATE}-${IMG_NAME}"}"
 
-export BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 export SCRIPT_DIR="${BASE_DIR}/scripts"
-export WORK_DIR=${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}
+export WORK_DIR="${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}"
 export DEPLOY_DIR=${DEPLOY_DIR:-"${BASE_DIR}/deploy"}
+export DEPLOY_ZIP="${DEPLOY_ZIP:-1}"
 export LOG_FILE="${WORK_DIR}/build.log"
 
+export FIRST_USER_NAME=${FIRST_USER_NAME:-pi}
+export FIRST_USER_PASS=${FIRST_USER_PASS:-raspberry}
+export WPA_ESSID
+export WPA_PASSWORD
+export WPA_COUNTRY
+export ENABLE_SSH="${ENABLE_SSH:-0}"
+
+export LOCALE_DEFAULT="${LOCALE_DEFAULT:-en_GB.UTF-8}"
+
+export KEYBOARD_KEYMAP="${KEYBOARD_KEYMAP:-gb}"
+export KEYBOARD_LAYOUT="${KEYBOARD_LAYOUT:-English (UK)}"
+
+export TIMEZONE_DEFAULT="${TIMEZONE_DEFAULT:-Europe/London}"
+
+export GIT_HASH=${GIT_HASH:-"$(git rev-parse HEAD)"}
+
+export BASE_DIR
+
 export CLEAN
 export IMG_NAME
 export APT_PROXY
@@ -158,30 +201,49 @@ export QUILT_NO_DIFF_INDEX=1
 export QUILT_NO_DIFF_TIMESTAMPS=1
 export QUILT_REFRESH_ARGS="-p ab"
 
-source ${SCRIPT_DIR}/common
+# shellcheck source=scripts/common
-source ${SCRIPT_DIR}/dependencies_check
+source "${SCRIPT_DIR}/common"
+# shellcheck source=scripts/dependencies_check
+source "${SCRIPT_DIR}/dependencies_check"
 
+dependencies_check "${BASE_DIR}/depends"
 
-dependencies_check ${BASE_DIR}/depends
+#check username is valid
+if [[ ! "$FIRST_USER_NAME" =~ ^[a-z][-a-z0-9_]*$ ]]; then
+	echo "Invalid FIRST_USER_NAME: $FIRST_USER_NAME"
+	exit 1
+fi
 
-mkdir -p ${WORK_DIR}
+if [[ -n "${APT_PROXY}" ]] && ! curl --silent "${APT_PROXY}" >/dev/null ; then
+	echo "Could not reach APT_PROXY server: ${APT_PROXY}"
+	exit 1
+fi
+
+mkdir -p "${WORK_DIR}"
 log "Begin ${BASE_DIR}"
 
-for STAGE_DIR in ${BASE_DIR}/stage*; do
+STAGE_LIST=${STAGE_LIST:-${BASE_DIR}/stage*}
+
+for STAGE_DIR in $STAGE_LIST; do
+	STAGE_DIR=$(realpath "${STAGE_DIR}")
 	run_stage
 done
 
 CLEAN=1
 for EXPORT_DIR in ${EXPORT_DIRS}; do
 	STAGE_DIR=${BASE_DIR}/export-image
+	# shellcheck source=/dev/null
 	source "${EXPORT_DIR}/EXPORT_IMAGE"
-	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename ${EXPORT_DIR})/rootfs
+	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename "${EXPORT_DIR}")/rootfs
 	run_stage
-	if [ -e ${EXPORT_DIR}/EXPORT_NOOBS ]; then
+	if [ "${USE_QEMU}" != "1" ]; then
-		source ${EXPORT_DIR}/EXPORT_NOOBS
+		if [ -e "${EXPORT_DIR}/EXPORT_NOOBS" ]; then
-		STAGE_DIR=${BASE_DIR}/export-noobs
+			# shellcheck source=/dev/null
+			source "${EXPORT_DIR}/EXPORT_NOOBS"
+			STAGE_DIR="${BASE_DIR}/export-noobs"
 			run_stage
 		fi
+	fi
 done
 
 if [ -x postrun.sh ]; then

depends

@@ -1,13 +1,18 @@
 quilt
 parted
-realpath
+realpath:coreutils
 qemu-arm-static:qemu-user-static
 debootstrap
 zerofree
-pxz zip
+zip
 mkdosfs:dosfstools
 capsh:libcap2-bin
 bsdtar
 grep
 rsync
 xz:xz-utils
+curl
+xxd
+file
+git
+lsmod:kmod

export-image/00-allow-rerun/00-run.sh

@@ -1,13 +1,9 @@
 #!/bin/bash -e
 
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload ]; then
+if [ ! -x "${ROOTFS_DIR}/usr/bin/qemu-arm-static" ]; then
-	mv ${ROOTFS_DIR}/etc/ld.so.preload ${ROOTFS_DIR}/etc/ld.so.preload.disabled
+	cp /usr/bin/qemu-arm-static "${ROOTFS_DIR}/usr/bin/"
 fi
 
-if [ ! -e ${ROOTFS_DIR}/usr/sbin/policy-rc.d ]; then
+if [ -e "${ROOTFS_DIR}/etc/ld.so.preload" ]; then
-	install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/
+	mv "${ROOTFS_DIR}/etc/ld.so.preload" "${ROOTFS_DIR}/etc/ld.so.preload.disabled"
-fi
-
-if [ ! -x ${ROOTFS_DIR}/usr/bin/qemu-arm-static ]; then
-	cp /usr/bin/qemu-arm-static ${ROOTFS_DIR}/usr/bin/
 fi

export-image/00-allow-rerun/files/policy-rc.d

@@ -1,3 +0,0 @@
-#!/bin/sh
-
-exit 101

export-image/01-set-sources/00-patches/0-sources.diff

@@ -1,9 +0,0 @@
-Index: export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
-===================================================================
---- export-jessie.orig/rootfs/etc/apt/sources.list.d/raspi.list
-+++ export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
-@@ -1,3 +1,3 @@
--deb http://archive.raspberrypi.org/debian/ jessie main ui staging
-+deb http://archive.raspberrypi.org/debian/ jessie main ui
- # Uncomment line below then 'apt-get update' to enable 'apt-get source'
- #deb-src http://archive.raspberrypi.org/debian/ jessie main ui

export-image/01-set-sources/00-patches/series

@@ -1 +0,0 @@
-0-sources.diff

export-image/02-network/01-run.sh

@@ -1,3 +1,3 @@
 #!/bin/bash -e
 
-install -m 644 files/resolv.conf ${ROOTFS_DIR}/etc/
+install -m 644 files/resolv.conf "${ROOTFS_DIR}/etc/"

export-image/03-set-partuuid/00-run.sh

@@ -1,13 +1,13 @@
 #!/bin/bash -e
 
-IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
 
-IMGID="$(fdisk -l ${IMG_FILE} | sed -n 's/Disk identifier: 0x\([^ ]*\)/\1/p')"
+IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
 
 BOOT_PARTUUID="${IMGID}-01"
 ROOT_PARTUUID="${IMGID}-02"
 
-sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
 
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/boot/cmdline.txt
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/cmdline.txt"

export-image/04-finalise/01-run.sh

@@ -1,64 +1,101 @@
 #!/bin/bash -e
 
-IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
+INFO_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.info"
 
 on_chroot << EOF
-/etc/init.d/fake-hwclock stop
+if [ -x /etc/init.d/fake-hwclock ]; then
-hardlink -t /usr/share/doc
+	/etc/init.d/fake-hwclock stop
+fi
+if hash hardlink 2>/dev/null; then
+	hardlink -t /usr/share/doc
+fi
 EOF
 
-if [ -d ${ROOTFS_DIR}/home/pi/.config ]; then
+if [ -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.config" ]; then
-	chmod 700 ${ROOTFS_DIR}/home/pi/.config
+	chmod 700 "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.config"
 fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-rm -f ${ROOTFS_DIR}/usr/sbin/policy-rc.d
+rm -f "${ROOTFS_DIR}/usr/bin/qemu-arm-static"
-rm -f ${ROOTFS_DIR}/usr/bin/qemu-arm-static
+
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload.disabled ]; then
+if [ "${USE_QEMU}" != "1" ]; then
-        mv ${ROOTFS_DIR}/etc/ld.so.preload.disabled ${ROOTFS_DIR}/etc/ld.so.preload
+	if [ -e "${ROOTFS_DIR}/etc/ld.so.preload.disabled" ]; then
+		mv "${ROOTFS_DIR}/etc/ld.so.preload.disabled" "${ROOTFS_DIR}/etc/ld.so.preload"
+	fi
 fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/sources.list~
+rm -f "${ROOTFS_DIR}/etc/network/interfaces.dpkg-old"
-rm -f ${ROOTFS_DIR}/etc/apt/trusted.gpg~
 
-rm -f ${ROOTFS_DIR}/etc/passwd-
+rm -f "${ROOTFS_DIR}/etc/apt/sources.list~"
-rm -f ${ROOTFS_DIR}/etc/group-
+rm -f "${ROOTFS_DIR}/etc/apt/trusted.gpg~"
-rm -f ${ROOTFS_DIR}/etc/shadow-
-rm -f ${ROOTFS_DIR}/etc/gshadow-
 
-rm -f ${ROOTFS_DIR}/var/cache/debconf/*-old
+rm -f "${ROOTFS_DIR}/etc/passwd-"
-rm -f ${ROOTFS_DIR}/var/lib/dpkg/*-old
+rm -f "${ROOTFS_DIR}/etc/group-"
+rm -f "${ROOTFS_DIR}/etc/shadow-"
+rm -f "${ROOTFS_DIR}/etc/gshadow-"
+rm -f "${ROOTFS_DIR}/etc/subuid-"
+rm -f "${ROOTFS_DIR}/etc/subgid-"
 
-rm -f ${ROOTFS_DIR}/usr/share/icons/*/icon-theme.cache
+rm -f "${ROOTFS_DIR}"/var/cache/debconf/*-old
+rm -f "${ROOTFS_DIR}"/var/lib/dpkg/*-old
 
-rm -f ${ROOTFS_DIR}/var/lib/dbus/machine-id
+rm -f "${ROOTFS_DIR}"/usr/share/icons/*/icon-theme.cache
 
-true > ${ROOTFS_DIR}/etc/machine-id
+rm -f "${ROOTFS_DIR}/var/lib/dbus/machine-id"
 
-ln -nsf /proc/mounts ${ROOTFS_DIR}/etc/mtab
+true > "${ROOTFS_DIR}/etc/machine-id"
 
-for _FILE in $(find ${ROOTFS_DIR}/var/log/ -type f); do
+ln -nsf /proc/mounts "${ROOTFS_DIR}/etc/mtab"
-	true > ${_FILE}
+
-done
+find "${ROOTFS_DIR}/var/log/" -type f -exec cp /dev/null {} \;
 
 rm -f "${ROOTFS_DIR}/root/.vnc/private.key"
+rm -f "${ROOTFS_DIR}/etc/vnc/updateid"
 
-update_issue $(basename ${EXPORT_DIR})
+update_issue "$(basename "${EXPORT_DIR}")"
-install -m 644 ${ROOTFS_DIR}/etc/rpi-issue ${ROOTFS_DIR}/boot/issue.txt
+install -m 644 "${ROOTFS_DIR}/etc/rpi-issue" "${ROOTFS_DIR}/boot/issue.txt"
-install files/LICENSE.oracle ${ROOTFS_DIR}/boot/
 
-ROOT_DEV=$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')
+cp "$ROOTFS_DIR/etc/rpi-issue" "$INFO_FILE"
 
-unmount ${ROOTFS_DIR}
-zerofree -v ${ROOT_DEV}
 
-unmount_image ${IMG_FILE}
+{
+	if [ -f "$ROOTFS_DIR/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" ]; then
+		firmware=$(zgrep "firmware as of" \
+			"$ROOTFS_DIR/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" | \
+			head -n1 | sed  -n 's|.* \([^ ]*\)$|\1|p')
+		printf "\nFirmware: https://github.com/raspberrypi/firmware/tree/%s\n" "$firmware"
 
-mkdir -p ${DEPLOY_DIR}
+		kernel="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/git_hash")"
+		printf "Kernel: https://github.com/raspberrypi/linux/tree/%s\n" "$kernel"
 
-rm -f ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip
+		uname="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/uname_string7")"
+		printf "Uname string: %s\n" "$uname"
+	fi
 
-echo zip ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip ${IMG_FILE}
+	printf "\nPackages:\n"
-pushd ${STAGE_WORK_DIR} > /dev/null
+	dpkg -l --root "$ROOTFS_DIR"
-zip ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip $(basename ${IMG_FILE})
+} >> "$INFO_FILE"
-popd > /dev/null
+
+ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
+
+unmount "${ROOTFS_DIR}"
+zerofree "${ROOT_DEV}"
+
+unmount_image "${IMG_FILE}"
+
+mkdir -p "${DEPLOY_DIR}"
+
+rm -f "${DEPLOY_DIR}/${ZIP_FILENAME}${IMG_SUFFIX}.zip"
+rm -f "${DEPLOY_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
+
+if [ "${DEPLOY_ZIP}" == "1" ]; then
+	pushd "${STAGE_WORK_DIR}" > /dev/null
+	zip "${DEPLOY_DIR}/${ZIP_FILENAME}${IMG_SUFFIX}.zip" \
+		"$(basename "${IMG_FILE}")"
+	popd > /dev/null
+else
+	cp "$IMG_FILE" "$DEPLOY_DIR"
+fi
+
+cp "$INFO_FILE" "$DEPLOY_DIR"

export-image/04-finalise/files/LICENSE.oracle

@@ -1,72 +0,0 @@
-Taken from http://www.java.com/license
-
-Oracle Binary Code License Agreement for the Java SE Platform Products and JavaFX
-
-ORACLE AMERICA, INC. ("ORACLE"), FOR AND ON BEHALF OF ITSELF AND ITS SUBSIDIARIES AND AFFILIATES UNDER COMMON CONTROL, IS WILLING TO LICENSE THE SOFTWARE TO YOU ONLY UPON THE CONDITION THAT YOU ACCEPT ALL OF THE TERMS CONTAINED IN THIS BINARY CODE LICENSE AGREEMENT AND SUPPLEMENTAL LICENSE TERMS (COLLECTIVELY "AGREEMENT"). PLEASE READ THE AGREEMENT CAREFULLY. BY SELECTING THE "ACCEPT LICENSE AGREEMENT" (OR THE EQUIVALENT) BUTTON AND/OR BY USING THE SOFTWARE YOU ACKNOWLEDGE THAT YOU HAVE READ THE TERMS AND AGREE TO THEM. IF YOU ARE AGREEING TO THESE TERMS ON BEHALF OF A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT YOU HAVE THE LEGAL AUTHORITY TO BIND THE LEGAL ENTITY TO THESE TERMS. IF YOU DO NOT HAVE SUCH AUTHORITY, OR IF YOU DO NOT WISH TO BE BOUND BY THE TERMS, THEN SELECT THE "DECLINE LICENSE AGREEMENT" (OR THE EQUIVALENT) BUTTON AND YOU MUST NOT USE THE SOFTWARE ON THIS SITE OR ANY OTHER MEDIA ON WHICH THE SOFTWARE IS CONTAINED.
-
-1. DEFINITIONS. "Software" means the software identified above in binary form that you selected for download, install or use (in the version You selected for download, install or use) from Oracle or its authorized licensees, any other machine readable materials (including, but not limited to, libraries, source files, header files, and data files), any updates or error corrections provided by Oracle, and any user manuals, programming guides and other documentation provided to you by Oracle under this Agreement. "General Purpose Desktop Computers and Servers" means computers, including desktop and laptop computers, or servers, used for general computing functions under end user control (such as but not specifically limited to email, general purpose Internet browsing, and office suite productivity tools). The use of Software in systems and solutions that provide dedicated functionality (other than as mentioned above) or designed for use in embedded or function-specific software applications, for example but not limited to: Software embedded in or bundled with industrial control systems, wireless mobile telephones, wireless handheld devices, kiosks, TV/STB, Blu-ray Disc devices, telematics and network control switching equipment, printers and storage management systems, and other related systems are excluded from this definition and not licensed under this Agreement. "Programs" means (a) Java technology applets and applications intended to run on the Java Platform, Standard Edition platform on Java-enabled General Purpose Desktop Computers and Servers; and (b) JavaFX technology applications intended to run on the JavaFX Runtime on JavaFX-enabled General Purpose Desktop Computers and Servers. “Commercial Features” means those features identified in Table 1-1 (Commercial Features In Java SE Product Editions) of the Java SE documentation accessible at http://www.oracle.com/technetwork/java/javase/documentation/index.html. “README File” means the README file for the Software accessible at http://www.oracle.com/technetwork/java/javase/documentation/index.html.
-
-2. LICENSE TO USE. Subject to the terms and conditions of this Agreement including, but not limited to, the Java Technology Restrictions of the Supplemental License Terms, Oracle grants you a non-exclusive, non-transferable, limited license without license fees to reproduce and use internally the Software complete and unmodified for the sole purpose of running Programs. THE LICENSE SET FORTH IN THIS SECTION 2 DOES NOT EXTEND TO THE COMMERCIAL FEATURES. YOUR RIGHTS AND OBLIGATIONS RELATED TO THE COMMERCIAL FEATURES ARE AS SET FORTH IN THE SUPPLEMENTAL TERMS ALONG WITH ADDITIONAL LICENSES FOR DEVELOPERS AND PUBLISHERS.
-
-3. RESTRICTIONS. Software is copyrighted. Title to Software and all associated intellectual property rights is retained by Oracle and/or its licensors. Unless enforcement is prohibited by applicable law, you may not modify, decompile, or reverse engineer Software. You acknowledge that the Software is developed for general use in a variety of information management applications; it is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use the Software in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle disclaims any express or implied warranty of fitness for such uses. No right, title or interest in or to any trademark, service mark, logo or trade name of Oracle or its licensors is granted under this Agreement. Additional restrictions for developers and/or publishers licenses are set forth in the Supplemental License Terms.
-
-4. DISCLAIMER OF WARRANTY. THE SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ORACLE FURTHER DISCLAIMS ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT.
-
-5. LIMITATION OF LIABILITY. IN NO EVENT SHALL ORACLE BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR DAMAGES FOR LOSS OF PROFITS, REVENUE, DATA OR DATA USE, INCURRED BY YOU OR ANY THIRD PARTY, WHETHER IN AN ACTION IN CONTRACT OR TORT, EVEN IF ORACLE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. ORACLE'S ENTIRE LIABILITY FOR DAMAGES HEREUNDER SHALL IN NO EVENT EXCEED ONE THOUSAND DOLLARS (U.S. $1,000).
-
-6. TERMINATION. This Agreement is effective until terminated. You may terminate this Agreement at any time by destroying all copies of Software. This Agreement will terminate immediately without notice from Oracle if you fail to comply with any provision of this Agreement. Either party may terminate this Agreement immediately should any Software become, or in either party's opinion be likely to become, the subject of a claim of infringement of any intellectual property right. Upon termination, you must destroy all copies of Software.
-
-7. EXPORT REGULATIONS. You agree that U.S. export control laws and other applicable export and import laws govern your use of the Software, including technical data; additional information can be found on Oracle's Global Trade Compliance web site (http://www.oracle.com/products/export). You agree that neither the Software nor any direct product thereof will be exported, directly, or indirectly, in violation of these laws, or will be used for any purpose prohibited by these laws including, without limitation, nuclear, chemical, or biological weapons proliferation.
-
-8. TRADEMARKS AND LOGOS. You acknowledge and agree as between you
-and Oracle that Oracle owns the ORACLE and JAVA trademarks and all ORACLE- and JAVA-related trademarks, service marks, logos and other brand
-designations ("Oracle Marks"), and you agree to comply with the Third
-Party Usage Guidelines for Oracle Trademarks currently located at
-http://www.oracle.com/us/legal/third-party-trademarks/index.html . Any use you make of the Oracle Marks inures to Oracle's benefit.
-
-9. U.S. GOVERNMENT LICENSE RIGHTS. If Software is being acquired by or on behalf of the U.S. Government or by a U.S. Government prime contractor or subcontractor (at any tier), then the Government's rights in Software and accompanying documentation shall be only those set forth in this Agreement.
-
-10. GOVERNING LAW. This agreement is governed by the substantive and procedural laws of California. You and Oracle agree to submit to the exclusive jurisdiction of, and venue in, the courts of San Francisco, or Santa Clara counties in California in any dispute arising out of or relating to this agreement.
-
-11. SEVERABILITY. If any provision of this Agreement is held to be unenforceable, this Agreement will remain in effect with the provision omitted, unless omission would frustrate the intent of the parties, in which case this Agreement will immediately terminate.
-
-12. INTEGRATION. This Agreement is the entire agreement between you and Oracle relating to its subject matter. It supersedes all prior or contemporaneous oral or written communications, proposals, representations and warranties and prevails over any conflicting or additional terms of any quote, order, acknowledgment, or other communication between the parties relating to its subject matter during the term of this Agreement. No modification of this Agreement will be binding, unless in writing and signed by an authorized representative of each party.
-
-SUPPLEMENTAL LICENSE TERMS
-
-These Supplemental License Terms add to or modify the terms of the Binary Code License Agreement. Capitalized terms not defined in these Supplemental Terms shall have the same meanings ascribed to them in the Binary Code License Agreement. These Supplemental Terms shall supersede any inconsistent or conflicting terms in the Binary Code License Agreement, or in any license contained within the Software.
-
-A. COMMERCIAL FEATURES. You may not use the Commercial Features for running Programs, Java applets or applications in your internal business operations or for any commercial or production purpose, or for any purpose other than as set forth in Sections B, C, D and E of these Supplemental Terms. If You want to use the Commercial Features for any purpose other than as permitted in this Agreement, You must obtain a separate license from Oracle.
-
-B. SOFTWARE INTERNAL USE FOR DEVELOPMENT LICENSE GRANT. Subject to the terms and conditions of this Agreement and restrictions and exceptions set forth in the README File incorporated herein by reference, including, but not limited to the Java Technology Restrictions of these Supplemental Terms, Oracle grants you a non-exclusive, non-transferable, limited license without fees to reproduce internally and use internally the Software complete and unmodified for the purpose of designing, developing, and testing your Programs.
-
-C. LICENSE TO DISTRIBUTE SOFTWARE. Subject to the terms and conditions of this Agreement and restrictions and exceptions set forth in the README File, including, but not limited to the Java Technology Restrictions and Limitations on Redistribution of these Supplemental Terms, Oracle grants you a non-exclusive, non-transferable, limited license without fees to reproduce and distribute the Software, provided that (i) you distribute the Software complete and unmodified and only bundled as part of, and for the sole purpose of running, your Programs, (ii) the Programs add significant and primary functionality to the Software, (iii) you do not distribute additional software intended to replace any component(s) of the Software, (iv) you do not remove or alter any proprietary legends or notices contained in the Software, (v) you only distribute the Software subject to a license agreement that: (a) is a complete, unmodified reproduction of this Agreement; or (b) protects Oracle's interests consistent with the terms contained in this Agreement and that includes the notice set forth in Section H, and (vi) you agree to defend and indemnify Oracle and its licensors from and against any damages, costs, liabilities, settlement amounts and/or expenses (including attorneys' fees) incurred in connection with any claim, lawsuit or action by any third party that arises or results from the use or distribution of any and all Programs and/or Software. The license set forth in this Section C does not extend to the Software identified in Section G.
-
-D. LICENSE TO DISTRIBUTE REDISTRIBUTABLES. Subject to the terms and conditions of this Agreement and restrictions and exceptions set forth in the README File, including but not limited to the Java Technology Restrictions and Limitations on Redistribution of these Supplemental Terms, Oracle grants you a non-exclusive, non-transferable, limited license without fees to reproduce and distribute those files specifically identified as redistributable in the README File ("Redistributables") provided that: (i) you distribute the Redistributables complete and unmodified, and only bundled as part of Programs, (ii) the Programs add significant and primary functionality to the Redistributables, (iii) you do not distribute additional software intended to supersede any component(s) of the Redistributables (unless otherwise specified in the applicable README File), (iv) you do not remove or alter any proprietary legends or notices contained in or on the Redistributables, (v) you only distribute the Redistributables pursuant to a license agreement that: (a) is a complete, unmodified reproduction of this Agreement; or (b) protects Oracle's interests consistent with the terms contained in the Agreement and includes the notice set forth in Section H, (vi) you agree to defend and indemnify Oracle and its licensors from and against any damages, costs, liabilities, settlement amounts and/or expenses (including attorneys' fees) incurred in connection with any claim, lawsuit or action by any third party that arises or results from the use or distribution of any and all Programs and/or Software. The license set forth in this Section D does not extend to the Software identified in Section G.
-
-E. DISTRIBUTION BY PUBLISHERS. This section pertains to your distribution of the JavaTM SE Development Kit Software (“JDK”) with your printed book or magazine (as those terms are commonly used in the industry) relating to Java technology ("Publication"). Subject to and conditioned upon your compliance with the restrictions and obligations contained in the Agreement, Oracle hereby grants to you a non-exclusive, nontransferable limited right to reproduce complete and unmodified copies of the JDK on electronic media (the "Media") for the sole purpose of inclusion and distribution with your Publication(s), subject to the following terms: (i) You may not distribute the JDK on a stand-alone basis; it must be distributed with your Publication(s); (ii) You are responsible for downloading the JDK from the applicable Oracle web site; (iii) You must refer to the JDK as JavaTM SE Development Kit; (iv) The JDK must be reproduced in its entirety and without any modification whatsoever (including with respect to all proprietary notices) and distributed with your Publication subject to a license agreement that is a complete, unmodified reproduction of this Agreement; (v) The Media label shall include the following information: “Copyright [YEAR], Oracle America, Inc. All rights reserved. Use is subject to license terms. ORACLE and JAVA trademarks and all ORACLE- and JAVA-related trademarks, service marks, logos and other brand designations are trademarks or registered trademarks of Oracle in the U.S. and other countries.” [YEAR] is the year of Oracle's release of the Software; the year information can typically be found in the Software’s “About” box or screen. This information must be placed on the Media label in such a manner as to only apply to the JDK; (vi) You must clearly identify the JDK as Oracle's product on the Media holder or Media label, and you may not state or imply that Oracle is responsible for any third-party software contained on the Media; (vii) You may not include any third party software on the Media which is intended to be a replacement or substitute for the JDK; (viii) You agree to defend and indemnify Oracle and its licensors from and against any damages, costs, liabilities, settlement amounts and/or expenses (including attorneys' fees) incurred in connection with any claim, lawsuit or action by any third party that arises or results from the use or distribution of the JDK and/or the Publication; ; and (ix) You shall provide Oracle with a written notice for each Publication; such notice shall include the following information: (1) title of Publication, (2) author(s), (3) date of Publication, and (4) ISBN or ISSN numbers. Such notice shall be sent to Oracle America, Inc., 500 Oracle Parkway, Redwood Shores, California 94065 U.S.A , Attention: General Counsel.
-
-F. JAVA TECHNOLOGY RESTRICTIONS. You may not create, modify, or change the behavior of, or authorize your licensees to create, modify, or change the behavior of, classes, interfaces, or subpackages that are in any way identified as "java", "javax", "sun", “oracle” or similar convention as specified by Oracle in any naming convention designation.
-
-G. LIMITATIONS ON REDISTRIBUTION. You may not redistribute or otherwise transfer patches, bug fixes or updates made available by Oracle through Oracle Premier Support, including those made available under Oracle's Java SE Support program.
-
-H. COMMERCIAL FEATURES NOTICE. For purpose of complying with Supplemental Term Section C.(v)(b) and D.(v)(b), your license agreement shall include the following notice, where the notice is displayed in a manner that anyone using the Software will see the notice:
-
-Use of the Commercial Features for any commercial or production purpose requires a separate license from Oracle. “Commercial Features” means those features identified Table 1-1 (Commercial Features In Java SE Product Editions) of the Java SE documentation accessible at http://www.oracle.com/technetwork/java/javase/documentation/index.html
-
- 
-
-I. SOURCE CODE. Software may contain source code that, unless expressly licensed for other purposes, is provided solely for reference purposes pursuant to the terms of this Agreement. Source code may not be redistributed unless expressly provided for in this Agreement.
-
-J. THIRD PARTY CODE. Additional copyright notices and license terms applicable to portions of the Software are set forth in the THIRDPARTYLICENSEREADME file accessible at http://www.oracle.com/technetwork/java/javase/documentation/index.html. In addition to any terms and conditions of any third party opensource/freeware license identified in the THIRDPARTYLICENSEREADME file, the disclaimer of warranty and limitation of liability provisions in paragraphs 4 and 5 of the Binary Code License Agreement shall apply to all Software in this distribution.
-
-K. TERMINATION FOR INFRINGEMENT. Either party may terminate this Agreement immediately should any Software become, or in either party's opinion be likely to become, the subject of a claim of infringement of any intellectual property right.
-
-L. INSTALLATION AND AUTO-UPDATE. The Software's installation and auto-update processes transmit a limited amount of data to Oracle (or its service provider) about those specific processes to help Oracle understand and optimize them. Oracle does not associate the data with personally identifiable information. You can find more information about the data Oracle collects as a result of your Software download at http://www.oracle.com/technetwork/java/javase/documentation/index.html.
-
-For inquiries please contact: Oracle America, Inc., 500 Oracle Parkway,
-
-Redwood Shores, California 94065, USA.
-
-Last updated 02 April 2013
-

export-image/prerun.sh

@@ -1,65 +1,61 @@
 #!/bin/bash -e
 
-IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
 
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-rm -f ${IMG_FILE}
+rm -f "${IMG_FILE}"
 
-rm -rf ${ROOTFS_DIR}
+rm -rf "${ROOTFS_DIR}"
-mkdir -p ${ROOTFS_DIR}
+mkdir -p "${ROOTFS_DIR}"
 
-BOOT_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR}/boot --block-size=1 | cut -f 1)
+BOOT_SIZE="$((256 * 1024 * 1024))"
-TOTAL_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR} --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
+ROOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot --block-size=1 | cut -f 1)
 
-IMG_SIZE=$((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024)))
+# All partition sizes and starts will be aligned to this size
+ALIGN="$((4 * 1024 * 1024))"
+# Add this much space to the calculated file size. This allows for
+# some overhead (since actual space usage is usually rounded up to the
+# filesystem block size) and gives some free space on the resulting
+# image.
+ROOT_MARGIN=$((800*1024*1024))
 
-truncate -s ${IMG_SIZE} ${IMG_FILE}
+BOOT_PART_START=$((ALIGN))
-fdisk -H 255 -S 63 ${IMG_FILE} <<EOF
+BOOT_PART_SIZE=$(((BOOT_SIZE + ALIGN - 1) / ALIGN * ALIGN))
-o
+ROOT_PART_START=$((BOOT_PART_START + BOOT_PART_SIZE))
-n
+ROOT_PART_SIZE=$(((ROOT_SIZE + ROOT_MARGIN + ALIGN  - 1) / ALIGN * ALIGN))
+IMG_SIZE=$((BOOT_PART_START + BOOT_PART_SIZE + ROOT_PART_SIZE))
 
+truncate -s "${IMG_SIZE}" "${IMG_FILE}"
 
-8192
+parted --script "${IMG_FILE}" mklabel msdos
-+$((BOOT_SIZE * 2 /512))
+parted --script "${IMG_FILE}" unit B mkpart primary fat32 "${BOOT_PART_START}" "$((BOOT_PART_START + BOOT_PART_SIZE - 1))"
-p
+parted --script "${IMG_FILE}" unit B mkpart primary ext4 "${ROOT_PART_START}" "$((ROOT_PART_START + ROOT_PART_SIZE - 1))"
-t
-c
-n
 
+PARTED_OUT=$(parted -sm "${IMG_FILE}" unit b print)
+BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 2 | tr -d B)
+BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 4 | tr -d B)
 
-8192
+ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 2 | tr -d B)
+ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 4 | tr -d B)
 
-
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-p
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
-w
-EOF
-
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
-BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
-| cut -d" " -f 2 | tr -d B)
-BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
-| cut -d" " -f 4 | tr -d B)
-
-ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
-| cut -d" " -f 2 | tr -d B)
-ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
-| cut -d" " -f 4 | tr -d B)
-
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
 ROOT_FEATURES="^huge_file"
-if grep -q "metadata_csum" /etc/mke2fs.conf; then
+for FEATURE in metadata_csum 64bit; do
-    ROOT_FEATURES="^metadata_csum,$ROOT_FEATURES"
+	if grep -q "$FEATURE" /etc/mke2fs.conf; then
-fi
+	    ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
-mkdosfs -n boot -F 32 -v $BOOT_DEV > /dev/null
+	fi
-mkfs.ext4 -O $ROOT_FEATURES $ROOT_DEV > /dev/null
+done
+mkdosfs -n boot -F 32 -v "$BOOT_DEV" > /dev/null
+mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
 
-mount -v $ROOT_DEV ${ROOTFS_DIR} -t ext4
+mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
-mkdir -p ${ROOTFS_DIR}/boot
+mkdir -p "${ROOTFS_DIR}/boot"
-mount -v $BOOT_DEV ${ROOTFS_DIR}/boot -t vfat
+mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot" -t vfat
 
-rsync -aHAXx --exclude var/cache/apt/archives ${EXPORT_ROOTFS_DIR}/ ${ROOTFS_DIR}/
+rsync -aHAXx --exclude /var/cache/apt/archives --exclude /boot "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"
+rsync -rtx "${EXPORT_ROOTFS_DIR}/boot/" "${ROOTFS_DIR}/boot/"

export-noobs/00-release/00-run.sh

@@ -2,35 +2,41 @@
 
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
 
-install -v -m 744	files/partition_setup.sh	${NOOBS_DIR}/
+install -v -m 744	files/partition_setup.sh	"${NOOBS_DIR}/"
-install -v		files/partitions.json		${NOOBS_DIR}/
+install -v		files/partitions.json		"${NOOBS_DIR}/"
-install -v		files/os.json			${NOOBS_DIR}/
+install -v		files/os.json			"${NOOBS_DIR}/"
-install -v		files/OS.png			${NOOBS_DIR}/
+install -v		files/OS.png			"${NOOBS_DIR}/"
-install -v		files/release_notes.txt		${NOOBS_DIR}/
+install -v		files/release_notes.txt		"${NOOBS_DIR}/"
 
-tar -v -c -C		files/marketing			-f ${NOOBS_DIR}/marketing.tar .
+tar -v -c -C		files/marketing			-f "${NOOBS_DIR}/marketing.tar" .
 
-BOOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/boot.tar.xz  | grep totals | cut -f 5)
+BOOT_SHASUM="$(sha256sum "${NOOBS_DIR}/boot.tar.xz" | cut -f1 -d' ')"
-ROOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/root.tar.xz  | grep totals | cut -f 5)
+ROOT_SHASUM="$(sha256sum "${NOOBS_DIR}/root.tar.xz" | cut -f1 -d' ')"
 
-BOOT_SIZE=$(expr ${BOOT_SIZE} / 1000000 \+ 1)
+BOOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/boot.tar.xz"  | grep totals | cut -f 5)"
-ROOT_SIZE=$(expr ${ROOT_SIZE} / 1000000 \+ 1)
+ROOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/root.tar.xz"  | grep totals | cut -f 5)"
 
-BOOT_NOM=$(expr ${BOOT_SIZE} \* 3)
+BOOT_SIZE="$(( BOOT_SIZE / 1024 / 1024 + 1))"
-ROOT_NOM=$(expr ${ROOT_SIZE} \+ 400)
+ROOT_SIZE="$(( ROOT_SIZE / 1024 / 1024 + 1))"
 
-mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/$(echo ${NOOBS_NAME} | sed 's/ /_/g').png"
+BOOT_NOM="256"
+ROOT_NOM="$(( ROOT_SIZE + 400 ))"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
+mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/${NOOBS_NAME// /_}.png"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_NOM|${BOOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_SHASUM|${BOOT_SHASUM}|"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_NOM|${ROOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_SHASUM|${ROOT_SHASUM}|"
 
-sed ${NOOBS_DIR}/os.json -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
 
-sed ${NOOBS_DIR}/release_notes.txt -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_NOM|${BOOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_NOM|${ROOT_NOM}|"
 
-cp -a ${NOOBS_DIR} ${DEPLOY_DIR}/
+sed "${NOOBS_DIR}/os.json" -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
+
+sed "${NOOBS_DIR}/release_notes.txt" -i -e "s|UNRELEASED|${IMG_DATE}|"
+
+cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"

export-noobs/00-release/files/os.json

@@ -1,7 +1,7 @@
 {
     "description": "NOOBS_DESCRIPTION",
     "feature_level": 35120124,
-    "kernel": "4.9",
+    "kernel": "4.19",
     "name": "NOOBS_NAME",
     "password": "raspberry",
     "release_date": "UNRELEASED",
@@ -10,9 +10,11 @@
         "Pi Model",
         "Pi 2",
         "Pi Zero",
-        "Pi 3"
+        "Pi 3",
+        "Pi Compute Module 3",
+        "Pi 4"
     ],
     "url": "http://www.raspbian.org/",
     "username": "pi",
-    "version": "jessie"
+    "version": "buster"
 }

export-noobs/00-release/files/partition_setup.sh

@@ -1,7 +1,9 @@
 #!/bin/sh
+#supports_backup in PINN
 
 set -ex
 
+# shellcheck disable=SC2154
 if [ -z "$part1" ] || [ -z "$part2" ]; then
   printf "Error: missing environment variable part1 or part2\n" 1>&2
   exit 1
@@ -13,23 +15,31 @@ mount "$part1" /tmp/1
 mount "$part2" /tmp/2
 
 sed /tmp/1/cmdline.txt -i -e "s|root=[^ ]*|root=${part2}|"
-sed /tmp/2/etc/fstab -i -e "s|^.* / |${part2}  / |"
+sed /tmp/2/etc/fstab -i -e "s|^[^#].* / |${part2}  / |"
-sed /tmp/2/etc/fstab -i -e "s|^.* /boot |${part1}  /boot |"
+sed /tmp/2/etc/fstab -i -e "s|^[^#].* /boot |${part1}  /boot |"
 
-if [ -f /mnt/ssh ]; then
+# shellcheck disable=SC2154
+if [ -z "$restore" ]; then
+  if [ -f /mnt/ssh ]; then
     cp /mnt/ssh /tmp/1/
-fi
+  fi
 
-if [ -f /mnt/ssh.txt ]; then
+  if [ -f /mnt/ssh.txt ]; then
     cp /mnt/ssh.txt /tmp/1/
-fi
+  fi
 
-if [ -f /settings/wpa_supplicant.conf ]; then
+  if [ -f /settings/wpa_supplicant.conf ]; then
     cp /settings/wpa_supplicant.conf /tmp/1/
-fi
+  fi
 
-if ! grep -q resize /proc/cmdline; then
+  if ! grep -q resize /proc/cmdline; then
-  sed -i 's| quiet init=/usr/lib/raspi-config/init_resize.sh||' /tmp/1/cmdline.txt
+    if ! grep -q splash /tmp/1/cmdline.txt; then
+      sed -i "s| quiet||g" /tmp/1/cmdline.txt
+    fi
+    sed -i 's| init=/usr/lib/raspi-config/init_resize.sh||' /tmp/1/cmdline.txt
+  else
+    sed -i '1 s|.*|& sdhci.debug_quirks2=4|' /tmp/1/cmdline.txt
+  fi
 fi
 
 umount /tmp/1

export-noobs/00-release/files/partitions.json

@@ -6,7 +6,8 @@
             "mkfs_options": "-F 32",
             "partition_size_nominal": BOOT_NOM,
             "uncompressed_tarball_size": BOOT_SIZE,
-            "want_maximised": false
+            "want_maximised": false,
+            "sha256sum": "BOOT_SHASUM"
         },
         {
             "filesystem_type": "ext4",
@@ -14,7 +15,8 @@
             "mkfs_options": "-O ^huge_file",
             "partition_size_nominal": ROOT_NOM,
             "uncompressed_tarball_size": ROOT_SIZE,
-            "want_maximised": true
+            "want_maximised": true,
+            "sha256sum": "ROOT_SHASUM"
         }
     ]
 }

export-noobs/00-release/files/release_notes.txt

@@ -1,5 +1,254 @@
 UNRELEASED:
-  * 
+  * rpi-eeprom included
+    - This will automatically update the SPI EEPROM on the Raspberry Pi 4 to the latest stable version.
+       See https://rpf.io/eeprom for more information.
+  * New icon theme for file manager icons
+  * Appearance Settings - option for identical desktop on both monitors
+  * Appearance Settings - option to show different desktop icons on both monitors
+  * Taskbar automatically moved to monitor 0 if monitor 1 not found at boot
+  * Switching of audio output between two HDMI devices added to volume plugin
+  * Switching of audio input devices added to volume plugin
+  * .asoundrc (ALSA config file) now uses 'plug' values to support more devices
+  * Audio Settings tool modified to integrate more closely with volume plugin to reduce duplicated code
+  * Screen Configuration tool now shows separate menus for resolution and refresh rate
+  * Primary and active monitor settings removed from Screen Configuration tool
+  * Overscan support added for FKMS driver
+  * New keyboard shortcuts added - Ctrl-Alt-End brings up shutdown menu; Ctrl-Alt-M moves taskbar between monitors
+  * Latest changes to Bluez ALSA interface integrated to improve connection to Bluetooth audio devices
+  * Mousepad used as simple text editor instead of leafpad
+  * Version 3.2 of Thonny added
+  * Version 74 of Chromium added
+  * Version 3.0.8 of VLC added
+  * Version 32.0.0.255 of Flash player added
+  * Version 6.5.0 of RealVNC Server added
+  * Version 6.19.715 of RealVNC Viewer added (full image only)
+  * Version 12.0.1 of Mathematica added (full image only)
+  * Version 0.20.8 of NodeRED added (full image only)
+  * Version 3.1.0 of Sonic Pi added (full image only)
+  * Scratch 3 added (full image only)
+  * Bug fix - URL handling in Terminal
+  * Bug fix - octal values in SSIDs in network plugin
+  * Bug fix - remaining value in progress bar when transferring files
+  * Bug fix - integration of xarchiver tool with file manager
+  * Bug fix - start menu opening on incorrect monitor
+  * Bug fix - minimised applications wrongly displayed on taskbar on second monitor
+  * Bug fix - Bluetooth icon disappearing on x86 platforms when Bluetooth turned off
+  * Bug fix - Screen Configuration tool not shown on x86 platforms and settings not being saved
+  * Various translation updates
+  * Various minor bug fixes
+  * Epiphany/Web removed
+  * ntfs-3g included
+  * pciutils added
+  * Linux kernel 4.19.75
+  * Raspberry Pi firmware 01508e81ec1e918448227ca864616d56c430b46d
+2019-07-10:
+  * Clearer options for switching of Pi 4 video output in Raspberry Pi Configuration
+  * Option added to Appearance Settings to move taskbar to second monitor
+  * Option added to Recommended Software to restrict package installs by architecture
+  * New version of Adobe Flash player (32.0.0.223)
+  * Selection of screen refresh rates added to Screen Configuration
+  * Fix for missing text insertion cursor in LibreOffice on Pi 4
+  * Fix for Wi-fi interruption when Wi-fi icon on taskbar is clicked
+  * FIx for incorrect desktop background behind desktop login prompt
+  * Fix for segmentation faults when launching obconf and lxapperarance
+  * Fix for unclosed file pointer in Screen Configuration
+  * Fix for Bluetooth plugin freeze when large numbers of devices detected
+  * Fix for opening URLs not working in lxterminal
+  * Fix for start menu opening on incorrect monitor when launched from keyboard
+  * Fix for taskbar item not having [] removed when un-minimising on second monitor
+  * Fix for Chromium video playback and WebGL performance on Pi 4
+  * Remove 4kp60 option from Raspberry Pi Configuration
+  * Rename hdmi_enable_4k to hdmi_enable_4kp60 in /boot/config.txt and raspi-config
+  * Linux kernel 4.19.57
+  * Raspberry Pi firmware 356f5c2880a3c7e8774025aa6fc934a617553e7b
+2019-06-20:
+  * Based on Debian Buster
+  * Support for Raspberry Pi 4 hardware
+  * FKMS OpenGL desktop graphics driver and xcompmgr compositing window manager used when running on Raspberry Pi 4
+  * Screen Configuration application added for use with FKMS driver
+  * Raspberry Pi 4 video output options added to Raspberry Pi Configuration
+  * Uses new PiXflat UI theme for GTK and Openbox
+  * CPU activity gauge plugin no longer shown on taskbar by default
+  * CPU temperature gauge plugin added (not shown by default)
+  * USB ejecter and Bluetooth taskbar icons hidden when not appropriate
+  * Version 74.0.3729.157 of Chromium web browser included
+  * Version 32.0.0.207 of Flash player included
+  * IDLE Python IDE removed
+  * Wolfram Mathematica removed temporarily due to incompatibility with Buster
+  * Display of package sizes removed from Recommended Software
+  * Appearance Settings modified to support independent settings for two monitors
+  * Oracle Java 7 and 8 replaced with OpenJDK 11
+  * Miscellaneous small bug fixes
+  * On-board 5GHz WiFi blocked by rfkill by default
+    The block is removed when taking one of the following actions:
+    - Selecting a locale in the first run wizard
+    - Setting the WiFi country in the Raspberry Pi Configuration tool or the Network Settings applet
+    - Setting the WiFi country in raspi-config
+    - Providing a wpa_supplicant.conf file through the boot partition
+    - Running 'rfkill unblock wifi'
+  * Boot partition size set to 256M
+  * Linux kernel 4.19.50
+  * Raspberry Pi firmware 88ca9081f5e51cdedd16d5dbc85ed12a25123201
+2019-04-08:
+  * Chromium browser updated to version 72
+  * VLC media player updated to version 3.0.6
+  * RealVNC Server updated to version 6.4.0
+  * Flash player updated to version 32.0.0.156
+  * Performance improvements to SDL library
+  * Performance improvements to pixman library
+  * Option to set display underscan added to startup wizard
+  * Mounted external drives now displayed on desktop by default
+  * Network plugin modified for improved compatibility with wpa_passphrase
+  * SD Card Copier tweaks to reduce copy failures
+  * Various minor bug fixes and appearance tweaks
+  * Added ethtool
+  * Added rng-tools
+  * Add PINN restore support
+  * Linux kernel 4.14.98
+  * Raspberry Pi firmware f8939644f7bd3065068787f1f92b3f3c79cf3de9
+2018-11-13:
+  * Two versions of image created - "base" image has no optional software packages included; "full" image has all optional packages
+    - Removed from "base" image - LibreOffice, Thonny, Scratch, Scratch 2, Sonic Pi, Minecraft, Python Games, SmartSim, SenseHAT Emulator
+    - Added to "full" image - Mathematica, BlueJ, Greenfoot, Node-RED, Claws Mail, VNC Viewer
+  * Python Games and SmartSim added to Recommended Software
+  * VLC media player with VideoCore hardware acceleration included in image
+  * Version 3.0.5 of Thonny included
+  * Modifications to LXDE components to enable local configuration to override global configuration correctly
+  * Modifications to Appearance Settings to support above configuration changes
+  * Modifications to various initial config defaults and relevant package to support above configuration changes
+  * Selecting default option in Appearance Settings now deletes relevant local configuration files
+  * PiX theme modified so that all changes made in Appearance Settings are in override files rather than in theme files
+  * Design of scrollbar buttons changed
+  * Image Viewer moved into Graphics category on main menu
+  * Recommended Software now installs LibreOffice language support files if needed, and suggests reboot if needed
+  * Latest version of Pepper Flash plugin included
+  * Chromium h264ify plugin permissions set correctly by default
+  * Corrections to various MIME types so that files open in sensible default applications
+  * Set default timezone to 'Europe/London'
+  * Linux kernel 4.14.79
+  * Raspberry Pi firmware 12e0bf86e08d6067372bc0a45d7e8a10d3113210
+2018-10-09:
+  * Raspberry Pi 3A+ support
+  * In startup wizard, assign keyboard to country as per Debian installer recommendations
+  * In startup wizard, add option to use US keyboard in preference to country-specific option
+  * In startup wizard, show IP address on first page
+  * In startup wizard, check for existing wifi network connection and show it if there is one
+  * In startup wizard, install language support packages for LibreOffice and other applications
+  * In startup wizard, improve operation with keyboard only and no mouse
+  * Password change in Raspberry Pi Configuration and startup wizard now works properly if passwords contain shell characters
+  * Battery indicator plugin modified to cope with Pi-top hardware monitor crashing
+  * Networking plugin hides wifi password characters by default
+  * In Scratch 2 GPIO plugin, set pin from dropdown list rather than free text
+  * In Scratch 2 SenseHAT plugin, swap x and y axis values for LED array
+  * Include latest Adobe Flash player (31.0.0.108)
+  * Include latest RealVNC Server (6.3.1)
+  * Include libav-tools
+  * Include ssh-import-id
+  * Removed Mathematica
+  * Merge in latest third-party code for Bluetooth ALSA interface
+  * Add ability to prevent software update changing configuration files, by creating ~/.config/.lock file
+  * Various other small bug fixes, tweaks and changes to text
+  * Make dhcpcd work with 3G devices
+  * Add hw acceleration to ffmpeg
+  * Improved WiFi-BT coexistence parameters
+  * Run fake-hwclock before systemd-fsck-root
+  * Raspberry Pi PoE HAT support
+  * Linux kernel 4.14.71
+  * Raspberry Pi firmware 5b49caa17e91d0e64024380119ad739bb201c674
+2018-06-27:
+  * New first-boot configuration wizard added
+  * Recommended Software installer added
+  * Bluej, Greenfoot, NodeRED, Claws Mail, VNC Viewer removed from image - can now be installed from Recommended Applications
+  * Qpdfview PDF viewer installed instead of Xpdf
+  * Version 65.0 of Chromium browser included, with latest Flash player
+  * Volume up / down keys now change by 5% increments and affect currently-selected output device rather than internal device only
+  * Network plugin now remembers previously-entered WiFi network passwords when prompting for reconnection
+  * Serial port and serial console can now be switched separately in Raspberry Pi Configuration
+  * Lxkeymap keyboard language setting application removed - replaced with dialog within Raspberry Pi Configuration
+  * Wifi country and keyboard language setting dialogs in Raspberry Pi Configuration now callable from other applications
+  * New version of Piboto font included to render with correct weight under some rogue applications
+  * Reconnection to Bluetooth audio devices on reboot improved
+  * Disable click-to-rename behaviour in file manager if single-click selection enabled
+  * Appearance Settings dialog makes config changes to some Qt files to match selected theme
+  * MIME file type associations improved
+  * Multiple desktop management options removed from mouse middle-click menu
+  * Menu shortcuts to Raspberry Pi website amended
+  * Python 2 IDLE menu link removed
+  * Sample Magpi PDF installed in /home/pi/MagPi
+  * Various minor tweaks, bug fixes and appearance changes
+  * Bluetooth updates
+    - Firmware with Bluetooth 4.2 features
+    - SCO profile suppot added via bthelper.service
+  * Linux kernel 4.14.50+
+  * Raspberry Pi firmware 748fb17992426bb29d99224b93cb962fefbdc833
+2018-04-18:
+  * Fixed race between wifi-country.service and raspberrypi-net-mods.service
+  * Linux kernel 4.14.34+
+  * Raspberry Pi firmware 5db8e4e1c63178e200d6fbea23ed4a9bf4656658
+2018-03-13:
+  * Raspberry Pi 3 B+ support
+  * WiFi is disabled until wireless regulatory domain is set (Pi 3 B+ only)
+    - The domain can be done through 'Raspberry Pi Configuration' (rc_gui),
+      'raspi-config' or by setting 'country=' to an appropriate ISO 3166
+      alpha2 country code in /etc/wpa_supplicant/wpa_supplicant.conf.
+  * Default wireless regulatory domain is now unset
+  * Added support to desktop for different screen sizes and resolutions,
+    including multiple preset options in Appearance Settings and pixel doubling
+    option in Raspberry Pi Configuration
+  * Version 2.1.16 of Thonny included
+  * Version 29.0.0.113 of Adobe PepperFlash player included
+  * Version 1.2.post1 of Pygame Zero included
+  * Bluetooth plugin now supports connection to Bluetooth LE HID devices
+  * Network plugin now indicates 5G-compatible APs
+  * Latest changes to Bluez ALSA service merged
+    - service now started on CLI boot as well as GUI boot
+  * Latest changes to dhcpcd networking plugin merged
+  * Improved support for running on pi-top devices
+  * Small design changes to PiX theme and icons
+  * Bug fix - hide spurious window resize handles
+  * Bug fix - Scratch 2 remote GPIO state block now works correctly
+  * Updated WiFi Firmware
+    - brcmfmac43455-sdio 7.45.154
+    - brcmfmac43430-sdio 7.45.98.38
+  * New packages:
+    - policykit-1
+    - obconf
+    - python-buttonshim python3-buttonshim
+    - python-unicornhathd  python3-unicornhathd
+    - python-pantilthat python3-pantilthat
+  * Linux kernel 4.9.80+
+  * Raspberry Pi firmware 3347884c7df574bbabeff6dca63caf686e629699
+2017-11-29:
+  * Added battery monitor plugin for taskbar - works on x86 images or first-generation Pi-Top
+  * Added cutdown mode to PCManFM file manager to reduce complexity
+  * Added ability to rename files in PCManFM by clicking name when selected
+  * Bug fix in Bluetooth ALSA module to reduce truncation of audio at end of playback
+  * Various small tweaks, bug fixes and theme modifications
+  * New kernel and firmware
+2017-09-07:
+  * Disable predictable network interface names for Ethernet devices
+  * Bug fix for keyboard settings dialog in Raspberry Pi Configuration
+  * Bug fix for crash on some videos and animations in Chromium
+  * Bug fix for taskbar crash when running RealVNC server
+  * Bug fix for reloading projects with extensions in Scratch 2
+  * Bug fix for MAC address problem in Bluetooth
+  * Simple mode and new icons in Thonny
+  * New Japanese translations in Raspberry Pi Configuration
+  * Install fonts-droid-fallback for international fonts
+2017-08-16:
+  * Based on Raspbian Stretch (Debian version 9)
+  * Version 60 of Chromium browser included
+  * Version 3.0.1 of Sonic Pi included
+  * Version 6.1.1 of RealVNC included
+  * Version 0.17.4 of NodeRED included
+  * Bluetooth audio routed via ALSA rather than Pulseaudio
+  * SenseHAT extension added to Scratch 2
+  * Various desktop applications modified to prompt for sudo password if needed
+  * lxinput control options for mouse speed simplified
+  * lxpanel plugins moved into separate packages
+  * Wireless firmware for Pi 3 and Pi 0W modified to address Broadpwn exploit
+  * Latest kernel and firmware
+  * Various small tweaks, bug fixes and theme modifications
 2017-07-05:
   * New kernel and firmware
   * Filesystem created without the metadata_csum feature

export-noobs/prerun.sh

@@ -1,38 +1,36 @@
 #!/bin/bash -e
 
-IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-mkdir -p ${STAGE_WORK_DIR}
+mkdir -p "${STAGE_WORK_DIR}"
-cp ${WORK_DIR}/export-image/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img ${STAGE_WORK_DIR}/
+cp "${WORK_DIR}/export-image/${IMG_FILENAME}${IMG_SUFFIX}.img" "${STAGE_WORK_DIR}/"
 
-rm -rf ${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}
+rm -rf "${NOOBS_DIR}"
 
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
+PARTED_OUT=$(parted -sm "${IMG_FILE}" unit b print)
-BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
+BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 2 | tr -d B)
-| cut -d" " -f 2 | tr -d B)
+BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 4 | tr -d B)
-BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
-| cut -d" " -f 4 | tr -d B)
 
-ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
+ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 2 | tr -d B)
-| cut -d" " -f 2 | tr -d B)
+ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 4 | tr -d B)
-ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
-| cut -d" " -f 4 | tr -d B)
 
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
-mkdir -p ${STAGE_WORK_DIR}/rootfs
+mkdir -p "${STAGE_WORK_DIR}/rootfs"
-mkdir -p ${NOOBS_DIR}
+mkdir -p "${NOOBS_DIR}"
 
-mount $ROOT_DEV ${STAGE_WORK_DIR}/rootfs
+mount "$ROOT_DEV" "${STAGE_WORK_DIR}/rootfs"
-mount $BOOT_DEV ${STAGE_WORK_DIR}/rootfs/boot
+mount "$BOOT_DEV" "${STAGE_WORK_DIR}/rootfs/boot"
 
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs/boot -cpf ${NOOBS_DIR}/boot.tar.xz .
+ln -sv "/lib/systemd/system/apply_noobs_os_config.service" "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
-umount ${STAGE_WORK_DIR}/rootfs/boot
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs --one-file-system -cpf ${NOOBS_DIR}/root.tar.xz .
 
-unmount_image ${IMG_FILE}
+bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs/boot" -cpf - . | xz -T0 > "${NOOBS_DIR}/boot.tar.xz"
+umount "${STAGE_WORK_DIR}/rootfs/boot"
+bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs" --one-file-system -cpf - . | xz -T0 > "${NOOBS_DIR}/root.tar.xz"
+
+unmount_image "${IMG_FILE}"

scripts/common

@@ -1,24 +1,28 @@
 log (){
-	date +"[%T] $@" | tee -a "${LOG_FILE}"
+	date +"[%T] $*" | tee -a "${LOG_FILE}"
 }
 export -f log
 
 bootstrap(){
-	local ARCH
+	local BOOTSTRAP_CMD=debootstrap
-	ARCH=$(dpkg --print-architecture)
+	local BOOTSTRAP_ARGS=()
 
 	export http_proxy=${APT_PROXY}
 
-	if [ "$ARCH" !=  "armhf" ]; then
+	if [ "$(dpkg --print-architecture)" !=  "armhf" ] && [ "$(dpkg --print-architecture)" !=  "aarch64" ]; then
-		local BOOTSTRAP_CMD=qemu-debootstrap
+		BOOTSTRAP_CMD=qemu-debootstrap
-	else
-		local BOOTSTRAP_CMD=debootstrap
 	fi
 
-	capsh --drop=cap_setfcap -- -c "${BOOTSTRAP_CMD} --components=main,contrib,non-free \
+	BOOTSTRAP_ARGS+=(--arch armhf)
-		--arch armhf \
+	BOOTSTRAP_ARGS+=(--components "main,contrib,non-free")
-		--keyring "${STAGE_DIR}/files/raspberrypi.gpg" \
+	BOOTSTRAP_ARGS+=(--keyring "${STAGE_DIR}/files/raspberrypi.gpg")
-		$1 $2 $3" || rmdir "$2/debootstrap"
+	BOOTSTRAP_ARGS+=("$@")
+
+	setarch linux32 capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" "${BOOTSTRAP_ARGS[@]}" || true
+
+	if [ -d "$2/debootstrap" ]; then
+		rmdir "$2/debootstrap"
+	fi
 }
 export -f bootstrap
 
@@ -53,7 +57,7 @@ unmount_image(){
 	sync
 	sleep 1
 	local LOOP_DEVICES
-	LOOP_DEVICES=$(losetup -j "${1}" | cut -f1 -d':')
+	LOOP_DEVICES=$(losetup --list | grep "$(basename "${1}")" | cut -f1 -d' ')
 	for LOOP_DEV in ${LOOP_DEVICES}; do
 		if [ -n "${LOOP_DEV}" ]; then
 			local MOUNTED_DIR
@@ -85,14 +89,11 @@ on_chroot() {
 		mount --bind /sys "${ROOTFS_DIR}/sys"
 	fi
 
-	capsh --drop=cap_setfcap "--chroot=${ROOTFS_DIR}/" -- "$@"
+	setarch linux32 capsh --drop=cap_setfcap "--chroot=${ROOTFS_DIR}/" -- -e "$@"
 }
 export -f on_chroot
 
 update_issue() {
-	local GIT_HASH
+	echo -e "Raspberry Pi reference ${IMG_DATE}\nGenerated using ${PI_GEN}, ${PI_GEN_REPO}, ${GIT_HASH}, ${1}" > "${ROOTFS_DIR}/etc/rpi-issue"
-	GIT_HASH=$(git rev-parse HEAD)
-	echo -e "Raspberry Pi reference ${IMG_DATE}\nGenerated using pi-gen, https://github.com/RPi-Distro/pi-gen, ${GIT_HASH}, ${1}" > "${ROOTFS_DIR}/etc/rpi-issue"
 }
 export -f update_issue
-

scripts/dependencies_check

@@ -10,11 +10,11 @@ dependencies_check()
 
 	for depfile in "$@"; do
 		if [[ -e "$depfile" ]]; then
-			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${BASE_DIR}/depends)"
+			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${BASE_DIR}/depends")"
 
 		fi
 		for dep in $deps; do
-			if ! hash ${dep%:*} 2>/dev/null; then
+			if ! hash "${dep%:*}" 2>/dev/null; then
 				missing="${missing:+$missing }${dep#*:}"
 			fi
 		done
@@ -27,4 +27,12 @@ dependencies_check()
 		echo "$missing"
 		false
 	fi
+
+
+	if ! grep -q "/proc/sys/fs/binfmt_misc" /proc/mounts; then
+		echo "Module binfmt_misc not loaded in host"
+		echo "Please run:"
+		echo "  sudo modprobe binfmt_misc"
+		exit 1
+	fi
 }

stage0/00-configure-apt/00-run.sh

@@ -1,13 +1,13 @@
 #!/bin/bash -e
 
-install -m 644 files/sources.list ${ROOTFS_DIR}/etc/apt/
+install -m 644 files/sources.list "${ROOTFS_DIR}/etc/apt/"
-install -m 644 files/raspi.list ${ROOTFS_DIR}/etc/apt/sources.list.d/
+install -m 644 files/raspi.list "${ROOTFS_DIR}/etc/apt/sources.list.d/"
 
 if [ -n "$APT_PROXY" ]; then
-	install -m 644 files/51cache ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	install -m 644 files/51cache "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-	sed ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache -i -e "s|APT_PROXY|${APT_PROXY}|"
+	sed "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache" -i -e "s|APT_PROXY|${APT_PROXY}|"
 else
-	rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
 fi
 
 on_chroot apt-key add - < files/raspberrypi.gpg.key

stage0/00-configure-apt/files/raspi.list

@@ -1,3 +1,3 @@
-deb http://archive.raspberrypi.org/debian/ jessie main ui staging
+deb http://archive.raspberrypi.org/debian/ buster main
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspberrypi.org/debian/ jessie main ui
+#deb-src http://archive.raspberrypi.org/debian/ buster main

stage0/00-configure-apt/files/sources.list

@@ -1,3 +1,3 @@
-deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
+deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi
+#deb-src http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

stage0/02-firmware/01-packages

@@ -1 +1,2 @@
 raspberrypi-bootloader
+raspberrypi-kernel

stage0/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
-	bootstrap jessie ${ROOTFS_DIR} http://mirrordirector.raspbian.org/raspbian/
+	bootstrap buster "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
 fi

stage1/00-boot-files/00-run.sh

@@ -1,4 +1,4 @@
 #!/bin/bash -e
 
-install -m 644 files/cmdline.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/cmdline.txt "${ROOTFS_DIR}/boot/"
-install -m 644 files/config.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/config.txt "${ROOTFS_DIR}/boot/"

stage1/00-boot-files/files/cmdline.txt

@@ -1 +1 @@
-dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
+console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

stage1/00-boot-files/files/config.txt

@@ -47,10 +47,19 @@
 #dtparam=i2s=on
 #dtparam=spi=on
 
-# Uncomment this to enable the lirc-rpi module
+# Uncomment this to enable infrared communication.
-#dtoverlay=lirc-rpi
+#dtoverlay=gpio-ir,gpio_pin=17
+#dtoverlay=gpio-ir-tx,gpio_pin=18
 
 # Additional overlays and parameters are documented /boot/overlays/README
 
 # Enable audio (loads snd_bcm2835)
 dtparam=audio=on
+
+[pi4]
+# Enable DRM VC4 V3D driver on top of the dispmanx display stack
+dtoverlay=vc4-fkms-v3d
+max_framebuffers=2
+
+[all]
+#dtoverlay=vc4-fkms-v3d

stage1/01-sys-tweaks/00-run.sh

@@ -1,15 +1,14 @@
 #!/bin/bash -e
 
-install -d ${ROOTFS_DIR}/etc/systemd/system/[email protected]
+install -d "${ROOTFS_DIR}/etc/systemd/system/[email protected]"
-install -m 644 files/noclear.conf ${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf
+install -m 644 files/noclear.conf "${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf"
-install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/policy-rc.d #TODO: Necessary in systemd?
+install -v -m 644 files/fstab "${ROOTFS_DIR}/etc/fstab"
-install -v -m 644 files/fstab ${ROOTFS_DIR}/etc/fstab
 
 on_chroot << EOF
-if ! id -u pi >/dev/null 2>&1; then
+if ! id -u ${FIRST_USER_NAME} >/dev/null 2>&1; then
-	adduser --disabled-password --gecos "" pi
+	adduser --disabled-password --gecos "" ${FIRST_USER_NAME}
 fi
-echo "pi:raspberry" | chpasswd
+echo "${FIRST_USER_NAME}:${FIRST_USER_PASS}" | chpasswd
 echo "root:root" | chpasswd
 EOF
 

stage1/01-sys-tweaks/files/policy-rc.d

@@ -1,3 +0,0 @@
-#!/bin/sh
-
-exit 101

stage1/02-net-tweaks/00-packages

@@ -0,0 +1 @@
+netbase

stage1/02-net-tweaks/00-patches/02-persistant-net.diff

@@ -1,13 +0,0 @@
-Index: jessie-stage1/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-===================================================================
---- jessie-stage1.orig/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-+++ jessie-stage1/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-@@ -22,7 +22,7 @@ IMPORT{cmdline}="net.ifnames"
- ENV{net.ifnames}=="1",			GOTO="persistent_net_generator_end"
- 
- # device name whitelist
--KERNEL!="eth*|ath*|wlan*[0-9]|msh*|ra*|sta*|ctc*|lcs*|hsi*", \
-+KERNEL!="ath*|msh*|ra*|sta*|ctc*|lcs*|hsi*", \
- 					GOTO="persistent_net_generator_end"
- 
- # ignore Xen virtual interfaces

stage1/02-net-tweaks/00-patches/series

@@ -1,2 +1 @@
 01-hosts.diff
-02-persistant-net.diff

stage1/02-net-tweaks/00-run.sh

@@ -1,9 +1,5 @@
 #!/bin/bash -e
 
-install -m 644 files/ipv6.conf ${ROOTFS_DIR}/etc/modprobe.d/ipv6.conf
+install -m 644 files/hostname "${ROOTFS_DIR}/etc/hostname"
-install -m 644 files/interfaces ${ROOTFS_DIR}/etc/network/interfaces
-install -m 644 files/hostname ${ROOTFS_DIR}/etc/hostname
 
-on_chroot << EOF
+ln -sf /dev/null "${ROOTFS_DIR}/etc/systemd/network/99-default.link"
-dpkg-divert --add --local /lib/udev/rules.d/75-persistent-net-generator.rules
-EOF

stage1/02-net-tweaks/files/interfaces

@@ -1,4 +0,0 @@
-auto lo
-
-iface lo inet loopback
-iface eth0 inet dhcp

stage1/02-net-tweaks/files/ipv6.conf

@@ -1,3 +0,0 @@
-# Don't load ipv6 by default
-alias net-pf-10 off
-#alias ipv6 off

stage1/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage2/00-copies-and-fills/00-run.sh

@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-touch ${ROOTFS_DIR}/spindle_install

stage2/00-copies-and-fills/02-run.sh

@@ -1,3 +1,6 @@
 #!/bin/bash -e
 
-rm -f ${ROOTFS_DIR}/spindle_install
+if [ -f "${ROOTFS_DIR}/etc/ld.so.preload" ]; then
+   mv "${ROOTFS_DIR}/etc/ld.so.preload" "${ROOTFS_DIR}/etc/ld.so.preload.disabled"
+fi
+

stage2/01-sys-tweaks/00-debconf

@@ -7,12 +7,6 @@ console-setup   console-setup/codeset47 select  Guess optimal character set
 # Font for the console:
 # Choices: Fixed, Goha, GohaClassic, Terminus, TerminusBold, TerminusBoldVGA, VGA, Do not change the boot/kernel font, Let the system select a suitable font
 console-setup   console-setup/fontface47        select  Do not change the boot/kernel font
-# Geographic area:
-# Choices: Africa, America, Antarctica, Australia, Arctic Ocean, Asia, Atlantic Ocean, Europe, Indian Ocean, Pacific Ocean, System V timezones, US, None of the above
-tzdata	tzdata/Areas	select	Etc
-# Time zone:
-# Choices: GMT, GMT+0, GMT+1, GMT+10, GMT+11, GMT+12, GMT+2, GMT+3, GMT+4, GMT+5, GMT+6, GMT+7, GMT+8, GMT+9, GMT-0, GMT-1, GMT-10, GMT-11, GMT-12, GMT-13, GMT-14, GMT-2, GMT-3, GMT-4, GMT-5, GMT-6, GMT-7, GMT-8, GMT-9, GMT0, Greenwich, UCT, UTC, Universal, Zulu
-tzdata	tzdata/Zones/Etc	select	UTC
 # Key to function as AltGr:
 # Choices: The default for the keyboard layout, No AltGr key, Right Alt (AltGr), Right Control, Right Logo key, Menu key, Left Alt, Left Logo key, Keypad Enter key, Both Logo keys, Both Alt keys
 keyboard-configuration	keyboard-configuration/altgr	select	The default for the keyboard layout
@@ -21,7 +15,7 @@ keyboard-configuration	keyboard-configuration/altgr	select	The default for the k
 keyboard-configuration	keyboard-configuration/model	select	Generic 105-key (Intl) PC
 # Keymap to use:
 # Choices: American English, Albanian, Arabic, Asturian, Bangladesh, Belarusian, Bengali, Belgian, Bosnian, Brazilian, British English, Bulgarian, Bulgarian (phonetic layout), Burmese, Canadian French, Canadian Multilingual, Catalan, Chinese, Croatian, Czech, Danish, Dutch, Dvorak, Dzongkha, Esperanto, Estonian, Ethiopian, Finnish, French, Georgian, German, Greek, Gujarati, Gurmukhi, Hebrew, Hindi, Hungarian, Icelandic, Irish, Italian, Japanese, Kannada, Kazakh, Khmer, Kirghiz, Korean, Kurdish (F layout), Kurdish (Q layout), Lao, Latin American, Latvian, Lithuanian, Macedonian, Malayalam, Nepali, Northern Sami, Norwegian, Persian, Philippines, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian (Cyrillic), Sindhi, Sinhala, Slovak, Slovenian, Spanish, Swedish, Swiss French, Swiss German, Tajik, Tamil, Telugu, Thai, Tibetan, Turkish (F layout), Turkish (Q layout), Ukrainian, Uyghur, Vietnamese
-keyboard-configuration	keyboard-configuration/xkb-keymap	select	gb
+keyboard-configuration	keyboard-configuration/xkb-keymap	select	${KEYBOARD_KEYMAP}
 # Compose key:
 # Choices: No compose key, Right Alt (AltGr), Right Control, Right Logo key, Menu key, Left Logo key, Caps Lock
 keyboard-configuration	keyboard-configuration/compose	select	No compose key
@@ -29,4 +23,4 @@ keyboard-configuration	keyboard-configuration/compose	select	No compose key
 keyboard-configuration	keyboard-configuration/ctrl_alt_bksp	boolean	true
 # Keyboard layout:
 # Choices: English (UK), English (UK) - English (UK\, Colemak), English (UK) - English (UK\, Dvorak with UK punctuation), English (UK) - English (UK\, Dvorak), English (UK) - English (UK\, Macintosh international), English (UK) - English (UK\, Macintosh), English (UK) - English (UK\, extended WinKeys), English (UK) - English (UK\, international with dead keys), Other
-keyboard-configuration  keyboard-configuration/variant  select  English (UK)
+keyboard-configuration  keyboard-configuration/variant  select  ${KEYBOARD_LAYOUT}

stage2/01-sys-tweaks/00-packages

@@ -1,4 +1,4 @@
-ssh less fbset sudo psmisc strace module-init-tools ed ncdu crda
+ssh less fbset sudo psmisc strace ed ncdu crda
 console-setup keyboard-configuration debconf-utils parted unzip
 build-essential manpages-dev python bash-completion gdb pkg-config
 python-rpi.gpio v4l-utils
@@ -6,7 +6,7 @@ avahi-daemon
 lua5.1
 luajit
 hardlink ca-certificates curl
-fake-hwclock ntp nfs-common usbutils
+fake-hwclock nfs-common usbutils
 libraspberrypi-dev libraspberrypi-doc libfreetype6-dev
 dosfstools
 dphys-swapfile
@@ -14,6 +14,17 @@ raspberrypi-sys-mods
 pi-bluetooth
 apt-listchanges
 usb-modeswitch
-apt-transport-https
 libpam-chksshpwd
 rpi-update
+libmtp-runtime
+rsync
+htop
+man-db
+policykit-1
+ssh-import-id
+rng-tools
+ethtool
+vl805fw
+ntfs-3g
+pciutils
+rpi-eeprom

stage2/01-sys-tweaks/00-patches/06-rc_local.diff

@@ -1,15 +0,0 @@
-Index: jessie-stage2/rootfs/etc/rc.local
-===================================================================
---- jessie-stage2.orig/rootfs/etc/rc.local
-+++ jessie-stage2/rootfs/etc/rc.local
-@@ -11,4 +11,10 @@
- #
- # By default this script does nothing.
- 
-+# Print the IP address
-+_IP=$(hostname -I) || true
-+if [ "$_IP" ]; then
-+  printf "My IP address is %s\n" "$_IP"
-+fi
-+
- exit 0

stage2/01-sys-tweaks/00-patches/07-resize-init.diff

@@ -1,5 +1,5 @@
---- a/rootfs/boot/cmdline.txt
+--- stage2.orig/rootfs/boot/cmdline.txt
-+++ b/rootfs/boot/cmdline.txt
++++ stage2/rootfs/boot/cmdline.txt
 @@ -1 +1 @@
--dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
+-console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
-+dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh
++console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh

stage2/01-sys-tweaks/00-patches/series

@@ -2,5 +2,4 @@
 02-swap.diff
 04-inputrc.diff
 05-path.diff
-06-rc_local.diff
 07-resize-init.diff

stage2/01-sys-tweaks/01-run.sh

@@ -1,32 +1,47 @@
 #!/bin/bash -e
 
-install -m 644 files/regenerate_ssh_host_keys.service	${ROOTFS_DIR}/lib/systemd/system/
+install -m 755 files/resize2fs_once	"${ROOTFS_DIR}/etc/init.d/"
-install -m 755 files/apply_noobs_os_config		${ROOTFS_DIR}/etc/init.d/
-install -m 755 files/resize2fs_once			${ROOTFS_DIR}/etc/init.d/
 
-install -d						${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d
+install -d				"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d"
-install -m 644 files/ttyoutput.conf			${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/
+install -m 644 files/ttyoutput.conf	"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/"
 
-install -m 644 files/50raspi				${ROOTFS_DIR}/etc/apt/apt.conf.d/
+install -m 644 files/50raspi		"${ROOTFS_DIR}/etc/apt/apt.conf.d/"
 
-install -m 644 files/console-setup   			${ROOTFS_DIR}/etc/default/
+install -m 644 files/console-setup   	"${ROOTFS_DIR}/etc/default/"
+
+install -m 755 files/rc.local		"${ROOTFS_DIR}/etc/"
 
 on_chroot << EOF
 systemctl disable hwclock.sh
 systemctl disable nfs-common
 systemctl disable rpcbind
-systemctl disable ssh
+if [ "${ENABLE_SSH}" == "1" ]; then
+	systemctl enable ssh
+else
+	systemctl disable ssh
+fi
 systemctl enable regenerate_ssh_host_keys
-systemctl enable apply_noobs_os_config
-systemctl enable resize2fs_once
 EOF
 
-on_chroot << \EOF
+if [ "${USE_QEMU}" = "1" ]; then
+	echo "enter QEMU mode"
+	install -m 644 files/90-qemu.rules "${ROOTFS_DIR}/etc/udev/rules.d/"
+	on_chroot << EOF
+systemctl disable resize2fs_once
+EOF
+	echo "leaving QEMU mode"
+else
+	on_chroot << EOF
+systemctl enable resize2fs_once
+EOF
+fi
+
+on_chroot <<EOF
 for GRP in input spi i2c gpio; do
-	groupadd -f -r $GRP
+	groupadd -f -r "\$GRP"
 done
 for GRP in adm dialout cdrom audio users sudo video games plugdev input gpio spi i2c netdev; do
-  adduser pi $GRP
+  adduser $FIRST_USER_NAME \$GRP
 done
 EOF
 
@@ -38,4 +53,4 @@ on_chroot << EOF
 usermod --pass='*' root
 EOF
 
-rm -f ${ROOTFS_DIR}/etc/ssh/ssh_host_*_key*
+rm -f "${ROOTFS_DIR}/etc/ssh/"ssh_host_*_key*

stage2/01-sys-tweaks/files/90-qemu.rules

@@ -0,0 +1,3 @@
+KERNEL=="sda", SYMLINK+="mmcblk0"
+KERNEL=="sda?", SYMLINK+="mmcblk0p%n"
+KERNEL=="sda2", SYMLINK+="root"

stage2/01-sys-tweaks/files/apply_noobs_os_config

@@ -1,31 +0,0 @@
-#!/bin/sh
-### BEGIN INIT INFO
-# Provides:          apply_noobs_os_config
-# Required-Start:
-# Required-Stop:
-# Default-Start: 2
-# Default-Stop:
-# Short-Description: Apply config from /boot/os_config.json
-# Description:
-### END INIT INFO
-
-. /lib/lsb/init-functions
-
-set -e
-
-case "$1" in
-  start)
-    log_daemon_msg "Applying config from /boot/os_config.json (if it exists)"
-    if raspi-config --apply-os-config; then
-      update-rc.d apply_noobs_os_config remove
-      rm /etc/init.d/apply_noobs_os_config
-      log_end_msg 0
-    else
-      log_end_msg 1
-    fi
-    ;;
-  *)
-    echo "Usage: $0 start" >&2
-    exit 3
-    ;;
-esac

stage2/01-sys-tweaks/files/rc.local

@@ -0,0 +1,20 @@
+#!/bin/sh -e
+#
+# rc.local
+#
+# This script is executed at the end of each multiuser runlevel.
+# Make sure that the script will "exit 0" on success or any other
+# value on error.
+#
+# In order to enable or disable this script just change the execution
+# bits.
+#
+# By default this script does nothing.
+
+# Print the IP address
+_IP=$(hostname -I) || true
+if [ "$_IP" ]; then
+  printf "My IP address is %s\n" "$_IP"
+fi
+
+exit 0

stage2/01-sys-tweaks/files/regenerate_ssh_host_keys.service

@@ -1,11 +0,0 @@
-[Unit]
-Description=Regenerate SSH host keys
-
-[Service]
-Type=oneshot
-ExecStartPre=/bin/sh -c "if [ -e /dev/hwrng ]; then dd if=/dev/hwrng of=/dev/urandom count=1 bs=4096; fi"
-ExecStart=/usr/bin/ssh-keygen -A
-ExecStartPost=/bin/rm /lib/systemd/system/regenerate_ssh_host_keys.service ; /usr/sbin/update-rc.d regenerate_ssh_host_keys remove
-
-[Install]
-WantedBy=multi-user.target

stage2/02-net-tweaks/00-packages

@@ -1,3 +1,4 @@
-wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-ralink firmware-realtek
+wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek
 raspberrypi-net-mods
 dhcpcd5
+net-tools

stage2/02-net-tweaks/01-run.sh

@@ -1,8 +1,22 @@
 #!/bin/bash -e
 
-install -v -d						${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d
+install -v -d					"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d"
-install -v -m 644 files/wait.conf			${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/
+install -v -m 644 files/wait.conf		"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/"
 
-install -v -d                                           ${ROOTFS_DIR}/etc/wpa_supplicant
+install -v -d					"${ROOTFS_DIR}/etc/wpa_supplicant"
-install -v -m 600 files/wpa_supplicant.conf             ${ROOTFS_DIR}/etc/wpa_supplicant/
+install -v -m 600 files/wpa_supplicant.conf	"${ROOTFS_DIR}/etc/wpa_supplicant/"
 
+if [ -v WPA_COUNTRY ]; then
+	echo "country=${WPA_COUNTRY}" >> "${ROOTFS_DIR}/etc/wpa_supplicant/wpa_supplicant.conf"
+fi
+
+if [ -v WPA_ESSID ] && [ -v WPA_PASSWORD ]; then
+on_chroot <<EOF
+wpa_passphrase "${WPA_ESSID}" "${WPA_PASSWORD}" >> "/etc/wpa_supplicant/wpa_supplicant.conf"
+EOF
+fi
+
+# Disable wifi on 5GHz models
+mkdir -p "${ROOTFS_DIR}/var/lib/systemd/rfkill/"
+echo 1 > "${ROOTFS_DIR}/var/lib/systemd/rfkill/platform-3f300000.mmc:wlan"
+echo 1 > "${ROOTFS_DIR}/var/lib/systemd/rfkill/platform-fe300000.mmc:wlan"

stage2/02-net-tweaks/files/wait.conf

@@ -1,3 +1,3 @@
 [Service]
 ExecStart=
-ExecStart=/sbin/dhcpcd -q -w
+ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w

stage2/02-net-tweaks/files/wpa_supplicant.conf

@@ -1,3 +1,2 @@
-country=GB
 ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
 update_config=1

stage2/03-set-timezone/02-run.sh

@@ -0,0 +1,8 @@
+#!/bin/bash -e
+
+echo "${TIMEZONE_DEFAULT}" > "${ROOTFS_DIR}/etc/timezone"
+rm "${ROOTFS_DIR}/etc/localtime"
+
+on_chroot << EOF
+dpkg-reconfigure -f noninteractive tzdata
+EOF

stage2/EXPORT_IMAGE

@@ -1 +1,4 @@
 IMG_SUFFIX="-lite"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage2/EXPORT_NOOBS

@@ -1,2 +1,2 @@
 NOOBS_NAME="Raspbian Lite"
-NOOBS_DESCRIPTION="A port of Debian jessie for the Raspberry Pi (minimal version)"
+NOOBS_DESCRIPTION="A port of Debian with no desktop environment"

stage2/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage3/00-install-packages/00-packages

@@ -1,5 +1,5 @@
 gstreamer1.0-x gstreamer1.0-omx gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-alsa gstreamer1.0-libav
-xpdf gtk2-engines alsa-utils
+qpdfview gtk2-engines alsa-utils
 desktop-base
 git
 omxplayer
@@ -8,3 +8,8 @@ policykit-1
 gvfs
 rfkill
 chromium-browser rpi-chromium-mods
+gldriver-test
+fonts-droid-fallback
+fonts-liberation2
+obconf
+arandr

stage3/00-install-packages/00-packages-nr

@@ -1,6 +1,6 @@
 xserver-xorg-video-fbdev xserver-xorg xinit xserver-xorg-video-fbturbo
-epiphany-browser
+mousepad
-lxde lxtask menu-xdg gksu
+lxde lxtask menu-xdg
-netsurf-gtk zenity xdg-utils
+zenity xdg-utils
 gvfs-backends gvfs-fuse
 lightdm gnome-themes-standard-data gnome-icon-theme

stage3/01-tweaks/00-run.sh

@@ -1,3 +1,3 @@
 #!/bin/bash -e
 
-rm -f ${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf
+rm -f "${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf"

stage3/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage4/00-install-packages/00-packages

@@ -1,37 +1,26 @@
-java-common oracle-java8-jdk
+python python3-pygame python-pygame python-tk
-libreoffice-sdbc-hsqldb
+python3 python3-tk thonny
-sonic-pi
-python idle python3-pygame python-pygame python-tk
-python3 idle3 python3-tk
 python3-pgzero
 python-serial python3-serial
 python-picamera python3-picamera
-debian-reference-en dillo x2x
+debian-reference-en dillo
 raspberrypi-net-mods raspberrypi-ui-mods
-smartsim penguinspuzzle
 python-pip python3-pip
 python3-numpy
 pypy
-python3-pifacecommon python3-pifacedigitalio python3-pifacedigital-scratch-handler python-pifacecommon python-pifacedigitalio
-minecraft-pi python-minecraftpi
 alacarte rc-gui sense-hat
-claws-mail
 tree
-scratch nuscratch
+libgl1-mesa-dri libgles1 libgles2-mesa xcompmgr
-greenfoot bluej
-nodered
-libgl1-mesa-dri libgles1-mesa libgles2-mesa xcompmgr
 geany
 piclone
-pulseaudio-module-bluetooth
 wiringpi pigpio python-pigpio python3-pigpio raspi-gpio python-gpiozero python3-gpiozero python3-rpi.gpio
 python-spidev python3-spidev
 python-twython python3-twython
 python-smbus python3-smbus
 python-flask python3-flask
 python-picraft python3-picraft
-python3-codebug-tether python3-codebug-i2c-tether
-python-sense-emu python3-sense-emu sense-emu-tools python-sense-emu-doc
 pprompt
-scratch2
+piwiz
-python3-thonny
+rp-prefapps
+ffmpeg
+vlc

stage4/00-install-packages/00-packages-nr

@@ -1,4 +1,2 @@
-libreoffice libreoffice-gtk
-timidity
 pi-package
-realvnc-vnc-server realvnc-vnc-viewer
+realvnc-vnc-server

stage4/00-install-packages/01-packages

@@ -17,3 +17,6 @@ python-scrollphathd python3-scrollphathd
 python-sn3218 python3-sn3218
 python-skywriter python3-skywriter
 python-touchphat python3-touchphat
+python-buttonshim python3-buttonshim
+python-unicornhathd  python3-unicornhathd
+python-pantilthat python3-pantilthat

stage4/00-install-packages/02-packages

@@ -0,0 +1,4 @@
+hunspell-en-gb
+hyphen-en-gb
+wamerican
+wbritish

stage4/01-console-autologin/00-run.sh

@@ -1,3 +1,5 @@
 #!/bin/bash -e
 
-ln -sf /etc/systemd/system/[email protected] ${ROOTFS_DIR}/etc/systemd/system/getty.target.wants/[email protected]
+on_chroot << EOF
+	SUDO_USER="${FIRST_USER_NAME}" raspi-config nonint do_boot_behaviour B4
+EOF

stage4/02-extras/00-run.sh

@@ -1,33 +1,7 @@
 #!/bin/bash -e
 
-HASH=`wget https://api.github.com/repos/KenT2/python-games/git/refs/heads/master -qO -| grep \"sha\" | cut -f 2 -d ':' | cut -f 2 -d \"`
-
-if [ -f files/python_games.hash ]; then
-	HASH_LOCAL=`cat files/python_games.hash`
-fi
-
-if [ ! -e files/python_games.tar.gz ] || [ "$HASH" != "$HASH_LOCAL"  ]; then
-	wget "https://github.com/KenT2/python-games/tarball/master" -O files/python_games.tar.gz
-	echo $HASH > files/python_games.hash
-fi
-
-ln -sf pip3 ${ROOTFS_DIR}/usr/bin/pip-3.2
-
-install -v -o 1000 -g 1000 -d ${ROOTFS_DIR}/home/pi/python_games
-tar xvf files/python_games.tar.gz -C ${ROOTFS_DIR}/home/pi/python_games --strip-components=1
-chown 1000:1000 ${ROOTFS_DIR}/home/pi/python_games -Rv
-chmod +x ${ROOTFS_DIR}/home/pi/python_games/launcher.sh
-
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/BlueJ/ "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/Greenfoot/ "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/scratch/Projects/Demos/ "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
-
 #Alacarte fixes
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local"
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.local"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local/share"
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.local/share"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local/share/applications"
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.local/share/applications"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local/share/desktop-directories"
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/.local/share/desktop-directories"

stage4/02-extras/files/.gitignore

@@ -1,2 +0,0 @@
-python_games.hash
-python_games.tar.gz

stage4/03-magpi/00-run.sh

@@ -0,0 +1,15 @@
+#!/bin/sh -e
+
+magpi_regex="MagPi[[:digit:]]*.pdf"
+magpi_loc="https://www.raspberrypi.org/magpi-issues"
+magpi_latest="$(curl "$magpi_loc/?C=M;O=D" -s | grep "$magpi_regex" -m 1 -o | head -n 1)"
+
+if [ ! -f "files/$magpi_latest" ]; then
+	find files/ -regextype grep -regex "files/$magpi_regex" -delete
+	wget "$magpi_loc/$magpi_latest" -O "files/$magpi_latest"
+fi
+
+file "files/$magpi_latest" | grep -q "PDF document"
+
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/MagPi"
+install -v -o 1000 -g 1000 -m 644 "files/$magpi_latest" "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/MagPi/"

stage4/03-magpi/files/.gitignore

@@ -0,0 +1 @@
+*.pdf

stage4/04-enable-xcompmgr/00-run.sh

@@ -0,0 +1,5 @@
+#!/bin/bash -e
+
+on_chroot << EOF
+	raspi-config nonint do_xcompmgr 0
+EOF

stage4/EXPORT_IMAGE

@@ -1 +1,4 @@
-IMG_SUFFIX="-4GB"
+IMG_SUFFIX=""
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage4/EXPORT_NOOBS

@@ -0,0 +1,2 @@
+NOOBS_NAME="Raspbian"
+NOOBS_DESCRIPTION="A port of Debian with the Raspberry Pi Desktop"

stage4/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage5/00-install-extras/00-packages

@@ -0,0 +1,15 @@
+mu-editor
+sonic-pi
+scratch nuscratch scratch2 scratch3
+smartsim
+
+minecraft-pi python-minecraftpi
+python-sense-emu sense-emu-tools python-sense-emu-doc
+
+wolfram-engine
+claws-mail
+greenfoot-unbundled bluej
+nodered
+realvnc-vnc-viewer
+
+python-games

stage5/00-install-libreoffice/00-packages

@@ -0,0 +1,3 @@
+libreoffice-pi
+libreoffice-help-en-gb
+libreoffice-l10n-en-gb

stage5/00-install-mathematica/00-packages

@@ -1 +0,0 @@
-wolfram-engine

stage5/EXPORT_IMAGE

@@ -1 +1,4 @@
-IMG_SUFFIX=""
+IMG_SUFFIX="-full"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage5/EXPORT_NOOBS

@@ -1,2 +1,2 @@
-NOOBS_NAME="Raspbian"
+NOOBS_NAME="Raspbian Full"
-NOOBS_DESCRIPTION="A port of Debian jessie for the Raspberry Pi (full desktop version)"
+NOOBS_DESCRIPTION="A port of Debian with desktop and recommended applications"

stage5/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi