Update release notes

Remove claws-mail, greenfoot, bluej, nodered & realvnc-vnc-server

.dockerignore

@@ -1,3 +1,5 @@
 output/
 work/
 deploy/
+apt-cacher-ng/
+.git/objects/*

.gitignore

@@ -3,6 +3,7 @@ work/*
 config
 postrun.sh
 SKIP
+SKIP_IMAGES
 .pc
 *-pc
 apt-cacher-ng/

Dockerfile

@@ -1,10 +1,12 @@
 FROM debian:stretch
 
+ENV DEBIAN_FRONTEND noninteractive
+
 RUN apt-get -y update && \
     apt-get -y install \
         git vim parted \
         quilt realpath qemu-user-static debootstrap zerofree pxz zip dosfstools \
-        bsdtar libcap2-bin rsync grep udev xz-utils \
+        bsdtar libcap2-bin rsync grep udev xz-utils curl xxd \
     && rm -rf /var/lib/apt/lists/*
 
 COPY . /pi-gen/

README.md

@@ -3,13 +3,13 @@
 _Tool used to create the raspberrypi.org Raspbian images_
 
 
-### TODO
-1. Documentation
-
-
 ## Dependencies
 
-On Debian-based systems:
+pi-gen runs on Debian based operating systems. Currently it is only supported on
+either Debian Stretch or Ubuntu Xenial and is known to have issues building on
+earlier releases of these systems.
+
+To install the required dependencies for pi-gen you should run:
 
 ```bash
 apt-get install quilt parted realpath qemu-user-static debootstrap zerofree pxz zip \
@@ -62,10 +62,17 @@ The following environment variables are supported:
    system for each build stage, amounting to tens of gigabytes in the case of
    Raspbian.
    
+   **CAUTION**: If your working directory is on an NTFS partition you probably won't be able to build. Make sure this is a proper Linux filesystem.
+
  * `DEPLOY_DIR`  (Default: `"$BASE_DIR/deploy"`)
 
    Output directory for target system images and NOOBS bundles.
 
+ * `USE_QEMU` (Default: `"0"`)
+
+   Setting to '1' enables the QEMU mode - creating an image that can be mounted via QEMU for an emulated
+   environment. These images include "-qemu" in the image file name.
+
 
 A simple example for building Raspbian:
 
@@ -74,6 +81,51 @@ IMG_NAME='Raspbian'
 ```
 
 
+## How the build process works
+
+The following process is followed to build images:
+
+ * Loop through all of the stage directories in alphanumeric order
+
+ * Move on to the next directory if this stage directory contains a file called
+   "SKIP"
+
+ * Run the script ```prerun.sh``` which is generally just used to copy the build
+   directory between stages.
+
+ * In each stage directory loop through each subdirectory and then run each of the
+   install scripts it contains, again in alphanumeric order. These need to be named
+   with a two digit padded number at the beginning.
+   There are a number of different files and directories which can be used to
+   control different parts of the build process:
+
+     - **00-run.sh** - A unix shell script. Needs to be made executable for it to run.
+
+     - **00-run-chroot.sh** - A unix shell script which will be run in the chroot
+       of the image build directory. Needs to be made executable for it to run.
+
+     - **00-debconf** - Contents of this file are passed to debconf-set-selections
+       to configure things like locale, etc.
+
+     - **00-packages** - A list of packages to install. Can have more than one, space
+       separated, per line.
+
+     - **00-packages-nr** - As 00-packages, except these will be installed using
+       the ```--no-install-recommends -y``` parameters to apt-get.
+
+     - **00-patches** - A directory containing patch files to be applied, using quilt.
+       If a file named 'EDIT' is present in the directory, the build process will
+       be interrupted with a bash session, allowing an opportunity to create/revise
+       the patches.
+
+  * If the stage directory contains files called "EXPORT_NOOBS" or "EXPORT_IMAGE" then
+    add this stage to a list of images to generate
+
+  * Generate the images for any stages that have specified them
+
+It is recommended to examine build.sh for finer details.
+
+
 ## Docker Build
 
 ```bash
@@ -91,6 +143,12 @@ continue:
 CONTINUE=1 ./build-docker.sh
 ```
 
+After successful build, the build container is by default removed. This may be undesired when making incremental changes to a customized build. To prevent the build script from remove the container add
+
+```bash
+PRESERVE_CONTAINER=1 ./build-docker.sh
+```
+
 There is a possibility that even when running from a docker container, the
 installation of `qemu-user-static` will silently fail when building the image
 because `binfmt-support` _must be enabled on the underlying kernel_. An easy
@@ -159,17 +217,35 @@ If you wish to build up to a specified stage (such as building up to stage 2
 for a lite system), place an empty file named `SKIP` in each of the `./stage`
 directories you wish not to include.
 
-Then remove the `EXPORT*` files from `./stage4` (if building up to stage 2) or
+Then add an empty file named `SKIP_IMAGES` to `./stage4` (if building up to stage 2) or
-from `./stage2` (if building a minimal system).
+to `./stage2` (if building a minimal system).
 
 ```bash
 # Example for building a lite system
 echo "IMG_NAME='Raspbian'" > config
 touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
-rm stage4/EXPORT*
+touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
 sudo ./build.sh  # or ./build-docker.sh
 ```
 
 If you wish to build further configurations upon (for example) the lite
 system, you can also delete the contents of `./stage3` and `./stage4` and
 replace with your own contents in the same format.
+
+
+## Skipping stages to speed up development
+
+If you're working on a specific stage the recommended development process is as
+follows:
+
+ * Add a file called SKIP_IMAGES into the directories containing EXPORT_* files
+   (currently stage2, stage4 and stage5)
+ * Add SKIP files to the stages you don't want to build. For example, if you're
+   basing your image on the lite image you would add these to stages 3, 4 and 5.
+ * Run build.sh to build all stages
+ * Add SKIP files to the earlier successfully built stages
+ * Modify the last stage
+ * Rebuild just the last stage using ```sudo CLEAN=1 ./build.sh```
+ * Once you're happy with the image you can remove the SKIP_IMAGES files and
+   export your image to test
+

build-docker.sh

@@ -13,19 +13,19 @@ if ! $DOCKER ps >/dev/null; then
 fi
 set -e
 
-
+config_file=()
-config_mount=()
 if [ -f config ]; then
-	config_mount=("-v" "$(pwd)/config:/pi-gen/config:ro")
+	config_file=("--env-file" "$(pwd)/config")
 	source config
 fi
 
 CONTAINER_NAME=${CONTAINER_NAME:-pigen_work}
 CONTINUE=${CONTINUE:-0}
+PRESERVE_CONTAINER=${PRESERVE_CONTAINER:-0}
 
 if [ "$*" != "" ] || [ -z "${IMG_NAME}" ]; then
 	if [ -z "${IMG_NAME}" ]; then
-		echo "IMG_NAME not set in 'build'" 1>&2
+		echo "IMG_NAME not set in 'config'" 1>&2
 		echo 1>&2
 	fi
 	cat >&2 <<EOF
@@ -33,7 +33,8 @@ Usage:
     build-docker.sh [options]
 Optional environment arguments: ( =<default> )
     CONTAINER_NAME=pigen_work  set a name for the build container
-    CONTINUE=0                  continue from a previously started container
+    CONTINUE=1                 continue from a previously started container
+    PRESERVE_CONTAINER=1       keep build container even on successful build
 EOF
 	exit 1
 fi
@@ -47,31 +48,39 @@ fi
 if [ "$CONTAINER_EXISTS" != "" ] && [ "$CONTINUE" != "1" ]; then
 	echo "Container $CONTAINER_NAME already exists and you did not specify CONTINUE=1. Aborting."
 	echo "You can delete the existing container like this:"
-	echo "  docker rm -v $CONTAINER_NAME"
+	echo "  $DOCKER rm -v $CONTAINER_NAME"
 	exit 1
 fi
 
 $DOCKER build -t pi-gen .
 if [ "$CONTAINER_EXISTS" != "" ]; then
-	trap "echo 'got CTRL+C... please wait 5s';docker stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
+	trap "echo 'got CTRL+C... please wait 5s'; $DOCKER stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
 	time $DOCKER run --rm --privileged \
 		--volumes-from="${CONTAINER_NAME}" --name "${CONTAINER_NAME}_cont" \
-		-e IMG_NAME=${IMG_NAME}\
+		-e IMG_NAME="${IMG_NAME}"\
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh;
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 else
-	trap "echo 'got CTRL+C... please wait 5s'; docker stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
+	trap "echo 'got CTRL+C... please wait 5s'; $DOCKER stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
-	$DOCKER run --name "${CONTAINER_NAME}" --privileged \
+	time $DOCKER run --name "${CONTAINER_NAME}" --privileged \
-		-e IMG_NAME=${IMG_NAME}\
+		-e IMG_NAME="${IMG_NAME}"\
-		-v "$(pwd)/deploy:/pi-gen/deploy" \
+		"${config_file[@]}" \
-		"${config_mount[@]}" \
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh &&
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 fi
+echo "copying results from deploy/"
+$DOCKER cp "${CONTAINER_NAME}":/pi-gen/deploy .
+ls -lah deploy
+
+# cleanup
+if [ "$PRESERVE_CONTAINER" != "1" ]; then
+	$DOCKER rm -v $CONTAINER_NAME
+fi
+
 echo "Done! Your image(s) should be in deploy/"

build.sh

@@ -1,22 +1,23 @@
 #!/bin/bash -e
-
+# shellcheck disable=SC2119,SC1091
 run_sub_stage()
 {
 	log "Begin ${SUB_STAGE_DIR}"
-	pushd ${SUB_STAGE_DIR} > /dev/null
+	pushd "${SUB_STAGE_DIR}" > /dev/null
 	for i in {00..99}; do
-		if [ -f ${i}-debconf ]; then
+		if [ -f "${i}-debconf" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-debconf"
 			on_chroot << EOF
 debconf-set-selections <<SELEOF
-`cat ${i}-debconf`
+$(cat "${i}-debconf")
 SELEOF
 EOF
+
 		log "End ${SUB_STAGE_DIR}/${i}-debconf"
 		fi
-		if [ -f ${i}-packages-nr ]; then
+		if [ -f "${i}-packages-nr" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages-nr"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages-nr)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages-nr")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install --no-install-recommends -y $PACKAGES
@@ -24,9 +25,9 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages-nr"
 		fi
-		if [ -f ${i}-packages ]; then
+		if [ -f "${i}-packages" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install -y $PACKAGES
@@ -34,18 +35,18 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages"
 		fi
-		if [ -d ${i}-patches ]; then
+		if [ -d "${i}-patches" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-patches"
-			pushd ${STAGE_WORK_DIR} > /dev/null
+			pushd "${STAGE_WORK_DIR}" > /dev/null
 			if [ "${CLEAN}" = "1" ]; then
 				rm -rf .pc
-				rm -rf *-pc
+				rm -rf ./*-pc
 			fi
-			QUILT_PATCHES=${SUB_STAGE_DIR}/${i}-patches
+			QUILT_PATCHES="${SUB_STAGE_DIR}/${i}-patches"
-			SUB_STAGE_QUILT_PATCH_DIR="$(basename $SUB_STAGE_DIR)-pc"
+			SUB_STAGE_QUILT_PATCH_DIR="$(basename "$SUB_STAGE_DIR")-pc"
-			mkdir -p $SUB_STAGE_QUILT_PATCH_DIR
+			mkdir -p "$SUB_STAGE_QUILT_PATCH_DIR"
-			ln -snf $SUB_STAGE_QUILT_PATCH_DIR .pc
+			ln -snf "$SUB_STAGE_QUILT_PATCH_DIR" .pc
-			if [ -e ${SUB_STAGE_DIR}/${i}-patches/EDIT ]; then
+			if [ -e "${SUB_STAGE_DIR}/${i}-patches/EDIT" ]; then
 				echo "Dropping into bash to edit patches..."
 				bash
 			fi
@@ -80,18 +81,20 @@ EOF
 
 run_stage(){
 	log "Begin ${STAGE_DIR}"
-	STAGE=$(basename ${STAGE_DIR})
+	STAGE="$(basename "${STAGE_DIR}")"
-	pushd ${STAGE_DIR} > /dev/null
+	pushd "${STAGE_DIR}" > /dev/null
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	STAGE_WORK_DIR=${WORK_DIR}/${STAGE}
+	STAGE_WORK_DIR="${WORK_DIR}/${STAGE}"
-	ROOTFS_DIR=${STAGE_WORK_DIR}/rootfs
+	ROOTFS_DIR="${STAGE_WORK_DIR}"/rootfs
-	if [ -f ${STAGE_DIR}/EXPORT_IMAGE ]; then
+	if [ ! -f SKIP_IMAGES ]; then
-		EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
+		if [ -f "${STAGE_DIR}/EXPORT_IMAGE" ]; then
+			EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
+		fi
 	fi
 	if [ ! -f SKIP ]; then
 		if [ "${CLEAN}" = "1" ]; then
-			if [ -d ${ROOTFS_DIR} ]; then
+			if [ -d "${ROOTFS_DIR}" ]; then
-				rm -rf ${ROOTFS_DIR}
+				rm -rf "${ROOTFS_DIR}"
 			fi
 		fi
 		if [ -x prerun.sh ]; then
@@ -100,16 +103,16 @@ run_stage(){
 			log "End ${STAGE_DIR}/prerun.sh"
 		fi
 		for SUB_STAGE_DIR in ${STAGE_DIR}/*; do
-			if [ -d ${SUB_STAGE_DIR} ] &&
+			if [ -d "${SUB_STAGE_DIR}" ] &&
-			   [ ! -f ${SUB_STAGE_DIR}/SKIP ]; then
+			   [ ! -f "${SUB_STAGE_DIR}/SKIP" ]; then
 				run_sub_stage
 			fi
 		done
 	fi
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	PREV_STAGE=${STAGE}
+	PREV_STAGE="${STAGE}"
-	PREV_STAGE_DIR=${STAGE_DIR}
+	PREV_STAGE_DIR="${STAGE_DIR}"
-	PREV_ROOTFS_DIR=${ROOTFS_DIR}
+	PREV_ROOTFS_DIR="${ROOTFS_DIR}"
 	popd > /dev/null
 	log "End ${STAGE_DIR}"
 }
@@ -119,6 +122,7 @@ if [ "$(id -u)" != "0" ]; then
 	exit 1
 fi
 
+
 if [ -f config ]; then
 	source config
 fi
@@ -128,14 +132,17 @@ if [ -z "${IMG_NAME}" ]; then
 	exit 1
 fi
 
-export IMG_DATE=${IMG_DATE:-"$(date +%Y-%m-%d)"}
+export USE_QEMU="${USE_QEMU:-0}"
+export IMG_DATE="${IMG_DATE:-"$(date +%Y-%m-%d)"}"
 
-export BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 export SCRIPT_DIR="${BASE_DIR}/scripts"
-export WORK_DIR=${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}
+export WORK_DIR="${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}"
 export DEPLOY_DIR=${DEPLOY_DIR:-"${BASE_DIR}/deploy"}
 export LOG_FILE="${WORK_DIR}/build.log"
 
+export BASE_DIR
+
 export CLEAN
 export IMG_NAME
 export APT_PROXY
@@ -158,29 +165,35 @@ export QUILT_NO_DIFF_INDEX=1
 export QUILT_NO_DIFF_TIMESTAMPS=1
 export QUILT_REFRESH_ARGS="-p ab"
 
-source ${SCRIPT_DIR}/common
+# shellcheck source=scripts/common
-source ${SCRIPT_DIR}/dependencies_check
+source "${SCRIPT_DIR}/common"
+# shellcheck source=scripts/dependencies_check
+source "${SCRIPT_DIR}/dependencies_check"
 
 
-dependencies_check ${BASE_DIR}/depends
+dependencies_check "${BASE_DIR}/depends"
 
-mkdir -p ${WORK_DIR}
+mkdir -p "${WORK_DIR}"
 log "Begin ${BASE_DIR}"
 
-for STAGE_DIR in ${BASE_DIR}/stage*; do
+for STAGE_DIR in "${BASE_DIR}/stage"*; do
 	run_stage
 done
 
 CLEAN=1
 for EXPORT_DIR in ${EXPORT_DIRS}; do
 	STAGE_DIR=${BASE_DIR}/export-image
+	# shellcheck source=/dev/null
 	source "${EXPORT_DIR}/EXPORT_IMAGE"
-	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename ${EXPORT_DIR})/rootfs
+	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename "${EXPORT_DIR}")/rootfs
 	run_stage
-	if [ -e ${EXPORT_DIR}/EXPORT_NOOBS ]; then
+	if [ "${USE_QEMU}" != "1" ]; then
-		source ${EXPORT_DIR}/EXPORT_NOOBS
+		if [ -e "${EXPORT_DIR}/EXPORT_NOOBS" ]; then
-		STAGE_DIR=${BASE_DIR}/export-noobs
+			# shellcheck source=/dev/null
-		run_stage
+			source "${EXPORT_DIR}/EXPORT_NOOBS"
+			STAGE_DIR="${BASE_DIR}/export-noobs"
+			run_stage
+		fi
 	fi
 done
 

depends

@@ -11,3 +11,5 @@ bsdtar
 grep
 rsync
 xz:xz-utils
+curl
+xxd

export-image/00-allow-rerun/00-run.sh

@@ -1,13 +1,5 @@
 #!/bin/bash -e
 
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload ]; then
+if [ ! -x "${ROOTFS_DIR}/usr/bin/qemu-arm-static" ]; then
-	mv ${ROOTFS_DIR}/etc/ld.so.preload ${ROOTFS_DIR}/etc/ld.so.preload.disabled
+	cp /usr/bin/qemu-arm-static "${ROOTFS_DIR}/usr/bin/"
-fi
-
-if [ ! -e ${ROOTFS_DIR}/usr/sbin/policy-rc.d ]; then
-	install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/
-fi
-
-if [ ! -x ${ROOTFS_DIR}/usr/bin/qemu-arm-static ]; then
-	cp /usr/bin/qemu-arm-static ${ROOTFS_DIR}/usr/bin/
 fi

export-image/00-allow-rerun/files/policy-rc.d

@@ -1,3 +0,0 @@
-#!/bin/sh
-
-exit 101

export-image/02-network/01-run.sh

@@ -1,3 +1,3 @@
 #!/bin/bash -e
 
-install -m 644 files/resolv.conf ${ROOTFS_DIR}/etc/
+install -m 644 files/resolv.conf "${ROOTFS_DIR}/etc/"

export-image/03-set-partuuid/00-run.sh

@@ -2,12 +2,12 @@
 
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
-IMGID="$(fdisk -l ${IMG_FILE} | sed -n 's/Disk identifier: 0x\([^ ]*\)/\1/p')"
+IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
 
 BOOT_PARTUUID="${IMGID}-01"
 ROOT_PARTUUID="${IMGID}-02"
 
-sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
 
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/boot/cmdline.txt
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/cmdline.txt"

export-image/04-finalise/01-run.sh

@@ -8,77 +8,77 @@ on_chroot << EOF
 hardlink -t /usr/share/doc
 EOF
 
-if [ -d ${ROOTFS_DIR}/home/pi/.config ]; then
+if [ -d "${ROOTFS_DIR}/home/pi/.config" ]; then
-	chmod 700 ${ROOTFS_DIR}/home/pi/.config
+	chmod 700 "${ROOTFS_DIR}/home/pi/.config"
 fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-rm -f ${ROOTFS_DIR}/usr/sbin/policy-rc.d
+rm -f "${ROOTFS_DIR}/usr/bin/qemu-arm-static"
-rm -f ${ROOTFS_DIR}/usr/bin/qemu-arm-static
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload.disabled ]; then
-        mv ${ROOTFS_DIR}/etc/ld.so.preload.disabled ${ROOTFS_DIR}/etc/ld.so.preload
-fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/sources.list~
+rm -f "${ROOTFS_DIR}/etc/apt/sources.list~"
-rm -f ${ROOTFS_DIR}/etc/apt/trusted.gpg~
+rm -f "${ROOTFS_DIR}/etc/apt/trusted.gpg~"
 
-rm -f ${ROOTFS_DIR}/etc/passwd-
+rm -f "${ROOTFS_DIR}/etc/passwd-"
-rm -f ${ROOTFS_DIR}/etc/group-
+rm -f "${ROOTFS_DIR}/etc/group-"
-rm -f ${ROOTFS_DIR}/etc/shadow-
+rm -f "${ROOTFS_DIR}/etc/shadow-"
-rm -f ${ROOTFS_DIR}/etc/gshadow-
+rm -f "${ROOTFS_DIR}/etc/gshadow-"
+rm -f "${ROOTFS_DIR}/etc/subuid-"
+rm -f "${ROOTFS_DIR}/etc/subgid-"
 
-rm -f ${ROOTFS_DIR}/var/cache/debconf/*-old
+rm -f "${ROOTFS_DIR}"/var/cache/debconf/*-old
-rm -f ${ROOTFS_DIR}/var/lib/dpkg/*-old
+rm -f "${ROOTFS_DIR}"/var/lib/dpkg/*-old
 
-rm -f ${ROOTFS_DIR}/usr/share/icons/*/icon-theme.cache
+rm -f "${ROOTFS_DIR}"/usr/share/icons/*/icon-theme.cache
 
-rm -f ${ROOTFS_DIR}/var/lib/dbus/machine-id
+rm -f "${ROOTFS_DIR}/var/lib/dbus/machine-id"
 
-true > ${ROOTFS_DIR}/etc/machine-id
+true > "${ROOTFS_DIR}/etc/machine-id"
 
-ln -nsf /proc/mounts ${ROOTFS_DIR}/etc/mtab
+ln -nsf /proc/mounts "${ROOTFS_DIR}/etc/mtab"
 
-for _FILE in $(find ${ROOTFS_DIR}/var/log/ -type f); do
+find "${ROOTFS_DIR}/var/log/" -type f -exec cp /dev/null {} \;
-	true > ${_FILE}
-done
 
 rm -f "${ROOTFS_DIR}/root/.vnc/private.key"
+rm -f "${ROOTFS_DIR}/etc/vnc/updateid"
 
-update_issue $(basename ${EXPORT_DIR})
+update_issue "$(basename "${EXPORT_DIR}")"
-install -m 644 ${ROOTFS_DIR}/etc/rpi-issue ${ROOTFS_DIR}/boot/issue.txt
+install -m 644 "${ROOTFS_DIR}/etc/rpi-issue" "${ROOTFS_DIR}/boot/issue.txt"
-install files/LICENSE.oracle ${ROOTFS_DIR}/boot/
+install files/LICENSE.oracle "${ROOTFS_DIR}/boot/"
 
 
 cp "$ROOTFS_DIR/etc/rpi-issue" "$INFO_FILE"
 
-firmware=$(zgrep "firmware as of" "$ROOTFS_DIR/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" | \
-	head -n1 | \
-	sed  -n 's|.* \([^ ]*\)$|\1|p')
 
-printf "\nFirmware: https://github.com/raspberrypi/firmware/tree/%s\n" "$firmware" >> "$INFO_FILE"
+{
+	firmware=$(zgrep "firmware as of" \
+		"$ROOTFS_DIR/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" | \
+		head -n1 | sed  -n 's|.* \([^ ]*\)$|\1|p')
+	printf "\nFirmware: https://github.com/raspberrypi/firmware/tree/%s\n" "$firmware"
 
-kernel=$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/git_hash")
+	kernel="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/git_hash")"
-printf "Kernel: https://github.com/raspberrypi/linux/tree/%s\n" "$kernel" >> "$INFO_FILE"
+	printf "Kernel: https://github.com/raspberrypi/linux/tree/%s\n" "$kernel"
 
-uname=$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/uname_string7")
+	uname="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/uname_string7")"
-printf "Uname string: %s\n" "$uname" >> "$INFO_FILE"
 
-printf "\nPackages:\n">> "$INFO_FILE"
+	printf "Uname string: %s\n" "$uname"
-dpkg -l --root "$ROOTFS_DIR" >> "$INFO_FILE"
+	printf "\nPackages:\n"
+	dpkg -l --root "$ROOTFS_DIR"
+} >> "$INFO_FILE"
 
-ROOT_DEV=$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')
+ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
 
-unmount ${ROOTFS_DIR}
+unmount "${ROOTFS_DIR}"
-zerofree -v ${ROOT_DEV}
+zerofree -v "${ROOT_DEV}"
 
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-mkdir -p ${DEPLOY_DIR}
+mkdir -p "${DEPLOY_DIR}"
 
-rm -f ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip
+rm -f "${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip"
 
-pushd ${STAGE_WORK_DIR} > /dev/null
+pushd "${STAGE_WORK_DIR}" > /dev/null
-zip ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip $(basename ${IMG_FILE})
+zip "${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip" \
+	"$(basename "${IMG_FILE}")"
 popd > /dev/null
 
 cp "$INFO_FILE" "$DEPLOY_DIR"

export-image/prerun.sh

@@ -2,20 +2,22 @@
 
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-rm -f ${IMG_FILE}
+rm -f "${IMG_FILE}"
 
-rm -rf ${ROOTFS_DIR}
+rm -rf "${ROOTFS_DIR}"
-mkdir -p ${ROOTFS_DIR}
+mkdir -p "${ROOTFS_DIR}"
 
-BOOT_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR}/boot --block-size=1 | cut -f 1)
+BOOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}/boot" --block-size=1 | cut -f 1)
-TOTAL_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR} --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
+TOTAL_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
 
-IMG_SIZE=$((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024)))
+ROUND_SIZE="$((4 * 1024 * 1024))"
+ROUNDED_ROOT_SECTOR=$(((2 * BOOT_SIZE + ROUND_SIZE) / ROUND_SIZE * ROUND_SIZE / 512 + 8192))
+IMG_SIZE=$(((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024) + ROUND_SIZE - 1) / ROUND_SIZE * ROUND_SIZE))
 
-truncate -s ${IMG_SIZE} ${IMG_FILE}
+truncate -s "${IMG_SIZE}" "${IMG_FILE}"
-fdisk -H 255 -S 63 ${IMG_FILE} <<EOF
+fdisk -H 255 -S 63 "${IMG_FILE}" <<EOF
 o
 n
 
@@ -28,14 +30,14 @@ c
 n
 
 
-8192
+${ROUNDED_ROOT_SECTOR}
 
 
 p
 w
 EOF
 
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
+PARTED_OUT=$(parted -s "${IMG_FILE}" unit b print)
 BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
 | cut -d" " -f 2 | tr -d B)
 BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
@@ -46,8 +48,8 @@ ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 | cut -d" " -f 4 | tr -d B)
 
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
@@ -57,11 +59,11 @@ for FEATURE in metadata_csum 64bit; do
 	    ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
 	fi
 done
-mkdosfs -n boot -F 32 -v $BOOT_DEV > /dev/null
+mkdosfs -n boot -F 32 -v "$BOOT_DEV" > /dev/null
-mkfs.ext4 -O $ROOT_FEATURES $ROOT_DEV > /dev/null
+mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
 
-mount -v $ROOT_DEV ${ROOTFS_DIR} -t ext4
+mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
-mkdir -p ${ROOTFS_DIR}/boot
+mkdir -p "${ROOTFS_DIR}/boot"
-mount -v $BOOT_DEV ${ROOTFS_DIR}/boot -t vfat
+mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot" -t vfat
 
-rsync -aHAXx --exclude var/cache/apt/archives ${EXPORT_ROOTFS_DIR}/ ${ROOTFS_DIR}/
+rsync -aHAXx --exclude var/cache/apt/archives "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"

export-noobs/00-release/00-run.sh

@@ -2,35 +2,35 @@
 
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
 
-install -v -m 744	files/partition_setup.sh	${NOOBS_DIR}/
+install -v -m 744	files/partition_setup.sh	"${NOOBS_DIR}/"
-install -v		files/partitions.json		${NOOBS_DIR}/
+install -v		files/partitions.json		"${NOOBS_DIR}/"
-install -v		files/os.json			${NOOBS_DIR}/
+install -v		files/os.json			"${NOOBS_DIR}/"
-install -v		files/OS.png			${NOOBS_DIR}/
+install -v		files/OS.png			"${NOOBS_DIR}/"
-install -v		files/release_notes.txt		${NOOBS_DIR}/
+install -v		files/release_notes.txt		"${NOOBS_DIR}/"
 
-tar -v -c -C		files/marketing			-f ${NOOBS_DIR}/marketing.tar .
+tar -v -c -C		files/marketing			-f "${NOOBS_DIR}/marketing.tar" .
 
-BOOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/boot.tar.xz  | grep totals | cut -f 5)
+BOOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/boot.tar.xz"  | grep totals | cut -f 5)"
-ROOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/root.tar.xz  | grep totals | cut -f 5)
+ROOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/root.tar.xz"  | grep totals | cut -f 5)"
 
-BOOT_SIZE=$(expr ${BOOT_SIZE} / 1000000 \+ 1)
+BOOT_SIZE="$(( BOOT_SIZE / 1000000 + 1))"
-ROOT_SIZE=$(expr ${ROOT_SIZE} / 1000000 \+ 1)
+ROOT_SIZE="$(( ROOT_SIZE / 1000000 + 1))"
 
-BOOT_NOM=$(expr ${BOOT_SIZE} \* 3)
+BOOT_NOM="$(( BOOT_SIZE * 3 ))"
-ROOT_NOM=$(expr ${ROOT_SIZE} \+ 400)
+ROOT_NOM="$(( ROOT_SIZE + 400 ))"
 
-mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/$(echo ${NOOBS_NAME} | sed 's/ /_/g').png"
+mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/${NOOBS_NAME// /_}.png"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_NOM|${BOOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_NOM|${BOOT_NOM}|"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_NOM|${ROOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_NOM|${ROOT_NOM}|"
 
-sed ${NOOBS_DIR}/os.json -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|UNRELEASED|${IMG_DATE}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
 
-sed ${NOOBS_DIR}/release_notes.txt -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/release_notes.txt" -i -e "s|UNRELEASED|${IMG_DATE}|"
 
-cp -a ${NOOBS_DIR} ${DEPLOY_DIR}/
+cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"

export-noobs/00-release/files/os.json

@@ -1,7 +1,7 @@
 {
     "description": "NOOBS_DESCRIPTION",
     "feature_level": 35120124,
-    "kernel": "4.9",
+    "kernel": "4.14",
     "name": "NOOBS_NAME",
     "password": "raspberry",
     "release_date": "UNRELEASED",
@@ -10,7 +10,8 @@
         "Pi Model",
         "Pi 2",
         "Pi Zero",
-        "Pi 3"
+        "Pi 3",
+        "Pi Compute Module 3"
     ],
     "url": "http://www.raspbian.org/",
     "username": "pi",

export-noobs/00-release/files/release_notes.txt

@@ -1,4 +1,84 @@
 UNRELEASED:
+  * New first-boot configuration wizard added
+  * Recommended Software installer added
+  * Bluej, Greenfoot, NodeRED, Claws Mail, VNC Viewer removed from image - can now be installed from Recommended Applications
+  * Qpdfview PDF viewer installed instead of Xpdf
+  * Version 65.0 of Chromium browser included, with latest Flash player
+  * Volume up / down keys now change by 5% increments and affect currently-selected output device rather than internal device only
+  * Network plugin now remembers previously-entered WiFi network passwords when prompting for reconnection
+  * Serial port and serial console can now be switched separately in Raspberry Pi Configuration
+  * Lxkeymap keyboard language setting application removed - replaced with dialog within Raspberry Pi Configuration
+  * Wifi country and keyboard language setting dialogs in Raspberry Pi Configuration now callable from other applications
+  * New version of Piboto font included to render with correct weight under some rogue applications
+  * Reconnection to Bluetooth audio devices on reboot improved
+  * Disable click-to-rename behaviour in file manager if single-click selection enabled
+  * Appearance Settings dialog makes config changes to some Qt files to match selected theme
+  * MIME file type associations improved
+  * Multiple desktop management options removed from mouse middle-click menu
+  * Menu shortcuts to Raspberry Pi website amended
+  * Python 2 IDLE menu link removed
+  * Sample Magpi PDF installed in /home/pi/MagPi
+  * Various minor tweaks, bug fixes and appearance changes
+  * Bluetooth updates
+    - Firmware with Bluetooth 4.2 features
+    - SCO profile suppot added via bthelper.service
+  * Linux kernel 4.14.50+
+  * Raspberry Pi firmware 748fb17992426bb29d99224b93cb962fefbdc833
+2018-08-14:
+  * Fixed race between wifi-country.service and raspberrypi-net-mods.service
+  * Linux kernel 4.14.34+
+  * Raspberry Pi firmware 5db8e4e1c63178e200d6fbea23ed4a9bf4656658
+2018-03-13:
+  * Raspberry Pi 3 B+ support
+  * WiFi is disabled until wireless regulatory domain is set (Pi 3 B+ only)
+    - The domain can be done through 'Raspberry Pi Configuration' (rc_gui),
+      'raspi-config' or by setting 'country=' to an appropriate ISO 3166
+      alpha2 country code in /etc/wpa_supplicant/wpa_supplicant.conf.
+  * Default wireless regulatory domain is now unset
+  * Added support to desktop for different screen sizes and resolutions,
+    including multiple preset options in Appearance Settings and pixel doubling
+    option in Raspberry Pi Configuration
+  * Version 2.1.16 of Thonny included
+  * Version 29.0.0.113 of Adobe PepperFlash player included
+  * Version 1.2.post1 of Pygame Zero included
+  * Bluetooth plugin now supports connection to Bluetooth LE HID devices
+  * Network plugin now indicates 5G-compatible APs
+  * Latest changes to Bluez ALSA service merged
+    - service now started on CLI boot as well as GUI boot
+  * Latest changes to dhcpcd networking plugin merged
+  * Improved support for running on pi-top devices
+  * Small design changes to PiX theme and icons
+  * Bug fix - hide spurious window resize handles
+  * Bug fix - Scratch 2 remote GPIO state block now works correctly
+  * Updated WiFi Firmware
+    - brcmfmac43455-sdio 7.45.154
+    - brcmfmac43430-sdio 7.45.98.38
+  * New packages:
+    - policykit-1
+    - obconf
+    - python-buttonshim python3-buttonshim
+    - python-unicornhathd  python3-unicornhathd
+    - python-pantilthat python3-pantilthat
+  * Linux kernel 4.9.80+
+  * Raspberry Pi firmware 3347884c7df574bbabeff6dca63caf686e629699
+2017-11-29:
+  * Added battery monitor plugin for taskbar - works on x86 images or first-generation Pi-Top
+  * Added cutdown mode to PCManFM file manager to reduce complexity
+  * Added ability to rename files in PCManFM by clicking name when selected
+  * Bug fix in Bluetooth ALSA module to reduce truncation of audio at end of playback
+  * Various small tweaks, bug fixes and theme modifications
+  * New kernel and firmware
+2017-09-07:
+  * Disable predictable network interface names for Ethernet devices
+  * Bug fix for keyboard settings dialog in Raspberry Pi Configuration
+  * Bug fix for crash on some videos and animations in Chromium
+  * Bug fix for taskbar crash when running RealVNC server
+  * Bug fix for reloading projects with extensions in Scratch 2
+  * Bug fix for MAC address problem in Bluetooth
+  * Simple mode and new icons in Thonny
+  * New Japanese translations in Raspberry Pi Configuration
+  * Install fonts-droid-fallback for international fonts
+2017-08-16:
   * Based on Raspbian Stretch (Debian version 9)
   * Version 60 of Chromium browser included
   * Version 3.0.1 of Sonic Pi included
@@ -9,6 +89,7 @@ UNRELEASED:
   * Various desktop applications modified to prompt for sudo password if needed
   * lxinput control options for mouse speed simplified
   * lxpanel plugins moved into separate packages
+  * Wireless firmware for Pi 3 and Pi 0W modified to address Broadpwn exploit
   * Latest kernel and firmware
   * Various small tweaks, bug fixes and theme modifications
 2017-07-05:

export-noobs/prerun.sh

@@ -2,14 +2,14 @@
 
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-mkdir -p ${STAGE_WORK_DIR}
+mkdir -p "${STAGE_WORK_DIR}"
-cp ${WORK_DIR}/export-image/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img ${STAGE_WORK_DIR}/
+cp "${WORK_DIR}/export-image/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img" "${STAGE_WORK_DIR}/"
 
-rm -rf ${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}
+rm -rf "${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
 
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
+PARTED_OUT=$(parted -s "${IMG_FILE}" unit b print)
 BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
 | cut -d" " -f 2 | tr -d B)
 BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
@@ -20,21 +20,21 @@ ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 | cut -d" " -f 4 | tr -d B)
 
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
-mkdir -p ${STAGE_WORK_DIR}/rootfs
+mkdir -p "${STAGE_WORK_DIR}/rootfs"
-mkdir -p ${NOOBS_DIR}
+mkdir -p "${NOOBS_DIR}"
 
-mount $ROOT_DEV ${STAGE_WORK_DIR}/rootfs
+mount "$ROOT_DEV" "${STAGE_WORK_DIR}/rootfs"
-mount $BOOT_DEV ${STAGE_WORK_DIR}/rootfs/boot
+mount "$BOOT_DEV" "${STAGE_WORK_DIR}/rootfs/boot"
 
 ln -sv "/lib/systemd/system/apply_noobs_os_config.service" "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
 
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs/boot -cpf ${NOOBS_DIR}/boot.tar.xz .
+bsdtar --numeric-owner --format gnutar --use-compress-program pxz -C "${STAGE_WORK_DIR}/rootfs/boot" -cpf "${NOOBS_DIR}/boot.tar.xz" .
-umount ${STAGE_WORK_DIR}/rootfs/boot
+umount "${STAGE_WORK_DIR}/rootfs/boot"
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs --one-file-system -cpf ${NOOBS_DIR}/root.tar.xz .
+bsdtar --numeric-owner --format gnutar --use-compress-program pxz -C "${STAGE_WORK_DIR}/rootfs" --one-file-system -cpf "${NOOBS_DIR}/root.tar.xz" .
 
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"

scripts/common

@@ -1,5 +1,5 @@
 log (){
-	date +"[%T] $@" | tee -a "${LOG_FILE}"
+	date +"[%T] $*" | tee -a "${LOG_FILE}"
 }
 export -f log
 
@@ -15,10 +15,10 @@ bootstrap(){
 		local BOOTSTRAP_CMD=debootstrap
 	fi
 
-	capsh --drop=cap_setfcap -- -c "${BOOTSTRAP_CMD} --components=main,contrib,non-free \
+	capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" --components=main,contrib,non-free \
 		--arch armhf \
 		--keyring "${STAGE_DIR}/files/raspberrypi.gpg" \
-		$1 $2 $3" || rmdir "$2/debootstrap"
+		"$1" "$2" "$3" || rmdir "$2/debootstrap"
 }
 export -f bootstrap
 

scripts/dependencies_check

@@ -10,11 +10,11 @@ dependencies_check()
 
 	for depfile in "$@"; do
 		if [[ -e "$depfile" ]]; then
-			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${BASE_DIR}/depends)"
+			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${BASE_DIR}/depends")"
 
 		fi
 		for dep in $deps; do
-			if ! hash ${dep%:*} 2>/dev/null; then
+			if ! hash "${dep%:*}" 2>/dev/null; then
 				missing="${missing:+$missing }${dep#*:}"
 			fi
 		done

stage0/00-configure-apt/00-run.sh

@@ -1,13 +1,13 @@
 #!/bin/bash -e
 
-install -m 644 files/sources.list ${ROOTFS_DIR}/etc/apt/
+install -m 644 files/sources.list "${ROOTFS_DIR}/etc/apt/"
-install -m 644 files/raspi.list ${ROOTFS_DIR}/etc/apt/sources.list.d/
+install -m 644 files/raspi.list "${ROOTFS_DIR}/etc/apt/sources.list.d/"
 
 if [ -n "$APT_PROXY" ]; then
-	install -m 644 files/51cache ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	install -m 644 files/51cache "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-	sed ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache -i -e "s|APT_PROXY|${APT_PROXY}|"
+	sed "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache" -i -e "s|APT_PROXY|${APT_PROXY}|"
 else
-	rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
 fi
 
 on_chroot apt-key add - < files/raspberrypi.gpg.key

stage0/00-configure-apt/files/sources.list

@@ -1,3 +1,3 @@
-deb http://mirrordirector.raspbian.org/raspbian/ stretch main contrib non-free rpi
+deb http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspbian.org/raspbian/ stretch main contrib non-free rpi
+#deb-src http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi

stage0/02-firmware/01-packages

@@ -1 +1,2 @@
 raspberrypi-bootloader
+raspberrypi-kernel

stage0/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
-	bootstrap stretch ${ROOTFS_DIR} http://mirrordirector.raspbian.org/raspbian/
+	bootstrap stretch "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
 fi

stage1/00-boot-files/00-run.sh

@@ -1,4 +1,4 @@
 #!/bin/bash -e
 
-install -m 644 files/cmdline.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/cmdline.txt "${ROOTFS_DIR}/boot/"
-install -m 644 files/config.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/config.txt "${ROOTFS_DIR}/boot/"

stage1/01-sys-tweaks/00-run.sh

@@ -1,9 +1,8 @@
 #!/bin/bash -e
 
-install -d ${ROOTFS_DIR}/etc/systemd/system/[email protected]
+install -d "${ROOTFS_DIR}/etc/systemd/system/[email protected]"
-install -m 644 files/noclear.conf ${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf
+install -m 644 files/noclear.conf "${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf"
-install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/policy-rc.d #TODO: Necessary in systemd?
+install -v -m 644 files/fstab "${ROOTFS_DIR}/etc/fstab"
-install -v -m 644 files/fstab ${ROOTFS_DIR}/etc/fstab
 
 on_chroot << EOF
 if ! id -u pi >/dev/null 2>&1; then

stage1/01-sys-tweaks/files/policy-rc.d

@@ -1,3 +0,0 @@
-#!/bin/sh
-
-exit 101

stage1/02-net-tweaks/00-packages

@@ -0,0 +1 @@
+netbase

stage1/02-net-tweaks/00-run.sh

@@ -1,8 +1,6 @@
 #!/bin/bash -e
 
-install -m 644 files/ipv6.conf ${ROOTFS_DIR}/etc/modprobe.d/ipv6.conf
+install -m 644 files/ipv6.conf "${ROOTFS_DIR}/etc/modprobe.d/ipv6.conf"
-install -m 644 files/hostname ${ROOTFS_DIR}/etc/hostname
+install -m 644 files/hostname "${ROOTFS_DIR}/etc/hostname"
 
-on_chroot << EOF
+ln -sf /dev/null "${ROOTFS_DIR}/etc/systemd/network/99-default.link"
-dpkg-divert --add --local /lib/udev/rules.d/75-persistent-net-generator.rules
-EOF

stage1/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage2/00-copies-and-fills/00-run.sh

@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-touch ${ROOTFS_DIR}/spindle_install

stage2/00-copies-and-fills/02-run.sh

@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-rm -f ${ROOTFS_DIR}/spindle_install

stage2/01-sys-tweaks/00-packages

@@ -20,3 +20,5 @@ rpi-update
 libmtp-runtime
 rsync
 htop
+man-db
+policykit-1

stage2/01-sys-tweaks/00-patches/06-rc_local.diff

@@ -1,15 +0,0 @@
-Index: jessie-stage2/rootfs/etc/rc.local
-===================================================================
---- jessie-stage2.orig/rootfs/etc/rc.local
-+++ jessie-stage2/rootfs/etc/rc.local
-@@ -11,4 +11,10 @@
- #
- # By default this script does nothing.
- 
-+# Print the IP address
-+_IP=$(hostname -I) || true
-+if [ "$_IP" ]; then
-+  printf "My IP address is %s\n" "$_IP"
-+fi
-+
- exit 0

stage2/01-sys-tweaks/01-run.sh

@@ -1,15 +1,15 @@
 #!/bin/bash -e
 
-install -m 755 files/resize2fs_once			${ROOTFS_DIR}/etc/init.d/
+install -m 755 files/resize2fs_once	"${ROOTFS_DIR}/etc/init.d/"
 
-install -d						${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d
+install -d				"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d"
-install -m 644 files/ttyoutput.conf			${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/
+install -m 644 files/ttyoutput.conf	"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/"
 
-install -m 644 files/50raspi				${ROOTFS_DIR}/etc/apt/apt.conf.d/
+install -m 644 files/50raspi		"${ROOTFS_DIR}/etc/apt/apt.conf.d/"
 
-install -m 644 files/console-setup   			${ROOTFS_DIR}/etc/default/
+install -m 644 files/console-setup   	"${ROOTFS_DIR}/etc/default/"
 
-install -m 755 files/rc.local				${ROOTFS_DIR}/etc/
+install -m 755 files/rc.local		"${ROOTFS_DIR}/etc/"
 
 on_chroot << EOF
 systemctl disable hwclock.sh
@@ -17,12 +17,24 @@ systemctl disable nfs-common
 systemctl disable rpcbind
 systemctl disable ssh
 systemctl enable regenerate_ssh_host_keys
+EOF
+
+if [ "${USE_QEMU}" = "1" ]; then
+	echo "enter QEMU mode"
+	install -m 644 files/90-qemu.rules "${ROOTFS_DIR}/etc/udev/rules.d/"
+	on_chroot << EOF
+systemctl disable resize2fs_once
+EOF
+	echo "leaving QEMU mode"
+else
+	on_chroot << EOF
 systemctl enable resize2fs_once
 EOF
+fi
 
 on_chroot << \EOF
 for GRP in input spi i2c gpio; do
-	groupadd -f -r $GRP
+	groupadd -f -r "$GRP"
 done
 for GRP in adm dialout cdrom audio users sudo video games plugdev input gpio spi i2c netdev; do
   adduser pi $GRP
@@ -37,4 +49,4 @@ on_chroot << EOF
 usermod --pass='*' root
 EOF
 
-rm -f ${ROOTFS_DIR}/etc/ssh/ssh_host_*_key*
+rm -f "${ROOTFS_DIR}/etc/ssh/"ssh_host_*_key*

stage2/01-sys-tweaks/files/90-qemu.rules

@@ -0,0 +1,3 @@
+KERNEL=="sda", SYMLINK+="mmcblk0"
+KERNEL=="sda?", SYMLINK+="mmcblk0p%n"
+KERNEL=="sda2", SYMLINK+="root"

stage2/02-net-tweaks/00-packages

@@ -1,3 +1,4 @@
 wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek
 raspberrypi-net-mods
 dhcpcd5
+net-tools

stage2/02-net-tweaks/01-run.sh

@@ -1,8 +1,8 @@
 #!/bin/bash -e
 
-install -v -d						${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d
+install -v -d					"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d"
-install -v -m 644 files/wait.conf			${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/
+install -v -m 644 files/wait.conf		"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/"
 
-install -v -d                                           ${ROOTFS_DIR}/etc/wpa_supplicant
+install -v -d					"${ROOTFS_DIR}/etc/wpa_supplicant"
-install -v -m 600 files/wpa_supplicant.conf             ${ROOTFS_DIR}/etc/wpa_supplicant/
+install -v -m 600 files/wpa_supplicant.conf	"${ROOTFS_DIR}/etc/wpa_supplicant/"
 

stage2/02-net-tweaks/files/wpa_supplicant.conf

@@ -1,3 +1,2 @@
-country=GB
 ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
 update_config=1

stage2/EXPORT_IMAGE

@@ -1 +1,4 @@
 IMG_SUFFIX="-lite"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage2/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage3/00-install-packages/00-packages

@@ -1,5 +1,5 @@
 gstreamer1.0-x gstreamer1.0-omx gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-alsa gstreamer1.0-libav
-xpdf gtk2-engines alsa-utils
+qpdfview gtk2-engines alsa-utils
 desktop-base
 git
 omxplayer
@@ -9,3 +9,6 @@ gvfs
 rfkill
 chromium-browser rpi-chromium-mods
 gldriver-test
+fonts-droid-fallback
+fonts-liberation2
+obconf

stage3/01-tweaks/00-run.sh

@@ -1,3 +1,3 @@
 #!/bin/bash -e
 
-rm -f ${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf
+rm -f "${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf"

stage3/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage4/00-install-packages/00-packages

@@ -8,18 +8,14 @@ python-serial python3-serial
 python-picamera python3-picamera
 debian-reference-en dillo x2x
 raspberrypi-net-mods raspberrypi-ui-mods
-fonts-liberation2
 smartsim
 python-pip python3-pip
 python3-numpy
 pypy
 minecraft-pi python-minecraftpi
 alacarte rc-gui sense-hat
-claws-mail
 tree
 scratch nuscratch
-greenfoot bluej
-nodered
 libgl1-mesa-dri libgles1-mesa libgles2-mesa xcompmgr
 geany
 piclone
@@ -33,3 +29,5 @@ python-sense-emu python3-sense-emu sense-emu-tools python-sense-emu-doc
 pprompt
 scratch2
 python3-thonny
+piwiz
+rp-prefapps

stage4/00-install-packages/00-packages-nr

@@ -1,4 +1,4 @@
-libreoffice libreoffice-gtk
+libreoffice-pi
 timidity
 pi-package
-realvnc-vnc-server realvnc-vnc-viewer
+realvnc-vnc-server

stage4/00-install-packages/01-packages

@@ -17,3 +17,6 @@ python-scrollphathd python3-scrollphathd
 python-sn3218 python3-sn3218
 python-skywriter python3-skywriter
 python-touchphat python3-touchphat
+python-buttonshim python3-buttonshim
+python-unicornhathd  python3-unicornhathd
+python-pantilthat python3-pantilthat

stage4/01-console-autologin/00-run.sh

@@ -1,3 +1,4 @@
 #!/bin/bash -e
 
-ln -sf /etc/systemd/system/[email protected] ${ROOTFS_DIR}/etc/systemd/system/getty.target.wants/[email protected]
+ln -sf /etc/systemd/system/[email protected] \
+	"${ROOTFS_DIR}/etc/systemd/system/getty.target.wants/[email protected]"

stage4/02-extras/00-run.sh

@@ -1,30 +1,22 @@
 #!/bin/bash -e
 
-HASH=`wget https://api.github.com/repos/KenT2/python-games/git/refs/heads/master -qO -| grep \"sha\" | cut -f 2 -d ':' | cut -f 2 -d \"`
+HASH="$(wget https://api.github.com/repos/KenT2/python-games/git/refs/heads/master -qO -| grep \"sha\" | cut -f 2 -d ':' | cut -f 2 -d \")"
 
 if [ -f files/python_games.hash ]; then
-	HASH_LOCAL=`cat files/python_games.hash`
+	HASH_LOCAL="$(cat files/python_games.hash)"
 fi
 
 if [ ! -e files/python_games.tar.gz ] || [ "$HASH" != "$HASH_LOCAL"  ]; then
 	wget "https://github.com/KenT2/python-games/tarball/master" -O files/python_games.tar.gz
-	echo $HASH > files/python_games.hash
+	echo "$HASH" > files/python_games.hash
 fi
 
-ln -sf pip3 ${ROOTFS_DIR}/usr/bin/pip-3.2
+ln -sf pip3 "${ROOTFS_DIR}/usr/bin/pip-3.2"
 
-install -v -o 1000 -g 1000 -d ${ROOTFS_DIR}/home/pi/python_games
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/python_games"
-tar xvf files/python_games.tar.gz -C ${ROOTFS_DIR}/home/pi/python_games --strip-components=1
+tar xvf files/python_games.tar.gz -C "${ROOTFS_DIR}/home/pi/python_games" --strip-components=1
-chown 1000:1000 ${ROOTFS_DIR}/home/pi/python_games -Rv
+chown 1000:1000 "${ROOTFS_DIR}/home/pi/python_games" -Rv
-chmod +x ${ROOTFS_DIR}/home/pi/python_games/launcher.sh
+chmod +x "${ROOTFS_DIR}/home/pi/python_games/launcher.sh"
-
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/BlueJ/ "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/Greenfoot/ "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/scratch/Projects/Demos/ "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
 
 #Alacarte fixes
 install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local"

stage4/03-magpi/00-run.sh

@@ -0,0 +1,16 @@
+#!/bin/sh -e
+
+magpi_regex="MagPi[[:digit:]]*.pdf"
+magpi_loc="https://www.raspberrypi.org/magpi-issues"
+magpi_latest="$(curl "$magpi_loc/?C=M;O=D" -s | grep "$magpi_regex" -m 1 -o | head -n 1)"
+
+if [ ! -f "files/$magpi_latest" ]; then
+	find files/ -regextype grep -regex "files/$magpi_regex" -delete
+	wget "$magpi_loc/$magpi_latest" -O "files/$magpi_latest"
+fi
+
+file "files/$magpi_latest" | grep -q "PDF document"
+
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/MagPi"
+install -v -o 1000 -g 1000 -m 644 "files/$magpi_latest" "${ROOTFS_DIR}/home/pi/MagPi/"
+

stage4/03-magpi/files/.gitignore

@@ -0,0 +1 @@
+*.pdf

stage4/EXPORT_IMAGE

@@ -1 +1,4 @@
 IMG_SUFFIX="-4GB"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage4/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage5/EXPORT_IMAGE

@@ -1 +1,4 @@
 IMG_SUFFIX=""
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage5/prerun.sh

@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
-if [ ! -d ${ROOTFS_DIR} ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi