Update release notes

Move python-games to Raspbian Full and install as a Debian package.

.dockerignore

@@ -1,3 +1,5 @@
 output/
 work/
 deploy/
+apt-cacher-ng/
+.git/objects/*

.gitignore

@@ -3,5 +3,7 @@ work/*
 config
 postrun.sh
 SKIP
+SKIP_IMAGES
 .pc
 *-pc
+apt-cacher-ng/

Dockerfile

@@ -1,10 +1,12 @@
-FROM debian:jessie
+FROM debian:stretch
+
+ENV DEBIAN_FRONTEND noninteractive
 
 RUN apt-get -y update && \
     apt-get -y install \
         git vim parted \
         quilt realpath qemu-user-static debootstrap zerofree pxz zip dosfstools \
-        bsdtar libcap2-bin rsync grep \
+        bsdtar libcap2-bin rsync grep udev xz-utils curl xxd file \
     && rm -rf /var/lib/apt/lists/*
 
 COPY . /pi-gen/

README.md

@@ -3,17 +3,17 @@
 _Tool used to create the raspberrypi.org Raspbian images_
 
 
-### TODO
-1. Documentation
-
-
 ## Dependencies
 
-On Debian-based systems:
+pi-gen runs on Debian based operating systems. Currently it is only supported on
+either Debian Stretch or Ubuntu Xenial and is known to have issues building on
+earlier releases of these systems.
+
+To install the required dependencies for pi-gen you should run:
 
 ```bash
 apt-get install quilt parted realpath qemu-user-static debootstrap zerofree pxz zip \
-dosfstools bsdtar libcap2-bin grep rsync
+dosfstools bsdtar libcap2-bin grep rsync xz-utils file git curl
 ```
 
 The file `depends` contains a list of tools needed.  The format of this
@@ -41,6 +41,12 @@ The following environment variables are supported:
    will not be included in the image, making it safe to use an `apt-cacher` or
    similar package for development.
 
+   If you have Docker installed, you can set up a local apt caching proxy to
+   like speed up subsequent builds like this:
+
+       docker-compose up -d
+       echo 'APT_PROXY=http://172.17.0.1:3142' >> config
+
  * `BASE_DIR`  (Default: location of `build.sh`)
 
    **CAUTION**: Currently, changing this value will probably break build.sh
@@ -56,10 +62,17 @@ The following environment variables are supported:
    system for each build stage, amounting to tens of gigabytes in the case of
    Raspbian.
    
+   **CAUTION**: If your working directory is on an NTFS partition you probably won't be able to build. Make sure this is a proper Linux filesystem.
+
  * `DEPLOY_DIR`  (Default: `"$BASE_DIR/deploy"`)
 
    Output directory for target system images and NOOBS bundles.
 
+ * `USE_QEMU` (Default: `"0"`)
+
+   Setting to '1' enables the QEMU mode - creating an image that can be mounted via QEMU for an emulated
+   environment. These images include "-qemu" in the image file name.
+
 
 A simple example for building Raspbian:
 
@@ -68,6 +81,51 @@ IMG_NAME='Raspbian'
 ```
 
 
+## How the build process works
+
+The following process is followed to build images:
+
+ * Loop through all of the stage directories in alphanumeric order
+
+ * Move on to the next directory if this stage directory contains a file called
+   "SKIP"
+
+ * Run the script ```prerun.sh``` which is generally just used to copy the build
+   directory between stages.
+
+ * In each stage directory loop through each subdirectory and then run each of the
+   install scripts it contains, again in alphanumeric order. These need to be named
+   with a two digit padded number at the beginning.
+   There are a number of different files and directories which can be used to
+   control different parts of the build process:
+
+     - **00-run.sh** - A unix shell script. Needs to be made executable for it to run.
+
+     - **00-run-chroot.sh** - A unix shell script which will be run in the chroot
+       of the image build directory. Needs to be made executable for it to run.
+
+     - **00-debconf** - Contents of this file are passed to debconf-set-selections
+       to configure things like locale, etc.
+
+     - **00-packages** - A list of packages to install. Can have more than one, space
+       separated, per line.
+
+     - **00-packages-nr** - As 00-packages, except these will be installed using
+       the ```--no-install-recommends -y``` parameters to apt-get.
+
+     - **00-patches** - A directory containing patch files to be applied, using quilt.
+       If a file named 'EDIT' is present in the directory, the build process will
+       be interrupted with a bash session, allowing an opportunity to create/revise
+       the patches.
+
+  * If the stage directory contains files called "EXPORT_NOOBS" or "EXPORT_IMAGE" then
+    add this stage to a list of images to generate
+
+  * Generate the images for any stages that have specified them
+
+It is recommended to examine build.sh for finer details.
+
+
 ## Docker Build
 
 ```bash
@@ -85,6 +143,12 @@ continue:
 CONTINUE=1 ./build-docker.sh
 ```
 
+After successful build, the build container is by default removed. This may be undesired when making incremental changes to a customized build. To prevent the build script from remove the container add
+
+```bash
+PRESERVE_CONTAINER=1 ./build-docker.sh
+```
+
 There is a possibility that even when running from a docker container, the
 installation of `qemu-user-static` will silently fail when building the image
 because `binfmt-support` _must be enabled on the underlying kernel_. An easy
@@ -126,11 +190,11 @@ maintenance and allows for more easy customization.
    standard console hardware permission groups.
 
    There are a few tools that may not make a whole lot of sense here for
-   development purposes on a minimal system such as basic python and lua
+   development purposes on a minimal system such as basic Python and Lua
    packages as well as the `build-essential` package.  They are lumped right
    in with more essential packages presently, though they need not be with
    pi-gen.  These are understandable for Raspbian's target audience, but if
-   you were looking for something between truly minimal and Raspbian-lite,
+   you were looking for something between truly minimal and Raspbian-Lite,
    here's where you start trimming.
 
  - **Stage 3** - desktop system.  Here's where you get the full desktop system
@@ -153,17 +217,58 @@ If you wish to build up to a specified stage (such as building up to stage 2
 for a lite system), place an empty file named `SKIP` in each of the `./stage`
 directories you wish not to include.
 
-Then remove the `EXPORT*` files from `./stage4` (if building up to stage 2) or
+Then add an empty file named `SKIP_IMAGES` to `./stage4` (if building up to stage 2) or
-from `./stage2` (if building a minimal system).
+to `./stage2` (if building a minimal system).
 
 ```bash
 # Example for building a lite system
 echo "IMG_NAME='Raspbian'" > config
 touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
-rm stage4/EXPORT*
+touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
 sudo ./build.sh  # or ./build-docker.sh
 ```
 
 If you wish to build further configurations upon (for example) the lite
 system, you can also delete the contents of `./stage3` and `./stage4` and
 replace with your own contents in the same format.
+
+
+## Skipping stages to speed up development
+
+If you're working on a specific stage the recommended development process is as
+follows:
+
+ * Add a file called SKIP_IMAGES into the directories containing EXPORT_* files
+   (currently stage2, stage4 and stage5)
+ * Add SKIP files to the stages you don't want to build. For example, if you're
+   basing your image on the lite image you would add these to stages 3, 4 and 5.
+ * Run build.sh to build all stages
+ * Add SKIP files to the earlier successfully built stages
+ * Modify the last stage
+ * Rebuild just the last stage using ```sudo CLEAN=1 ./build.sh```
+ * Once you're happy with the image you can remove the SKIP_IMAGES files and
+   export your image to test
+
+# Troubleshooting
+
+## `binfmt_misc`
+
+Linux is able execute binaries from other architectures, meaning that it should be
+possible to make use of `pi-gen` on an x86_64 system, even though it will be running
+ARM binaries. This requires support from the [`binfmt_misc`](https://en.wikipedia.org/wiki/Binfmt_misc)
+kernel module.
+
+You may see the following error:
+
+```
+update-binfmts: warning: Couldn't load the binfmt_misc module.
+```
+
+To resolve this, ensure that the following files are available (install them if necessary):
+
+```
+/lib/modules/$(uname -r)/kernel/fs/binfmt_misc.ko
+/usr/bin/qemu-arm-static
+```
+
+You may also need to load the module by hand - run `modprobe binfmt_misc`.

build-docker.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
+
 DOCKER="docker"
 set +e
 $DOCKER ps >/dev/null 2>&1
@@ -12,18 +13,19 @@ if ! $DOCKER ps >/dev/null; then
 fi
 set -e
 
-config_mount="/dev/null:/dev/null"
+config_file=()
 if [ -f config ]; then
-	config_mount="$(pwd)/config:/pi-gen/config:ro"
+	config_file=("--env-file" "$(pwd)/config")
 	source config
 fi
 
 CONTAINER_NAME=${CONTAINER_NAME:-pigen_work}
 CONTINUE=${CONTINUE:-0}
+PRESERVE_CONTAINER=${PRESERVE_CONTAINER:-0}
 
 if [ "$*" != "" ] || [ -z "${IMG_NAME}" ]; then
 	if [ -z "${IMG_NAME}" ]; then
-		echo "IMG_NAME not set in 'build'" 1>&2
+		echo "IMG_NAME not set in 'config'" 1>&2
 		echo 1>&2
 	fi
 	cat >&2 <<EOF
@@ -31,7 +33,8 @@ Usage:
     build-docker.sh [options]
 Optional environment arguments: ( =<default> )
     CONTAINER_NAME=pigen_work  set a name for the build container
-    CONTINUE=0                  continue from a previously started container
+    CONTINUE=1                 continue from a previously started container
+    PRESERVE_CONTAINER=1       keep build container even on successful build
 EOF
 	exit 1
 fi
@@ -45,31 +48,39 @@ fi
 if [ "$CONTAINER_EXISTS" != "" ] && [ "$CONTINUE" != "1" ]; then
 	echo "Container $CONTAINER_NAME already exists and you did not specify CONTINUE=1. Aborting."
 	echo "You can delete the existing container like this:"
-	echo "  docker rm -v $CONTAINER_NAME"
+	echo "  $DOCKER rm -v $CONTAINER_NAME"
 	exit 1
 fi
 
 $DOCKER build -t pi-gen .
 if [ "$CONTAINER_EXISTS" != "" ]; then
-	trap "echo 'got CTRL+C... please wait 5s';docker stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
+	trap "echo 'got CTRL+C... please wait 5s'; $DOCKER stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
 	time $DOCKER run --rm --privileged \
 		--volumes-from="${CONTAINER_NAME}" --name "${CONTAINER_NAME}_cont" \
-		-e IMG_NAME=${IMG_NAME}\
+		-e IMG_NAME="${IMG_NAME}"\
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh;
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 else
-	trap "echo 'got CTRL+C... please wait 5s'; docker stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
+	trap "echo 'got CTRL+C... please wait 5s'; $DOCKER stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
-	$DOCKER run --name "${CONTAINER_NAME}" --privileged \
+	time $DOCKER run --name "${CONTAINER_NAME}" --privileged \
-		-e IMG_NAME=${IMG_NAME}\
+		-e IMG_NAME="${IMG_NAME}"\
-		-v "$(pwd)/deploy:/pi-gen/deploy" \
+		"${config_file[@]}" \
-		-v "${config_mount}" \
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh &&
 	rsync -av work/*/build.log deploy/" &
-	wait
+	wait "$!"
 fi
+echo "copying results from deploy/"
+$DOCKER cp "${CONTAINER_NAME}":/pi-gen/deploy .
+ls -lah deploy
+
+# cleanup
+if [ "$PRESERVE_CONTAINER" != "1" ]; then
+	$DOCKER rm -v $CONTAINER_NAME
+fi
+
 echo "Done! Your image(s) should be in deploy/"

build.sh

@@ -1,22 +1,23 @@
 #!/bin/bash -e
-
+# shellcheck disable=SC2119,SC1091
 run_sub_stage()
 {
 	log "Begin ${SUB_STAGE_DIR}"
-	pushd ${SUB_STAGE_DIR} > /dev/null
+	pushd "${SUB_STAGE_DIR}" > /dev/null
 	for i in {00..99}; do
-		if [ -f ${i}-debconf ]; then
+		if [ -f "${i}-debconf" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-debconf"
 			on_chroot << EOF
 debconf-set-selections <<SELEOF
-`cat ${i}-debconf`
+$(cat "${i}-debconf")
 SELEOF
 EOF
+
 		log "End ${SUB_STAGE_DIR}/${i}-debconf"
 		fi
-		if [ -f ${i}-packages-nr ]; then
+		if [ -f "${i}-packages-nr" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages-nr"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages-nr)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages-nr")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install --no-install-recommends -y $PACKAGES
@@ -24,9 +25,9 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages-nr"
 		fi
-		if [ -f ${i}-packages ]; then
+		if [ -f "${i}-packages" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-packages"
-			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages)"
+			PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${i}-packages")"
 			if [ -n "$PACKAGES" ]; then
 				on_chroot << EOF
 apt-get install -y $PACKAGES
@@ -34,18 +35,18 @@ EOF
 			fi
 			log "End ${SUB_STAGE_DIR}/${i}-packages"
 		fi
-		if [ -d ${i}-patches ]; then
+		if [ -d "${i}-patches" ]; then
 			log "Begin ${SUB_STAGE_DIR}/${i}-patches"
-			pushd ${STAGE_WORK_DIR} > /dev/null
+			pushd "${STAGE_WORK_DIR}" > /dev/null
 			if [ "${CLEAN}" = "1" ]; then
 				rm -rf .pc
-				rm -rf *-pc
+				rm -rf ./*-pc
 			fi
-			QUILT_PATCHES=${SUB_STAGE_DIR}/${i}-patches
+			QUILT_PATCHES="${SUB_STAGE_DIR}/${i}-patches"
-			SUB_STAGE_QUILT_PATCH_DIR="$(basename $SUB_STAGE_DIR)-pc"
+			SUB_STAGE_QUILT_PATCH_DIR="$(basename "$SUB_STAGE_DIR")-pc"
-			mkdir -p $SUB_STAGE_QUILT_PATCH_DIR
+			mkdir -p "$SUB_STAGE_QUILT_PATCH_DIR"
-			ln -snf $SUB_STAGE_QUILT_PATCH_DIR .pc
+			ln -snf "$SUB_STAGE_QUILT_PATCH_DIR" .pc
-			if [ -e ${SUB_STAGE_DIR}/${i}-patches/EDIT ]; then
+			if [ -e "${SUB_STAGE_DIR}/${i}-patches/EDIT" ]; then
 				echo "Dropping into bash to edit patches..."
 				bash
 			fi
@@ -80,18 +81,20 @@ EOF
 
 run_stage(){
 	log "Begin ${STAGE_DIR}"
-	STAGE=$(basename ${STAGE_DIR})
+	STAGE="$(basename "${STAGE_DIR}")"
-	pushd ${STAGE_DIR} > /dev/null
+	pushd "${STAGE_DIR}" > /dev/null
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	STAGE_WORK_DIR=${WORK_DIR}/${STAGE}
+	STAGE_WORK_DIR="${WORK_DIR}/${STAGE}"
-	ROOTFS_DIR=${STAGE_WORK_DIR}/rootfs
+	ROOTFS_DIR="${STAGE_WORK_DIR}"/rootfs
-	if [ -f ${STAGE_DIR}/EXPORT_IMAGE ]; then
+	if [ ! -f SKIP_IMAGES ]; then
-		EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
+		if [ -f "${STAGE_DIR}/EXPORT_IMAGE" ]; then
+			EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
+		fi
 	fi
 	if [ ! -f SKIP ]; then
 		if [ "${CLEAN}" = "1" ]; then
-			if [ -d ${ROOTFS_DIR} ]; then
+			if [ -d "${ROOTFS_DIR}" ]; then
-				rm -rf ${ROOTFS_DIR}
+				rm -rf "${ROOTFS_DIR}"
 			fi
 		fi
 		if [ -x prerun.sh ]; then
@@ -100,16 +103,16 @@ run_stage(){
 			log "End ${STAGE_DIR}/prerun.sh"
 		fi
 		for SUB_STAGE_DIR in ${STAGE_DIR}/*; do
-			if [ -d ${SUB_STAGE_DIR} ] &&
+			if [ -d "${SUB_STAGE_DIR}" ] &&
-			   [ ! -f ${SUB_STAGE_DIR}/SKIP ]; then
+			   [ ! -f "${SUB_STAGE_DIR}/SKIP" ]; then
 				run_sub_stage
 			fi
 		done
 	fi
-	unmount ${WORK_DIR}/${STAGE}
+	unmount "${WORK_DIR}/${STAGE}"
-	PREV_STAGE=${STAGE}
+	PREV_STAGE="${STAGE}"
-	PREV_STAGE_DIR=${STAGE_DIR}
+	PREV_STAGE_DIR="${STAGE_DIR}"
-	PREV_ROOTFS_DIR=${ROOTFS_DIR}
+	PREV_ROOTFS_DIR="${ROOTFS_DIR}"
 	popd > /dev/null
 	log "End ${STAGE_DIR}"
 }
@@ -119,6 +122,7 @@ if [ "$(id -u)" != "0" ]; then
 	exit 1
 fi
 
+
 if [ -f config ]; then
 	source config
 fi
@@ -128,14 +132,17 @@ if [ -z "${IMG_NAME}" ]; then
 	exit 1
 fi
 
-export IMG_DATE=${IMG_DATE:-"$(date +%Y-%m-%d)"}
+export USE_QEMU="${USE_QEMU:-0}"
+export IMG_DATE="${IMG_DATE:-"$(date +%Y-%m-%d)"}"
 
-export BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 export SCRIPT_DIR="${BASE_DIR}/scripts"
-export WORK_DIR=${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}
+export WORK_DIR="${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}"
 export DEPLOY_DIR=${DEPLOY_DIR:-"${BASE_DIR}/deploy"}
 export LOG_FILE="${WORK_DIR}/build.log"
 
+export BASE_DIR
+
 export CLEAN
 export IMG_NAME
 export APT_PROXY
@@ -158,29 +165,35 @@ export QUILT_NO_DIFF_INDEX=1
 export QUILT_NO_DIFF_TIMESTAMPS=1
 export QUILT_REFRESH_ARGS="-p ab"
 
-source ${SCRIPT_DIR}/common
+# shellcheck source=scripts/common
-source ${SCRIPT_DIR}/dependencies_check
+source "${SCRIPT_DIR}/common"
+# shellcheck source=scripts/dependencies_check
+source "${SCRIPT_DIR}/dependencies_check"
 
 
-dependencies_check ${BASE_DIR}/depends
+dependencies_check "${BASE_DIR}/depends"
 
-mkdir -p ${WORK_DIR}
+mkdir -p "${WORK_DIR}"
 log "Begin ${BASE_DIR}"
 
-for STAGE_DIR in ${BASE_DIR}/stage*; do
+for STAGE_DIR in "${BASE_DIR}/stage"*; do
 	run_stage
 done
 
 CLEAN=1
 for EXPORT_DIR in ${EXPORT_DIRS}; do
 	STAGE_DIR=${BASE_DIR}/export-image
+	# shellcheck source=/dev/null
 	source "${EXPORT_DIR}/EXPORT_IMAGE"
-	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename ${EXPORT_DIR})/rootfs
+	EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename "${EXPORT_DIR}")/rootfs
 	run_stage
-	if [ -e ${EXPORT_DIR}/EXPORT_NOOBS ]; then
+	if [ "${USE_QEMU}" != "1" ]; then
-		source ${EXPORT_DIR}/EXPORT_NOOBS
+		if [ -e "${EXPORT_DIR}/EXPORT_NOOBS" ]; then
-		STAGE_DIR=${BASE_DIR}/export-noobs
+			# shellcheck source=/dev/null
-		run_stage
+			source "${EXPORT_DIR}/EXPORT_NOOBS"
+			STAGE_DIR="${BASE_DIR}/export-noobs"
+			run_stage
+		fi
 	fi
 done
 

depends

@@ -10,3 +10,8 @@ capsh:libcap2-bin
 bsdtar
 grep
 rsync
+xz:xz-utils
+curl
+xxd
+file
+git

docker-compose.yml

@@ -0,0 +1,10 @@
+version: '2'
+
+services:
+  apt-cacher-ng:
+    restart: unless-stopped
+    image: sameersbn/apt-cacher-ng:latest
+    ports:
+    - "3142:3142"
+    volumes:
+    - ./apt-cacher-ng:/var/cache/apt-cacher-ng

export-image/00-allow-rerun/00-run.sh

@@ -1,13 +1,5 @@
 #!/bin/bash -e
 
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload ]; then
+if [ ! -x "${ROOTFS_DIR}/usr/bin/qemu-arm-static" ]; then
-	mv ${ROOTFS_DIR}/etc/ld.so.preload ${ROOTFS_DIR}/etc/ld.so.preload.disabled
+	cp /usr/bin/qemu-arm-static "${ROOTFS_DIR}/usr/bin/"
-fi
-
-if [ ! -e ${ROOTFS_DIR}/usr/sbin/policy-rc.d ]; then
-	install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/
-fi
-
-if [ ! -x ${ROOTFS_DIR}/usr/bin/qemu-arm-static ]; then
-	cp /usr/bin/qemu-arm-static ${ROOTFS_DIR}/usr/bin/
 fi

export-image/00-allow-rerun/files/policy-rc.d

@@ -1,2 +0,0 @@
-#!/bin/sh
-exit 101

export-image/01-set-sources/00-patches/0-sources.diff

@@ -3,7 +3,7 @@ Index: export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
 --- export-jessie.orig/rootfs/etc/apt/sources.list.d/raspi.list
 +++ export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
 @@ -1,3 +1,3 @@
--deb http://archive.raspberrypi.org/debian/ jessie main ui staging
+-deb http://archive.raspberrypi.org/debian/ stretch main ui staging
-+deb http://archive.raspberrypi.org/debian/ jessie main ui
++deb http://archive.raspberrypi.org/debian/ stretch main ui
  # Uncomment line below then 'apt-get update' to enable 'apt-get source'
- #deb-src http://archive.raspberrypi.org/debian/ jessie main ui
+ #deb-src http://archive.raspberrypi.org/debian/ stretch main ui

export-image/02-network/01-run.sh

@@ -1,3 +1,3 @@
-#/bin/bash -e
+#!/bin/bash -e
 
-install -m 644 files/resolv.conf ${ROOTFS_DIR}/etc/
+install -m 644 files/resolv.conf "${ROOTFS_DIR}/etc/"

export-image/03-set-partuuid/00-run.sh

@@ -1,12 +1,13 @@
 #!/bin/bash -e
+
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
-IMGID="$(fdisk -l ${IMG_FILE} | sed -n 's/Disk identifier: 0x\([^ ]*\)/\1/p')"
+IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
 
 BOOT_PARTUUID="${IMGID}-01"
 ROOT_PARTUUID="${IMGID}-02"
 
-sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/etc/fstab
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
 
-sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" ${ROOTFS_DIR}/boot/cmdline.txt
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/cmdline.txt"

export-image/04-finalise/01-run.sh

@@ -1,64 +1,84 @@
 #!/bin/bash -e
 
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
+INFO_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.info"
 
 on_chroot << EOF
 /etc/init.d/fake-hwclock stop
 hardlink -t /usr/share/doc
 EOF
 
-if [ -d ${ROOTFS_DIR}/home/pi/.config ]; then
+if [ -d "${ROOTFS_DIR}/home/pi/.config" ]; then
-	chmod 700 ${ROOTFS_DIR}/home/pi/.config
+	chmod 700 "${ROOTFS_DIR}/home/pi/.config"
 fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-rm -f ${ROOTFS_DIR}/usr/sbin/policy-rc.d
+rm -f "${ROOTFS_DIR}/usr/bin/qemu-arm-static"
-rm -f ${ROOTFS_DIR}/usr/bin/qemu-arm-static
-if [ -e ${ROOTFS_DIR}/etc/ld.so.preload.disabled ]; then
-        mv ${ROOTFS_DIR}/etc/ld.so.preload.disabled ${ROOTFS_DIR}/etc/ld.so.preload
-fi
 
-rm -f ${ROOTFS_DIR}/etc/apt/sources.list~
+rm -f "${ROOTFS_DIR}/etc/apt/sources.list~"
-rm -f ${ROOTFS_DIR}/etc/apt/trusted.gpg~
+rm -f "${ROOTFS_DIR}/etc/apt/trusted.gpg~"
 
-rm -f ${ROOTFS_DIR}/etc/passwd-
+rm -f "${ROOTFS_DIR}/etc/passwd-"
-rm -f ${ROOTFS_DIR}/etc/group-
+rm -f "${ROOTFS_DIR}/etc/group-"
-rm -f ${ROOTFS_DIR}/etc/shadow-
+rm -f "${ROOTFS_DIR}/etc/shadow-"
-rm -f ${ROOTFS_DIR}/etc/gshadow-
+rm -f "${ROOTFS_DIR}/etc/gshadow-"
+rm -f "${ROOTFS_DIR}/etc/subuid-"
+rm -f "${ROOTFS_DIR}/etc/subgid-"
 
-rm -f ${ROOTFS_DIR}/var/cache/debconf/*-old
+rm -f "${ROOTFS_DIR}"/var/cache/debconf/*-old
-rm -f ${ROOTFS_DIR}/var/lib/dpkg/*-old
+rm -f "${ROOTFS_DIR}"/var/lib/dpkg/*-old
 
-rm -f ${ROOTFS_DIR}/usr/share/icons/*/icon-theme.cache
+rm -f "${ROOTFS_DIR}"/usr/share/icons/*/icon-theme.cache
 
-rm -f ${ROOTFS_DIR}/var/lib/dbus/machine-id
+rm -f "${ROOTFS_DIR}/var/lib/dbus/machine-id"
 
-true > ${ROOTFS_DIR}/etc/machine-id
+true > "${ROOTFS_DIR}/etc/machine-id"
 
-ln -nsf /proc/mounts ${ROOTFS_DIR}/etc/mtab
+ln -nsf /proc/mounts "${ROOTFS_DIR}/etc/mtab"
 
-for _FILE in $(find ${ROOTFS_DIR}/var/log/ -type f); do
+find "${ROOTFS_DIR}/var/log/" -type f -exec cp /dev/null {} \;
-	true > ${_FILE}
-done
 
 rm -f "${ROOTFS_DIR}/root/.vnc/private.key"
+rm -f "${ROOTFS_DIR}/etc/vnc/updateid"
 
-update_issue $(basename ${EXPORT_DIR})
+update_issue "$(basename "${EXPORT_DIR}")"
-install -m 644 ${ROOTFS_DIR}/etc/rpi-issue ${ROOTFS_DIR}/boot/issue.txt
+install -m 644 "${ROOTFS_DIR}/etc/rpi-issue" "${ROOTFS_DIR}/boot/issue.txt"
-install files/LICENSE.oracle ${ROOTFS_DIR}/boot/
+install files/LICENSE.oracle "${ROOTFS_DIR}/boot/"
 
-ROOT_DEV=$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')
 
-unmount ${ROOTFS_DIR}
+cp "$ROOTFS_DIR/etc/rpi-issue" "$INFO_FILE"
-zerofree -v ${ROOT_DEV}
 
-unmount_image ${IMG_FILE}
 
-mkdir -p ${DEPLOY_DIR}
+{
+	firmware=$(zgrep "firmware as of" \
+		"$ROOTFS_DIR/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" | \
+		head -n1 | sed  -n 's|.* \([^ ]*\)$|\1|p')
+	printf "\nFirmware: https://github.com/raspberrypi/firmware/tree/%s\n" "$firmware"
 
-rm -f ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip
+	kernel="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/git_hash")"
+	printf "Kernel: https://github.com/raspberrypi/linux/tree/%s\n" "$kernel"
 
-echo zip ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip ${IMG_FILE}
+	uname="$(curl -s -L "https://github.com/raspberrypi/firmware/raw/$firmware/extra/uname_string7")"
-pushd ${STAGE_WORK_DIR} > /dev/null
+
-zip ${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip $(basename ${IMG_FILE})
+	printf "Uname string: %s\n" "$uname"
+	printf "\nPackages:\n"
+	dpkg -l --root "$ROOTFS_DIR"
+} >> "$INFO_FILE"
+
+ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
+
+unmount "${ROOTFS_DIR}"
+zerofree -v "${ROOT_DEV}"
+
+unmount_image "${IMG_FILE}"
+
+mkdir -p "${DEPLOY_DIR}"
+
+rm -f "${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip"
+
+pushd "${STAGE_WORK_DIR}" > /dev/null
+zip "${DEPLOY_DIR}/image_${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.zip" \
+	"$(basename "${IMG_FILE}")"
 popd > /dev/null
+
+cp "$INFO_FILE" "$DEPLOY_DIR"

export-image/prerun.sh

@@ -1,20 +1,23 @@
 #!/bin/bash -e
+
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-rm -f ${IMG_FILE}
+rm -f "${IMG_FILE}"
 
-rm -rf ${ROOTFS_DIR}
+rm -rf "${ROOTFS_DIR}"
-mkdir -p ${ROOTFS_DIR}
+mkdir -p "${ROOTFS_DIR}"
 
-BOOT_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR}/boot --block-size=1 | cut -f 1)
+BOOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}/boot" --block-size=1 | cut -f 1)
-TOTAL_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR} --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
+TOTAL_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
 
-IMG_SIZE=$((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024)))
+ROUND_SIZE="$((4 * 1024 * 1024))"
+ROUNDED_ROOT_SECTOR=$(((2 * BOOT_SIZE + ROUND_SIZE) / ROUND_SIZE * ROUND_SIZE / 512 + 8192))
+IMG_SIZE=$(((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024) + ROUND_SIZE - 1) / ROUND_SIZE * ROUND_SIZE))
 
-truncate -s ${IMG_SIZE} ${IMG_FILE}
+truncate -s "${IMG_SIZE}" "${IMG_FILE}"
-fdisk -H 255 -S 63 ${IMG_FILE} <<EOF
+fdisk -H 255 -S 63 "${IMG_FILE}" <<EOF
 o
 n
 
@@ -27,14 +30,14 @@ c
 n
 
 
-8192
+${ROUNDED_ROOT_SECTOR}
 
 
 p
 w
 EOF
 
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
+PARTED_OUT=$(parted -s "${IMG_FILE}" unit b print)
 BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
 | cut -d" " -f 2 | tr -d B)
 BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
@@ -45,16 +48,22 @@ ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 | cut -d" " -f 4 | tr -d B)
 
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
-mkdosfs -n boot -F 32 -v $BOOT_DEV > /dev/null
+ROOT_FEATURES="^huge_file"
-mkfs.ext4 -O ^metadata_csum,^huge_file $ROOT_DEV > /dev/null
+for FEATURE in metadata_csum 64bit; do
+	if grep -q "$FEATURE" /etc/mke2fs.conf; then
+	    ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
+	fi
+done
+mkdosfs -n boot -F 32 -v "$BOOT_DEV" > /dev/null
+mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
 
-mount -v $ROOT_DEV ${ROOTFS_DIR} -t ext4
+mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
-mkdir -p ${ROOTFS_DIR}/boot
+mkdir -p "${ROOTFS_DIR}/boot"
-mount -v $BOOT_DEV ${ROOTFS_DIR}/boot -t vfat
+mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot" -t vfat
 
-rsync -aHAXx --exclude var/cache/apt/archives ${EXPORT_ROOTFS_DIR}/ ${ROOTFS_DIR}/
+rsync -aHAXx --exclude var/cache/apt/archives "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"

export-noobs/00-release/00-run.sh

@@ -2,35 +2,35 @@
 
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
 
-install -v -m 744	files/partition_setup.sh	${NOOBS_DIR}/
+install -v -m 744	files/partition_setup.sh	"${NOOBS_DIR}/"
-install -v		files/partitions.json		${NOOBS_DIR}/
+install -v		files/partitions.json		"${NOOBS_DIR}/"
-install -v		files/os.json			${NOOBS_DIR}/
+install -v		files/os.json			"${NOOBS_DIR}/"
-install -v		files/OS.png			${NOOBS_DIR}/
+install -v		files/OS.png			"${NOOBS_DIR}/"
-install -v		files/release_notes.txt		${NOOBS_DIR}/
+install -v		files/release_notes.txt		"${NOOBS_DIR}/"
 
-tar -v -c -C		files/marketing			-f ${NOOBS_DIR}/marketing.tar .
+tar -v -c -C		files/marketing			-f "${NOOBS_DIR}/marketing.tar" .
 
-BOOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/boot.tar.xz  | grep totals | cut -f 5)
+BOOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/boot.tar.xz"  | grep totals | cut -f 5)"
-ROOT_SIZE=$(xz --robot -l ${NOOBS_DIR}/root.tar.xz  | grep totals | cut -f 5)
+ROOT_SIZE="$(xz --robot -l "${NOOBS_DIR}/root.tar.xz"  | grep totals | cut -f 5)"
 
-BOOT_SIZE=$(expr ${BOOT_SIZE} / 1000000 \+ 1)
+BOOT_SIZE="$(( BOOT_SIZE / 1024 / 1024 + 1))"
-ROOT_SIZE=$(expr ${ROOT_SIZE} / 1000000 \+ 1)
+ROOT_SIZE="$(( ROOT_SIZE / 1024 / 1024 + 1))"
 
-BOOT_NOM=$(expr ${BOOT_SIZE} \* 3)
+BOOT_NOM="$(( BOOT_SIZE * 3 ))"
-ROOT_NOM=$(expr ${ROOT_SIZE} \+ 400)
+ROOT_NOM="$(( ROOT_SIZE + 400 ))"
 
-mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/$(echo ${NOOBS_NAME} | sed 's/ /_/g').png"
+mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/${NOOBS_NAME// /_}.png"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_SIZE|${BOOT_SIZE}|"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_SIZE|${ROOT_SIZE}|"
 
-sed ${NOOBS_DIR}/partitions.json -i -e "s|BOOT_NOM|${BOOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|BOOT_NOM|${BOOT_NOM}|"
-sed ${NOOBS_DIR}/partitions.json -i -e "s|ROOT_NOM|${ROOT_NOM}|"
+sed "${NOOBS_DIR}/partitions.json" -i -e "s|ROOT_NOM|${ROOT_NOM}|"
 
-sed ${NOOBS_DIR}/os.json -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|UNRELEASED|${IMG_DATE}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
-sed ${NOOBS_DIR}/os.json -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
+sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
 
-sed ${NOOBS_DIR}/release_notes.txt -i -e "s|UNRELEASED|${IMG_DATE}|"
+sed "${NOOBS_DIR}/release_notes.txt" -i -e "s|UNRELEASED|${IMG_DATE}|"
 
-cp -a ${NOOBS_DIR} ${DEPLOY_DIR}/
+cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"

export-noobs/00-release/files/os.json

@@ -1,7 +1,7 @@
 {
     "description": "NOOBS_DESCRIPTION",
     "feature_level": 35120124,
-    "kernel": "4.9",
+    "kernel": "4.14",
     "name": "NOOBS_NAME",
     "password": "raspberry",
     "release_date": "UNRELEASED",
@@ -10,9 +10,10 @@
         "Pi Model",
         "Pi 2",
         "Pi Zero",
-        "Pi 3"
+        "Pi 3",
+        "Pi Compute Module 3"
     ],
     "url": "http://www.raspbian.org/",
     "username": "pi",
-    "version": "jessie"
+    "version": "stretch"
 }

export-noobs/00-release/files/partition_setup.sh

@@ -29,7 +29,7 @@ if [ -f /settings/wpa_supplicant.conf ]; then
 fi
 
 if ! grep -q resize /proc/cmdline; then
-  sed -i 's| quiet init=/usr/lib/raspi-config/init_resize.sh||' /tmp/1/cmdline.txt
+  sed -i 's| init=/usr/lib/raspi-config/init_resize.sh||;s| quiet||2g' /tmp/1/cmdline.txt
 fi
 
 umount /tmp/1

export-noobs/00-release/files/release_notes.txt

@@ -1,4 +1,149 @@
 UNRELEASED:
+  *
+2018-11-13:
+  * Two versions of image created - "base" image has no optional software packages included; "full" image has all optional packages
+    - Removed from "base" image - LibreOffice, Thonny, Scratch, Scratch 2, Sonic Pi, Minecraft, Python Games, SmartSim, SenseHAT Emulator
+    - Added to "full" image - Mathematica, BlueJ, Greenfoot, Node-RED, Claws Mail, VNC Viewer
+  * Python Games and SmartSim added to Recommended Software
+  * VLC media player with VideoCore hardware acceleration included in image
+  * Version 3.0.5 of Thonny included
+  * Modifications to LXDE components to enable local configuration to override global configuration correctly
+  * Modifications to Appearance Settings to support above configuration changes
+  * Modifications to various initial config defaults and relevant package to support above configuration changes
+  * Selecting default option in Appearance Settings now deletes relevant local configuration files
+  * PiX theme modified so that all changes made in Appearance Settings are in override files rather than in theme files
+  * Design of scrollbar buttons changed
+  * Image Viewer moved into Graphics category on main menu
+  * Recommended Software now installs LibreOffice language support files if needed, and suggests reboot if needed
+  * Latest version of Pepper Flash plugin included
+  * Chromium h264ify plugin permissions set correctly by default
+  * Corrections to various MIME types so that files open in sensible default applications
+  * Set default timezone to 'Europe/London'
+  * Linux kernel 4.14.79
+  * Raspberry Pi firmware 12e0bf86e08d6067372bc0a45d7e8a10d3113210
+2018-10-09:
+  * Raspberry Pi 3A+ support
+  * In startup wizard, assign keyboard to country as per Debian installer recommendations
+  * In startup wizard, add option to use US keyboard in preference to country-specific option
+  * In startup wizard, show IP address on first page
+  * In startup wizard, check for existing wifi network connection and show it if there is one
+  * In startup wizard, install language support packages for LibreOffice and other applications
+  * In startup wizard, improve operation with keyboard only and no mouse
+  * Password change in Raspberry Pi Configuration and startup wizard now works properly if passwords contain shell characters
+  * Battery indicator plugin modified to cope with Pi-top hardware monitor crashing
+  * Networking plugin hides wifi password characters by default
+  * In Scratch 2 GPIO plugin, set pin from dropdown list rather than free text
+  * In Scratch 2 SenseHAT plugin, swap x and y axis values for LED array
+  * Include latest Adobe Flash player (31.0.0.108)
+  * Include latest RealVNC Server (6.3.1)
+  * Include libav-tools
+  * Include ssh-import-id
+  * Removed Mathematica
+  * Merge in latest third-party code for Bluetooth ALSA interface
+  * Add ability to prevent software update changing configuration files, by creating ~/.config/.lock file
+  * Various other small bug fixes, tweaks and changes to text
+  * Make dhcpcd work with 3G devices
+  * Add hw acceleration to ffmpeg
+  * Improved WiFi-BT coexistence parameters
+  * Run fake-hwclock before systemd-fsck-root
+  * Raspberry Pi PoE HAT support
+  * Linux kernel 4.14.71
+  * Raspberry Pi firmware 5b49caa17e91d0e64024380119ad739bb201c674
+2018-06-27:
+  * New first-boot configuration wizard added
+  * Recommended Software installer added
+  * Bluej, Greenfoot, NodeRED, Claws Mail, VNC Viewer removed from image - can now be installed from Recommended Applications
+  * Qpdfview PDF viewer installed instead of Xpdf
+  * Version 65.0 of Chromium browser included, with latest Flash player
+  * Volume up / down keys now change by 5% increments and affect currently-selected output device rather than internal device only
+  * Network plugin now remembers previously-entered WiFi network passwords when prompting for reconnection
+  * Serial port and serial console can now be switched separately in Raspberry Pi Configuration
+  * Lxkeymap keyboard language setting application removed - replaced with dialog within Raspberry Pi Configuration
+  * Wifi country and keyboard language setting dialogs in Raspberry Pi Configuration now callable from other applications
+  * New version of Piboto font included to render with correct weight under some rogue applications
+  * Reconnection to Bluetooth audio devices on reboot improved
+  * Disable click-to-rename behaviour in file manager if single-click selection enabled
+  * Appearance Settings dialog makes config changes to some Qt files to match selected theme
+  * MIME file type associations improved
+  * Multiple desktop management options removed from mouse middle-click menu
+  * Menu shortcuts to Raspberry Pi website amended
+  * Python 2 IDLE menu link removed
+  * Sample Magpi PDF installed in /home/pi/MagPi
+  * Various minor tweaks, bug fixes and appearance changes
+  * Bluetooth updates
+    - Firmware with Bluetooth 4.2 features
+    - SCO profile suppot added via bthelper.service
+  * Linux kernel 4.14.50+
+  * Raspberry Pi firmware 748fb17992426bb29d99224b93cb962fefbdc833
+2018-04-18:
+  * Fixed race between wifi-country.service and raspberrypi-net-mods.service
+  * Linux kernel 4.14.34+
+  * Raspberry Pi firmware 5db8e4e1c63178e200d6fbea23ed4a9bf4656658
+2018-03-13:
+  * Raspberry Pi 3 B+ support
+  * WiFi is disabled until wireless regulatory domain is set (Pi 3 B+ only)
+    - The domain can be done through 'Raspberry Pi Configuration' (rc_gui),
+      'raspi-config' or by setting 'country=' to an appropriate ISO 3166
+      alpha2 country code in /etc/wpa_supplicant/wpa_supplicant.conf.
+  * Default wireless regulatory domain is now unset
+  * Added support to desktop for different screen sizes and resolutions,
+    including multiple preset options in Appearance Settings and pixel doubling
+    option in Raspberry Pi Configuration
+  * Version 2.1.16 of Thonny included
+  * Version 29.0.0.113 of Adobe PepperFlash player included
+  * Version 1.2.post1 of Pygame Zero included
+  * Bluetooth plugin now supports connection to Bluetooth LE HID devices
+  * Network plugin now indicates 5G-compatible APs
+  * Latest changes to Bluez ALSA service merged
+    - service now started on CLI boot as well as GUI boot
+  * Latest changes to dhcpcd networking plugin merged
+  * Improved support for running on pi-top devices
+  * Small design changes to PiX theme and icons
+  * Bug fix - hide spurious window resize handles
+  * Bug fix - Scratch 2 remote GPIO state block now works correctly
+  * Updated WiFi Firmware
+    - brcmfmac43455-sdio 7.45.154
+    - brcmfmac43430-sdio 7.45.98.38
+  * New packages:
+    - policykit-1
+    - obconf
+    - python-buttonshim python3-buttonshim
+    - python-unicornhathd  python3-unicornhathd
+    - python-pantilthat python3-pantilthat
+  * Linux kernel 4.9.80+
+  * Raspberry Pi firmware 3347884c7df574bbabeff6dca63caf686e629699
+2017-11-29:
+  * Added battery monitor plugin for taskbar - works on x86 images or first-generation Pi-Top
+  * Added cutdown mode to PCManFM file manager to reduce complexity
+  * Added ability to rename files in PCManFM by clicking name when selected
+  * Bug fix in Bluetooth ALSA module to reduce truncation of audio at end of playback
+  * Various small tweaks, bug fixes and theme modifications
+  * New kernel and firmware
+2017-09-07:
+  * Disable predictable network interface names for Ethernet devices
+  * Bug fix for keyboard settings dialog in Raspberry Pi Configuration
+  * Bug fix for crash on some videos and animations in Chromium
+  * Bug fix for taskbar crash when running RealVNC server
+  * Bug fix for reloading projects with extensions in Scratch 2
+  * Bug fix for MAC address problem in Bluetooth
+  * Simple mode and new icons in Thonny
+  * New Japanese translations in Raspberry Pi Configuration
+  * Install fonts-droid-fallback for international fonts
+2017-08-16:
+  * Based on Raspbian Stretch (Debian version 9)
+  * Version 60 of Chromium browser included
+  * Version 3.0.1 of Sonic Pi included
+  * Version 6.1.1 of RealVNC included
+  * Version 0.17.4 of NodeRED included
+  * Bluetooth audio routed via ALSA rather than Pulseaudio
+  * SenseHAT extension added to Scratch 2
+  * Various desktop applications modified to prompt for sudo password if needed
+  * lxinput control options for mouse speed simplified
+  * lxpanel plugins moved into separate packages
+  * Wireless firmware for Pi 3 and Pi 0W modified to address Broadpwn exploit
+  * Latest kernel and firmware
+  * Various small tweaks, bug fixes and theme modifications
+2017-07-05:
   * New kernel and firmware
   * Filesystem created without the metadata_csum feature
 2017-06-21:

export-noobs/prerun.sh

@@ -2,14 +2,14 @@
 
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
-unmount_image ${IMG_FILE}
+unmount_image "${IMG_FILE}"
 
-mkdir -p ${STAGE_WORK_DIR}
+mkdir -p "${STAGE_WORK_DIR}"
-cp ${WORK_DIR}/export-image/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img ${STAGE_WORK_DIR}/
+cp "${WORK_DIR}/export-image/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img" "${STAGE_WORK_DIR}/"
 
-rm -rf ${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}
+rm -rf "${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
 
-PARTED_OUT=$(parted -s ${IMG_FILE} unit b print)
+PARTED_OUT=$(parted -s "${IMG_FILE}" unit b print)
 BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
 | cut -d" " -f 2 | tr -d B)
 BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 1'| xargs echo -n \
@@ -20,19 +20,21 @@ ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^ 2'| xargs echo -n \
 | cut -d" " -f 4 | tr -d B)
 
-BOOT_DEV=$(losetup --show -f -o ${BOOT_OFFSET} --sizelimit ${BOOT_LENGTH} ${IMG_FILE})
+BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}")
-ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_FILE})
+ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}")
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
-mkdir -p ${STAGE_WORK_DIR}/rootfs
+mkdir -p "${STAGE_WORK_DIR}/rootfs"
-mkdir -p ${NOOBS_DIR}
+mkdir -p "${NOOBS_DIR}"
 
-mount $ROOT_DEV ${STAGE_WORK_DIR}/rootfs
+mount "$ROOT_DEV" "${STAGE_WORK_DIR}/rootfs"
-mount $BOOT_DEV ${STAGE_WORK_DIR}/rootfs/boot
+mount "$BOOT_DEV" "${STAGE_WORK_DIR}/rootfs/boot"
 
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs/boot -cpf ${NOOBS_DIR}/boot.tar.xz .
+ln -sv "/lib/systemd/system/apply_noobs_os_config.service" "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
-umount ${STAGE_WORK_DIR}/rootfs/boot
-bsdtar --format gnutar --use-compress-program pxz -C ${STAGE_WORK_DIR}/rootfs --one-file-system -cpf ${NOOBS_DIR}/root.tar.xz .
 
-unmount_image ${IMG_FILE}
+bsdtar --numeric-owner --format gnutar --use-compress-program pxz -C "${STAGE_WORK_DIR}/rootfs/boot" -cpf "${NOOBS_DIR}/boot.tar.xz" .
+umount "${STAGE_WORK_DIR}/rootfs/boot"
+bsdtar --numeric-owner --format gnutar --use-compress-program pxz -C "${STAGE_WORK_DIR}/rootfs" --one-file-system -cpf "${NOOBS_DIR}/root.tar.xz" .
+
+unmount_image "${IMG_FILE}"

scripts/common

@@ -1,5 +1,5 @@
 log (){
-	date +"[%T] $@" | tee -a "${LOG_FILE}"
+	date +"[%T] $*" | tee -a "${LOG_FILE}"
 }
 export -f log
 
@@ -15,10 +15,13 @@ bootstrap(){
 		local BOOTSTRAP_CMD=debootstrap
 	fi
 
-	capsh --drop=cap_setfcap -- -c "${BOOTSTRAP_CMD} --components=main,contrib,non-free \
+	capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" --components=main,contrib,non-free \
 		--arch armhf \
 		--keyring "${STAGE_DIR}/files/raspberrypi.gpg" \
-		$1 $2 $3" || rmdir "$2/debootstrap"
+		"$1" "$2" "$3" || true
+	if [ -d "$2/debootstrap" ]; then
+		rmdir "$2/debootstrap"
+	fi
 }
 export -f bootstrap
 
@@ -85,7 +88,7 @@ on_chroot() {
 		mount --bind /sys "${ROOTFS_DIR}/sys"
 	fi
 
-	capsh --drop=cap_setfcap "--chroot=${ROOTFS_DIR}/" -- "$@"
+	capsh --drop=cap_setfcap "--chroot=${ROOTFS_DIR}/" -- -e "$@"
 }
 export -f on_chroot
 

scripts/dependencies_check

@@ -1,5 +1,5 @@
 # dependencies_check
-# $@	Dependnecy files to check
+# $@	Dependency files to check
 #
 # Each dependency is in the form of a tool to test for, optionally followed by
 # a : and the name of a package if the package on a Debian-ish system is not
@@ -10,18 +10,18 @@ dependencies_check()
 
 	for depfile in "$@"; do
 		if [[ -e "$depfile" ]]; then
-			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${BASE_DIR}/depends)"
+			deps="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < "${BASE_DIR}/depends")"
 
 		fi
 		for dep in $deps; do
-			if ! hash ${dep%:*} 2>/dev/null; then
+			if ! hash "${dep%:*}" 2>/dev/null; then
 				missing="${missing:+$missing }${dep#*:}"
 			fi
 		done
 	done
 
 	if [[ "$missing" ]]; then
-		echo "Reqired dependencies not installed"
+		echo "Required dependencies not installed"
 		echo
 		echo "This can be resolved on Debian/Raspbian systems by installing:"
 		echo "$missing"

stage0/00-configure-apt/00-run.sh

@@ -1,13 +1,13 @@
 #!/bin/bash -e
 
-install -m 644 files/sources.list ${ROOTFS_DIR}/etc/apt/
+install -m 644 files/sources.list "${ROOTFS_DIR}/etc/apt/"
-install -m 644 files/raspi.list ${ROOTFS_DIR}/etc/apt/sources.list.d/
+install -m 644 files/raspi.list "${ROOTFS_DIR}/etc/apt/sources.list.d/"
 
 if [ -n "$APT_PROXY" ]; then
-	install -m 644 files/51cache ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	install -m 644 files/51cache "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
-	sed ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache -i -e "s|APT_PROXY|${APT_PROXY}|"
+	sed "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache" -i -e "s|APT_PROXY|${APT_PROXY}|"
 else
-	rm -f ${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache
+	rm -f "${ROOTFS_DIR}/etc/apt/apt.conf.d/51cache"
 fi
 
 on_chroot apt-key add - < files/raspberrypi.gpg.key

stage0/00-configure-apt/files/raspi.list

@@ -1,3 +1,3 @@
-deb http://archive.raspberrypi.org/debian/ jessie main ui staging
+deb http://archive.raspberrypi.org/debian/ stretch main ui staging
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspberrypi.org/debian/ jessie main ui
+#deb-src http://archive.raspberrypi.org/debian/ stretch main ui

stage0/00-configure-apt/files/sources.list

@@ -1,3 +1,3 @@
-deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi
+deb http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi
+#deb-src http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi

stage0/02-firmware/01-packages

@@ -1 +1,2 @@
 raspberrypi-bootloader
+raspberrypi-kernel

stage0/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
-	bootstrap jessie ${ROOTFS_DIR} http://mirrordirector.raspbian.org/raspbian/
+if [ ! -d "${ROOTFS_DIR}" ]; then
+	bootstrap stretch "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
 fi

stage1/00-boot-files/00-run.sh

@@ -1,4 +1,4 @@
 #!/bin/bash -e
 
-install -m 644 files/cmdline.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/cmdline.txt "${ROOTFS_DIR}/boot/"
-install -m 644 files/config.txt ${ROOTFS_DIR}/boot/
+install -m 644 files/config.txt "${ROOTFS_DIR}/boot/"

stage1/01-sys-tweaks/00-run.sh

@@ -1,9 +1,8 @@
 #!/bin/bash -e
 
-install -d ${ROOTFS_DIR}/etc/systemd/system/[email protected]
+install -d "${ROOTFS_DIR}/etc/systemd/system/[email protected]"
-install -m 644 files/noclear.conf ${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf
+install -m 644 files/noclear.conf "${ROOTFS_DIR}/etc/systemd/system/[email protected]/noclear.conf"
-install -m 744 files/policy-rc.d ${ROOTFS_DIR}/usr/sbin/policy-rc.d #TODO: Necessary in systemd?
+install -v -m 644 files/fstab "${ROOTFS_DIR}/etc/fstab"
-install -v -m 644 files/fstab ${ROOTFS_DIR}/etc/fstab
 
 on_chroot << EOF
 if ! id -u pi >/dev/null 2>&1; then

stage1/01-sys-tweaks/files/policy-rc.d

@@ -1,2 +0,0 @@
-#!/bin/sh
-exit 101

stage1/02-net-tweaks/00-packages

@@ -0,0 +1 @@
+netbase

stage1/02-net-tweaks/00-patches/02-persistant-net.diff

@@ -1,13 +0,0 @@
-Index: jessie-stage1/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-===================================================================
---- jessie-stage1.orig/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-+++ jessie-stage1/rootfs/lib/udev/rules.d/75-persistent-net-generator.rules
-@@ -22,7 +22,7 @@ IMPORT{cmdline}="net.ifnames"
- ENV{net.ifnames}=="1",			GOTO="persistent_net_generator_end"
- 
- # device name whitelist
--KERNEL!="eth*|ath*|wlan*[0-9]|msh*|ra*|sta*|ctc*|lcs*|hsi*", \
-+KERNEL!="ath*|msh*|ra*|sta*|ctc*|lcs*|hsi*", \
- 					GOTO="persistent_net_generator_end"
- 
- # ignore Xen virtual interfaces

stage1/02-net-tweaks/00-patches/series

@@ -1,2 +1 @@
 01-hosts.diff
-02-persistant-net.diff

stage1/02-net-tweaks/00-run.sh

@@ -1,9 +1,6 @@
 #!/bin/bash -e
 
-install -m 644 files/ipv6.conf ${ROOTFS_DIR}/etc/modprobe.d/ipv6.conf
+install -m 644 files/ipv6.conf "${ROOTFS_DIR}/etc/modprobe.d/ipv6.conf"
-install -m 644 files/interfaces ${ROOTFS_DIR}/etc/network/interfaces
+install -m 644 files/hostname "${ROOTFS_DIR}/etc/hostname"
-install -m 644 files/hostname ${ROOTFS_DIR}/etc/hostname
 
-on_chroot << EOF
+ln -sf /dev/null "${ROOTFS_DIR}/etc/systemd/network/99-default.link"
-dpkg-divert --add --local /lib/udev/rules.d/75-persistent-net-generator.rules
-EOF

stage1/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage2/00-copies-and-fills/00-run.sh

@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-touch ${ROOTFS_DIR}/spindle_install

stage2/00-copies-and-fills/02-run.sh

@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-rm -f ${ROOTFS_DIR}/spindle_install

stage2/01-sys-tweaks/00-debconf

@@ -7,12 +7,6 @@ console-setup   console-setup/codeset47 select  Guess optimal character set
 # Font for the console:
 # Choices: Fixed, Goha, GohaClassic, Terminus, TerminusBold, TerminusBoldVGA, VGA, Do not change the boot/kernel font, Let the system select a suitable font
 console-setup   console-setup/fontface47        select  Do not change the boot/kernel font
-# Geographic area:
-# Choices: Africa, America, Antarctica, Australia, Arctic Ocean, Asia, Atlantic Ocean, Europe, Indian Ocean, Pacific Ocean, System V timezones, US, None of the above
-tzdata	tzdata/Areas	select	Etc
-# Time zone:
-# Choices: GMT, GMT+0, GMT+1, GMT+10, GMT+11, GMT+12, GMT+2, GMT+3, GMT+4, GMT+5, GMT+6, GMT+7, GMT+8, GMT+9, GMT-0, GMT-1, GMT-10, GMT-11, GMT-12, GMT-13, GMT-14, GMT-2, GMT-3, GMT-4, GMT-5, GMT-6, GMT-7, GMT-8, GMT-9, GMT0, Greenwich, UCT, UTC, Universal, Zulu
-tzdata	tzdata/Zones/Etc	select	UTC
 # Key to function as AltGr:
 # Choices: The default for the keyboard layout, No AltGr key, Right Alt (AltGr), Right Control, Right Logo key, Menu key, Left Alt, Left Logo key, Keypad Enter key, Both Logo keys, Both Alt keys
 keyboard-configuration	keyboard-configuration/altgr	select	The default for the keyboard layout

stage2/01-sys-tweaks/00-packages

@@ -1,4 +1,4 @@
-ssh less fbset sudo psmisc strace module-init-tools ed ncdu crda
+ssh less fbset sudo psmisc strace ed ncdu crda
 console-setup keyboard-configuration debconf-utils parted unzip
 build-essential manpages-dev python bash-completion gdb pkg-config
 python-rpi.gpio v4l-utils
@@ -6,7 +6,7 @@ avahi-daemon
 lua5.1
 luajit
 hardlink ca-certificates curl
-fake-hwclock ntp nfs-common usbutils
+fake-hwclock nfs-common usbutils
 libraspberrypi-dev libraspberrypi-doc libfreetype6-dev
 dosfstools
 dphys-swapfile
@@ -17,3 +17,9 @@ usb-modeswitch
 apt-transport-https
 libpam-chksshpwd
 rpi-update
+libmtp-runtime
+rsync
+htop
+man-db
+policykit-1
+ssh-import-id

stage2/01-sys-tweaks/00-patches/06-rc_local.diff

@@ -1,15 +0,0 @@
-Index: jessie-stage2/rootfs/etc/rc.local
-===================================================================
---- jessie-stage2.orig/rootfs/etc/rc.local
-+++ jessie-stage2/rootfs/etc/rc.local
-@@ -11,4 +11,10 @@
- #
- # By default this script does nothing.
- 
-+# Print the IP address
-+_IP=$(hostname -I) || true
-+if [ "$_IP" ]; then
-+  printf "My IP address is %s\n" "$_IP"
-+fi
-+
- exit 0

stage2/01-sys-tweaks/00-patches/series

@@ -2,5 +2,4 @@
 02-swap.diff
 04-inputrc.diff
 05-path.diff
-06-rc_local.diff
 07-resize-init.diff

stage2/01-sys-tweaks/01-run.sh

@@ -1,15 +1,15 @@
 #!/bin/bash -e
 
-install -m 644 files/regenerate_ssh_host_keys.service	${ROOTFS_DIR}/lib/systemd/system/
+install -m 755 files/resize2fs_once	"${ROOTFS_DIR}/etc/init.d/"
-install -m 755 files/apply_noobs_os_config		${ROOTFS_DIR}/etc/init.d/
-install -m 755 files/resize2fs_once			${ROOTFS_DIR}/etc/init.d/
 
-install -d						${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d
+install -d				"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d"
-install -m 644 files/ttyoutput.conf			${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/
+install -m 644 files/ttyoutput.conf	"${ROOTFS_DIR}/etc/systemd/system/rc-local.service.d/"
 
-install -m 644 files/50raspi				${ROOTFS_DIR}/etc/apt/apt.conf.d/
+install -m 644 files/50raspi		"${ROOTFS_DIR}/etc/apt/apt.conf.d/"
 
-install -m 644 files/console-setup   			${ROOTFS_DIR}/etc/default/
+install -m 644 files/console-setup   	"${ROOTFS_DIR}/etc/default/"
+
+install -m 755 files/rc.local		"${ROOTFS_DIR}/etc/"
 
 on_chroot << EOF
 systemctl disable hwclock.sh
@@ -17,13 +17,24 @@ systemctl disable nfs-common
 systemctl disable rpcbind
 systemctl disable ssh
 systemctl enable regenerate_ssh_host_keys
-systemctl enable apply_noobs_os_config
+EOF
+
+if [ "${USE_QEMU}" = "1" ]; then
+	echo "enter QEMU mode"
+	install -m 644 files/90-qemu.rules "${ROOTFS_DIR}/etc/udev/rules.d/"
+	on_chroot << EOF
+systemctl disable resize2fs_once
+EOF
+	echo "leaving QEMU mode"
+else
+	on_chroot << EOF
 systemctl enable resize2fs_once
 EOF
+fi
 
 on_chroot << \EOF
 for GRP in input spi i2c gpio; do
-	groupadd -f -r $GRP
+	groupadd -f -r "$GRP"
 done
 for GRP in adm dialout cdrom audio users sudo video games plugdev input gpio spi i2c netdev; do
   adduser pi $GRP
@@ -38,4 +49,4 @@ on_chroot << EOF
 usermod --pass='*' root
 EOF
 
-rm -f ${ROOTFS_DIR}/etc/ssh/ssh_host_*_key*
+rm -f "${ROOTFS_DIR}/etc/ssh/"ssh_host_*_key*

stage2/01-sys-tweaks/files/90-qemu.rules

@@ -0,0 +1,3 @@
+KERNEL=="sda", SYMLINK+="mmcblk0"
+KERNEL=="sda?", SYMLINK+="mmcblk0p%n"
+KERNEL=="sda2", SYMLINK+="root"

stage2/01-sys-tweaks/files/apply_noobs_os_config

@@ -1,31 +0,0 @@
-#!/bin/sh
-### BEGIN INIT INFO
-# Provides:          apply_noobs_os_config
-# Required-Start:
-# Required-Stop:
-# Default-Start: 2
-# Default-Stop:
-# Short-Description: Apply config from /boot/os_config.json
-# Description:
-### END INIT INFO
-
-. /lib/lsb/init-functions
-
-set -e
-
-case "$1" in
-  start)
-    log_daemon_msg "Applying config from /boot/os_config.json (if it exists)"
-    if raspi-config --apply-os-config; then
-      update-rc.d apply_noobs_os_config remove
-      rm /etc/init.d/apply_noobs_os_config
-      log_end_msg 0
-    else
-      log_end_msg 1
-    fi
-    ;;
-  *)
-    echo "Usage: $0 start" >&2
-    exit 3
-    ;;
-esac

stage2/01-sys-tweaks/files/rc.local

@@ -0,0 +1,20 @@
+#!/bin/sh -e
+#
+# rc.local
+#
+# This script is executed at the end of each multiuser runlevel.
+# Make sure that the script will "exit 0" on success or any other
+# value on error.
+#
+# In order to enable or disable this script just change the execution
+# bits.
+#
+# By default this script does nothing.
+
+# Print the IP address
+_IP=$(hostname -I) || true
+if [ "$_IP" ]; then
+  printf "My IP address is %s\n" "$_IP"
+fi
+
+exit 0

stage2/01-sys-tweaks/files/regenerate_ssh_host_keys.service

@@ -1,11 +0,0 @@
-[Unit]
-Description=Regenerate SSH host keys
-
-[Service]
-Type=oneshot
-ExecStartPre=/bin/sh -c "if [ -e /dev/hwrng ]; then dd if=/dev/hwrng of=/dev/urandom count=1 bs=4096; fi"
-ExecStart=/usr/bin/ssh-keygen -A
-ExecStartPost=/bin/rm /lib/systemd/system/regenerate_ssh_host_keys.service ; /usr/sbin/update-rc.d regenerate_ssh_host_keys remove
-
-[Install]
-WantedBy=multi-user.target

stage2/02-net-tweaks/00-packages

@@ -1,3 +1,4 @@
-wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-ralink firmware-realtek
+wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek
 raspberrypi-net-mods
 dhcpcd5
+net-tools

stage2/02-net-tweaks/01-run.sh

@@ -1,8 +1,8 @@
 #!/bin/bash -e
 
-install -v -d						${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d
+install -v -d					"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d"
-install -v -m 644 files/wait.conf			${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/
+install -v -m 644 files/wait.conf		"${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/"
 
-install -v -d                                           ${ROOTFS_DIR}/etc/wpa_supplicant
+install -v -d					"${ROOTFS_DIR}/etc/wpa_supplicant"
-install -v -m 600 files/wpa_supplicant.conf             ${ROOTFS_DIR}/etc/wpa_supplicant/
+install -v -m 600 files/wpa_supplicant.conf	"${ROOTFS_DIR}/etc/wpa_supplicant/"
 

stage2/02-net-tweaks/files/wait.conf

@@ -1,3 +1,3 @@
 [Service]
 ExecStart=
-ExecStart=/sbin/dhcpcd -q -w
+ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w

stage2/02-net-tweaks/files/wpa_supplicant.conf

@@ -1,3 +1,2 @@
-country=GB
 ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
 update_config=1

stage2/03-set-timezone/02-run.sh

@@ -0,0 +1,8 @@
+#!/bin/bash -e
+
+echo "Europe/London" > "${ROOTFS_DIR}/etc/timezone"
+rm "${ROOTFS_DIR}/etc/localtime"
+
+on_chroot << EOF
+dpkg-reconfigure -f noninteractive tzdata
+EOF

stage2/EXPORT_IMAGE

@@ -1 +1,4 @@
 IMG_SUFFIX="-lite"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage2/EXPORT_NOOBS

@@ -1,2 +1,2 @@
 NOOBS_NAME="Raspbian Lite"
-NOOBS_DESCRIPTION="A port of Debian jessie for the Raspberry Pi (minimal version)"
+NOOBS_DESCRIPTION="A port of Debian with no desktop environment"

stage2/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage3/00-install-packages/00-packages

@@ -1,5 +1,5 @@
 gstreamer1.0-x gstreamer1.0-omx gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-alsa gstreamer1.0-libav
-xpdf gtk2-engines alsa-utils
+qpdfview gtk2-engines alsa-utils
 desktop-base
 git
 omxplayer
@@ -8,3 +8,7 @@ policykit-1
 gvfs
 rfkill
 chromium-browser rpi-chromium-mods
+gldriver-test
+fonts-droid-fallback
+fonts-liberation2
+obconf

stage3/00-install-packages/01-run.sh

@@ -6,4 +6,3 @@ update-alternatives --install /usr/bin/x-www-browser \
 update-alternatives --install /usr/bin/gnome-www-browser \
   gnome-www-browser /usr/bin/chromium-browser 86
 EOF
-

stage3/01-tweaks/00-run.sh

@@ -1,3 +1,3 @@
 #!/bin/bash -e
 
-rm -f ${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf
+rm -f "${ROOTFS_DIR}/etc/systemd/system/dhcpcd.service.d/wait.conf"

stage3/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage4/00-install-packages/00-packages

@@ -1,37 +1,27 @@
 java-common oracle-java8-jdk
-libreoffice-sdbc-hsqldb
-sonic-pi
 python idle python3-pygame python-pygame python-tk
 python3 idle3 python3-tk
 python3-pgzero
 python-serial python3-serial
 python-picamera python3-picamera
-debian-reference-en dillo x2x
+debian-reference-en dillo
 raspberrypi-net-mods raspberrypi-ui-mods
-smartsim penguinspuzzle
 python-pip python3-pip
 python3-numpy
 pypy
-python3-pifacecommon python3-pifacedigitalio python3-pifacedigital-scratch-handler python-pifacecommon python-pifacedigitalio
-minecraft-pi python-minecraftpi
 alacarte rc-gui sense-hat
-claws-mail
 tree
-scratch nuscratch
-greenfoot bluej
-nodered
 libgl1-mesa-dri libgles1-mesa libgles2-mesa xcompmgr
 geany
 piclone
-pulseaudio-module-bluetooth
 wiringpi pigpio python-pigpio python3-pigpio raspi-gpio python-gpiozero python3-gpiozero python3-rpi.gpio
 python-spidev python3-spidev
 python-twython python3-twython
 python-smbus python3-smbus
 python-flask python3-flask
 python-picraft python3-picraft
-python3-codebug-tether python3-codebug-i2c-tether
-python-sense-emu python3-sense-emu sense-emu-tools python-sense-emu-doc
 pprompt
-scratch2
+piwiz
-python3-thonny
+rp-prefapps
+libav-tools
+vlc

stage4/00-install-packages/00-packages-nr

@@ -1,4 +1,3 @@
-libreoffice libreoffice-gtk
 timidity
 pi-package
-realvnc-vnc-server realvnc-vnc-viewer
+realvnc-vnc-server

stage4/00-install-packages/01-packages

@@ -17,3 +17,6 @@ python-scrollphathd python3-scrollphathd
 python-sn3218 python3-sn3218
 python-skywriter python3-skywriter
 python-touchphat python3-touchphat
+python-buttonshim python3-buttonshim
+python-unicornhathd  python3-unicornhathd
+python-pantilthat python3-pantilthat

stage4/00-install-packages/02-packages

@@ -0,0 +1,4 @@
+hunspell-en-gb
+hyphen-en-gb
+wamerican
+wbritish

stage4/01-console-autologin/00-run.sh

@@ -1,3 +1,4 @@
 #!/bin/bash -e
 
-ln -sf /etc/systemd/system/[email protected] ${ROOTFS_DIR}/etc/systemd/system/getty.target.wants/[email protected]
+ln -sf /etc/systemd/system/[email protected] \
+	"${ROOTFS_DIR}/etc/systemd/system/getty.target.wants/[email protected]"

stage4/02-extras/00-run.sh

@@ -1,31 +1,5 @@
 #!/bin/bash -e
 
-HASH=`wget https://api.github.com/repos/KenT2/python-games/git/refs/heads/master -qO -| grep \"sha\" | cut -f 2 -d ':' | cut -f 2 -d \"`
-
-if [ -f files/python_games.hash ]; then
-	HASH_LOCAL=`cat files/python_games.hash`
-fi
-
-if [ ! -e files/python_games.tar.gz ] || [ "$HASH" != "$HASH_LOCAL"  ]; then
-	wget "https://github.com/KenT2/python-games/tarball/master" -O files/python_games.tar.gz
-	echo $HASH > files/python_games.hash
-fi
-
-ln -sf pip3 ${ROOTFS_DIR}/usr/bin/pip-3.2
-
-install -v -o 1000 -g 1000 -d ${ROOTFS_DIR}/home/pi/python_games
-tar xvf files/python_games.tar.gz -C ${ROOTFS_DIR}/home/pi/python_games --strip-components=1
-chown 1000:1000 ${ROOTFS_DIR}/home/pi/python_games -Rv
-chmod +x ${ROOTFS_DIR}/home/pi/python_games/launcher.sh
-
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/BlueJ/ "${ROOTFS_DIR}/home/pi/Documents/BlueJ Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/doc/Greenfoot/ "${ROOTFS_DIR}/home/pi/Documents/Greenfoot Projects"
-rsync -a --chown=1000:1000 ${ROOTFS_DIR}/usr/share/scratch/Projects/Demos/ "${ROOTFS_DIR}/home/pi/Documents/Scratch Projects"
-
 #Alacarte fixes
 install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local"
 install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/.local/share"

stage4/02-extras/files/.gitignore

@@ -1,2 +0,0 @@
-python_games.hash
-python_games.tar.gz

stage4/03-magpi/00-run.sh

@@ -0,0 +1,16 @@
+#!/bin/sh -e
+
+magpi_regex="MagPi[[:digit:]]*.pdf"
+magpi_loc="https://www.raspberrypi.org/magpi-issues"
+magpi_latest="$(curl "$magpi_loc/?C=M;O=D" -s | grep "$magpi_regex" -m 1 -o | head -n 1)"
+
+if [ ! -f "files/$magpi_latest" ]; then
+	find files/ -regextype grep -regex "files/$magpi_regex" -delete
+	wget "$magpi_loc/$magpi_latest" -O "files/$magpi_latest"
+fi
+
+file "files/$magpi_latest" | grep -q "PDF document"
+
+install -v -o 1000 -g 1000 -d "${ROOTFS_DIR}/home/pi/MagPi"
+install -v -o 1000 -g 1000 -m 644 "files/$magpi_latest" "${ROOTFS_DIR}/home/pi/MagPi/"
+

stage4/03-magpi/files/.gitignore

@@ -0,0 +1 @@
+*.pdf

stage4/EXPORT_IMAGE

@@ -1 +1,4 @@
-IMG_SUFFIX="-4GB"
+IMG_SUFFIX=""
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage4/EXPORT_NOOBS

@@ -0,0 +1,2 @@
+NOOBS_NAME="Raspbian"
+NOOBS_DESCRIPTION="A port of Debian with the Raspberry Pi Desktop"

stage4/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi

stage5/00-install-extras/00-packages

@@ -0,0 +1,15 @@
+python3-thonny
+sonic-pi
+scratch nuscratch scratch2
+smartsim
+
+minecraft-pi python-minecraftpi
+python-sense-emu sense-emu-tools python-sense-emu-doc
+
+wolfram-engine
+claws-mail
+greenfoot bluej
+nodered
+realvnc-vnc-viewer
+
+python-games

stage5/00-install-libreoffice/00-packages

@@ -0,0 +1,3 @@
+libreoffice-pi
+libreoffice-help-en-gb
+libreoffice-l10n-en-gb

stage5/00-install-mathematica/00-packages

@@ -1 +0,0 @@
-wolfram-engine

stage5/EXPORT_IMAGE

@@ -1 +1,4 @@
-IMG_SUFFIX=""
+IMG_SUFFIX="-full"
+if [ "${USE_QEMU}" = "1" ]; then
+	export IMG_SUFFIX="${IMG_SUFFIX}-qemu"
+fi

stage5/EXPORT_NOOBS

@@ -1,2 +1,2 @@
-NOOBS_NAME="Raspbian"
+NOOBS_NAME="Raspbian Full"
-NOOBS_DESCRIPTION="A port of Debian jessie for the Raspberry Pi (full desktop version)"
+NOOBS_DESCRIPTION="A port of Debian with desktop and recommended applications"

stage5/prerun.sh

@@ -1,4 +1,5 @@
 #!/bin/bash -e
-if [ ! -d ${ROOTFS_DIR} ]; then
+
+if [ ! -d "${ROOTFS_DIR}" ]; then
 	copy_previous
 fi