Add .gitlab-ci.yml

.gitignore

@@ -5,4 +5,3 @@ postrun.sh
 SKIP
 .pc
 *-pc
-apt-cacher-ng/

.gitlab-ci.yml

@@ -0,0 +1,3 @@
+test:
+  script:
+    - echo "test"

Dockerfile

@@ -1,10 +1,10 @@
-FROM debian:stretch
+FROM debian:jessie
 
 RUN apt-get -y update && \
     apt-get -y install \
         git vim parted \
         quilt realpath qemu-user-static debootstrap zerofree pxz zip dosfstools \
-        bsdtar libcap2-bin rsync grep udev xz-utils \
+        bsdtar libcap2-bin rsync grep \
     && rm -rf /var/lib/apt/lists/*
 
 COPY . /pi-gen/

README.md

@@ -1,24 +1,12 @@
 # pi-gen
-
 _Tool used to create the raspberrypi.org Raspbian images_
 
-
 ### TODO
 1. Documentation
 
-
 ## Dependencies
 
-On Debian-based systems:
-
-```bash
-apt-get install quilt parted realpath qemu-user-static debootstrap zerofree pxz zip \
-dosfstools bsdtar libcap2-bin grep rsync xz-utils
-```
-
-The file `depends` contains a list of tools needed.  The format of this
-package is `<tool>[:<debian-package>]`.
-
+`quilt parted realpath qemu-user-static debootstrap zerofree pxz zip dosfstools bsdtar libcap2-bin grep rsync`
 
 ## Config
 
@@ -41,63 +29,29 @@ The following environment variables are supported:
    will not be included in the image, making it safe to use an `apt-cacher` or
    similar package for development.
 
-   If you have Docker installed, you can set up a local apt caching proxy to
-   like speed up subsequent builds like this:
-
-       docker-compose up -d
-       echo 'APT_PROXY=http://172.17.0.1:3142' >> config
-
- * `BASE_DIR`  (Default: location of `build.sh`)
-
-   **CAUTION**: Currently, changing this value will probably break build.sh
-
-   Top-level directory for `pi-gen`.  Contains stage directories, build
-   scripts, and by default both work and deployment directories.
-
- * `WORK_DIR`  (Default: `"$BASE_DIR/work"`)
-
-   Directory in which `pi-gen` builds the target system.  This value can be
-   changed if you have a suitably large, fast storage location for stages to
-   be built and cached.  Note, `WORK_DIR` stores a complete copy of the target
-   system for each build stage, amounting to tens of gigabytes in the case of
-   Raspbian.
-
- * `DEPLOY_DIR`  (Default: `"$BASE_DIR/deploy"`)
-
-   Output directory for target system images and NOOBS bundles.
-
-
 A simple example for building Raspbian:
 
 ```bash
 IMG_NAME='Raspbian'
 ```
 
-
 ## Docker Build
 
 ```bash
 vi config         # Edit your config file. See above.
 ./build-docker.sh
 ```
-
 If everything goes well, your finished image will be in the `deploy/` folder.
-You can then remove the build container with `docker rm -v pigen_work`
+You can then remove the build container with `docker rm pigen_work`
 
 If something breaks along the line, you can edit the corresponding scripts, and
 continue:
 
-```bash
+```
 CONTINUE=1 ./build-docker.sh
 ```
 
-There is a possibility that even when running from a docker container, the
-installation of `qemu-user-static` will silently fail when building the image
-because `binfmt-support` _must be enabled on the underlying kernel_. An easy
-fix is to ensure `binfmt-support` is installed on the host machine before
-starting the `./build-docker.sh` script (or using your own docker build
-solution).
-
+There is a possibility that even when running from a docker container, the installation of `qemu-user-static` will silently fail when building the image because `binfmt-support` _must be enabled on the underlying kernel_. An easy fix is to ensure `binfmt-support` is installed on the host machine before starting the `./build-docker.sh` script (or using your own docker build solution).
 
 ## Stage Anatomy
 
@@ -132,11 +86,11 @@ maintenance and allows for more easy customization.
    standard console hardware permission groups.
 
    There are a few tools that may not make a whole lot of sense here for
-   development purposes on a minimal system such as basic Python and Lua
+   development purposes on a minimal system such as basic python and lua
    packages as well as the `build-essential` package.  They are lumped right
    in with more essential packages presently, though they need not be with
    pi-gen.  These are understandable for Raspbian's target audience, but if
-   you were looking for something between truly minimal and Raspbian-Lite,
+   you were looking for something between truly minimal and Raspbian-lite,
    here's where you start trimming.
 
  - **Stage 3** - desktop system.  Here's where you get the full desktop system
@@ -144,32 +98,20 @@ maintenance and allows for more easy customization.
    enhancements, etc.  This is a base desktop system, with some development
    tools installed.
 
- - **Stage 4** - Raspbian system meant to fit on a 4GB card.  More development
-   tools, an email client, learning tools like Scratch, specialized packages
-   like sonic-pi, system documentation, office productivity, etc.  This is the
-   stage that installs all of the things that make Raspbian friendly to new
-   users.
-
- - **Stage 5** - The official Raspbian Desktop image. Right now only adds
-   Mathematica.
+ - **Stage 4** - complete Raspbian system.  More development tools, an email
+   client, learning tools like Scratch, specialized packages like sonic-pi and
+   wolfram-engine, system documentation, office productivity, etc.  This is
+   the stage that installs all of the things that make Raspbian friendly to
+   new users.
 
 ### Stage specification
+If you wish to build up to a specified stage (such as building up to stage 2 for a lite system), place an empty file named `SKIP` in each of the `./stage` directories you wish not to include.
 
-If you wish to build up to a specified stage (such as building up to stage 2
-for a lite system), place an empty file named `SKIP` in each of the `./stage`
-directories you wish not to include.
+Then remove the `EXPORT*` files from `./stage4` (if building up to stage 2) or from `./stage2` (if building a minimal system).
 
-Then remove the `EXPORT*` files from `./stage4` (if building up to stage 2) or
-from `./stage2` (if building a minimal system).
-
-```bash
-# Example for building a lite system
-echo "IMG_NAME='Raspbian'" > config
-touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
-rm stage4/EXPORT*
-sudo ./build.sh  # or ./build-docker.sh
 ```
-
-If you wish to build further configurations upon (for example) the lite
-system, you can also delete the contents of `./stage3` and `./stage4` and
-replace with your own contents in the same format.
+# Example for building a lite system
+$ touch ./stage3/SKIP ./stage4/SKIP
+$ rm stage4/EXPORT*
+```
+If you wish to build further configurations upon (for example) the lite system, you can also delete the contents of `./stage3` and `./stage4` and replace with your own contents in the same format.

build-docker.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 DOCKER="docker"
 set +e
 $DOCKER ps >/dev/null 2>&1
@@ -13,10 +12,9 @@ if ! $DOCKER ps >/dev/null; then
 fi
 set -e
 
-
-config_mount=()
+config_mount=
 if [ -f config ]; then
-	config_mount=("-v" "$(pwd)/config:/pi-gen/config:ro")
+	config_mount="-v $(pwd)/config:/pi-gen/config:ro"
 	source config
 fi
 
@@ -47,7 +45,7 @@ fi
 if [ "$CONTAINER_EXISTS" != "" ] && [ "$CONTINUE" != "1" ]; then
 	echo "Container $CONTAINER_NAME already exists and you did not specify CONTINUE=1. Aborting."
 	echo "You can delete the existing container like this:"
-	echo "  docker rm -v $CONTAINER_NAME"
+	echo "  docker rm $CONTAINER_NAME"
 	exit 1
 fi
 
@@ -56,7 +54,6 @@ if [ "$CONTAINER_EXISTS" != "" ]; then
 	trap "echo 'got CTRL+C... please wait 5s';docker stop -t 5 ${CONTAINER_NAME}_cont" SIGINT SIGTERM
 	time $DOCKER run --rm --privileged \
 		--volumes-from="${CONTAINER_NAME}" --name "${CONTAINER_NAME}_cont" \
-		-e IMG_NAME=${IMG_NAME}\
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh;
@@ -65,9 +62,8 @@ if [ "$CONTAINER_EXISTS" != "" ]; then
 else
 	trap "echo 'got CTRL+C... please wait 5s'; docker stop -t 5 ${CONTAINER_NAME}" SIGINT SIGTERM
 	$DOCKER run --name "${CONTAINER_NAME}" --privileged \
-		-e IMG_NAME=${IMG_NAME}\
-		-v "$(pwd)/deploy:/pi-gen/deploy" \
-		"${config_mount[@]}" \
+		-v $(pwd)/deploy:/pi-gen/deploy \
+		${config_mount} \
 		pi-gen \
 		bash -e -o pipefail -c "dpkg-reconfigure qemu-user-static &&
 	cd /pi-gen; ./build.sh &&

build.sh

@@ -128,7 +128,7 @@ if [ -z "${IMG_NAME}" ]; then
 	exit 1
 fi
 
-export IMG_DATE=${IMG_DATE:-"$(date +%Y-%m-%d)"}
+export IMG_DATE=${IMG_DATE:-"$(date -u +%Y-%m-%d)"}
 
 export BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 export SCRIPT_DIR="${BASE_DIR}/scripts"

depends

@@ -10,4 +10,3 @@ capsh:libcap2-bin
 bsdtar
 grep
 rsync
-xz:xz-utils

docker-compose.yml

@@ -1,10 +0,0 @@
-version: '2'
-
-services:
-  apt-cacher-ng:
-    restart: unless-stopped
-    image: sameersbn/apt-cacher-ng:latest
-    ports:
-    - "3142:3142"
-    volumes:
-    - ./apt-cacher-ng:/var/cache/apt-cacher-ng

export-image/00-allow-rerun/files/policy-rc.d

@@ -1,3 +1,2 @@
 #!/bin/sh
-
 exit 101

export-image/02-network/01-run.sh

@@ -1,3 +1,3 @@
-#!/bin/bash -e
+#/bin/bash -e
 
 install -m 644 files/resolv.conf ${ROOTFS_DIR}/etc/

export-image/03-set-partuuid/00-run.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
 IMGID="$(fdisk -l ${IMG_FILE} | sed -n 's/Disk identifier: 0x\([^ ]*\)/\1/p')"

export-image/04-finalise/01-run.sh

@@ -35,8 +35,6 @@ rm -f ${ROOTFS_DIR}/var/lib/dbus/machine-id
 
 true > ${ROOTFS_DIR}/etc/machine-id
 
-ln -nsf /proc/mounts ${ROOTFS_DIR}/etc/mtab
-
 for _FILE in $(find ${ROOTFS_DIR}/var/log/ -type f); do
 	true > ${_FILE}
 done

export-image/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 IMG_FILE="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}.img"
 
 unmount_image ${IMG_FILE}
@@ -9,12 +8,12 @@ rm -f ${IMG_FILE}
 rm -rf ${ROOTFS_DIR}
 mkdir -p ${ROOTFS_DIR}
 
-BOOT_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR}/boot --block-size=1 | cut -f 1)
-TOTAL_SIZE=$(du --apparent-size -s ${EXPORT_ROOTFS_DIR} --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
+BOOT_SIZE=$(du -s ${EXPORT_ROOTFS_DIR}/boot --block-size=1 | cut -f 1)
+TOTAL_SIZE=$(du -s ${EXPORT_ROOTFS_DIR} --exclude var/cache/apt/archives --block-size=1 | cut -f 1)
 
-IMG_SIZE=$((BOOT_SIZE + TOTAL_SIZE + (800 * 1024 * 1024)))
+IMG_SIZE=$((BOOT_SIZE + TOTAL_SIZE + (400 * 1024 * 1024)))
 
-truncate -s ${IMG_SIZE} ${IMG_FILE}
+fallocate -l ${IMG_SIZE} ${IMG_FILE}
 fdisk -H 255 -S 63 ${IMG_FILE} <<EOF
 o
 n
@@ -51,12 +50,8 @@ ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${ROOT_LENGTH} ${IMG_
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"
 
-ROOT_FEATURES="^huge_file"
-if grep -q "metadata_csum" /etc/mke2fs.conf; then
-    ROOT_FEATURES="^metadata_csum,$ROOT_FEATURES"
-fi
 mkdosfs -n boot -F 32 -v $BOOT_DEV > /dev/null
-mkfs.ext4 -O $ROOT_FEATURES $ROOT_DEV > /dev/null
+mkfs.ext4 -O ^huge_file $ROOT_DEV > /dev/null
 
 mount -v $ROOT_DEV ${ROOTFS_DIR} -t ext4
 mkdir -p ${ROOTFS_DIR}/boot

export-noobs/00-release/files/os.json

@@ -1,7 +1,7 @@
 {
     "description": "NOOBS_DESCRIPTION",
     "feature_level": 35120124,
-    "kernel": "4.9",
+    "kernel": "4.4",
     "name": "NOOBS_NAME",
     "password": "raspberry",
     "release_date": "UNRELEASED",

export-noobs/00-release/files/release_notes.txt

@@ -1,18 +1,4 @@
 UNRELEASED:
-  * 
-2017-07-05:
-  * New kernel and firmware
-  * Filesystem created without the metadata_csum feature
-2017-06-21:
-  * Scratch 2 application included
-  * Thonny Python IDE included
-  * New icons with thinner outlines
-  * Volume control more linear in behaviour
-  * Updated Flash player
-  * Updated RealVNC server and viewer
-  * Various tweaks and bugfixes
-  * New kernel and firmware
-2017-04-10:
   * Wolfram Mathematica updated to version 11.0.1
   * Adobe Flash Player updated to version 25.0.0.127
   * Use PARTUUID to support USB boot

scripts/common

@@ -16,9 +16,9 @@ bootstrap(){
 	fi
 
 	capsh --drop=cap_setfcap -- -c "${BOOTSTRAP_CMD} --components=main,contrib,non-free \
-		--arch armhf \
-		--keyring "${STAGE_DIR}/files/raspberrypi.gpg" \
-		$1 $2 $3" || rmdir "$2/debootstrap"
+		--arch armhf\
+		--no-check-gpg \
+		$1 $2 $3"
 }
 export -f bootstrap
 

scripts/dependencies_check

@@ -1,5 +1,5 @@
 # dependencies_check
-# $@	Dependency files to check
+# $@	Dependnecy files to check
 #
 # Each dependency is in the form of a tool to test for, optionally followed by
 # a : and the name of a package if the package on a Debian-ish system is not
@@ -21,7 +21,7 @@ dependencies_check()
 	done
 
 	if [[ "$missing" ]]; then
-		echo "Required dependencies not installed"
+		echo "Reqired dependencies not installed"
 		echo
 		echo "This can be resolved on Debian/Raspbian systems by installing:"
 		echo "$missing"

stage0/01-locale/00-packages

@@ -1 +0,0 @@
-locales

stage0/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	bootstrap jessie ${ROOTFS_DIR} http://mirrordirector.raspbian.org/raspbian/
 fi

stage1/00-boot-files/files/config.txt

@@ -1,5 +1,5 @@
 # For more options and information see
-# http://rpf.io/configtxt
+# http://rpf.io/configtxtreadme
 # Some settings may impact device functionality. See link above for details
 
 # uncomment if you get no picture on HDMI for a default "safe" mode

stage1/01-sys-tweaks/files/policy-rc.d

@@ -1,3 +1,2 @@
 #!/bin/sh
-
 exit 101

stage1/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	copy_previous
 fi

stage2/01-sys-tweaks/00-packages

@@ -1,4 +1,4 @@
-ssh less fbset sudo psmisc strace module-init-tools ed ncdu crda
+ssh locales less fbset sudo psmisc strace module-init-tools ed ncdu crda
 console-setup keyboard-configuration debconf-utils parted unzip
 build-essential manpages-dev python bash-completion gdb pkg-config
 python-rpi.gpio v4l-utils
@@ -16,4 +16,3 @@ apt-listchanges
 usb-modeswitch
 apt-transport-https
 libpam-chksshpwd
-rpi-update

stage2/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	copy_previous
 fi

stage3/00-install-packages/00-packages

@@ -1,7 +1,7 @@
 gstreamer1.0-x gstreamer1.0-omx gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-alsa gstreamer1.0-libav
 xpdf gtk2-engines alsa-utils
 desktop-base
-git
+git-core rpi-update
 omxplayer
 raspberrypi-artwork
 policykit-1

stage3/00-install-packages/01-run.sh

@@ -6,3 +6,4 @@ update-alternatives --install /usr/bin/x-www-browser \
 update-alternatives --install /usr/bin/gnome-www-browser \
   gnome-www-browser /usr/bin/chromium-browser 86
 EOF
+

stage3/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	copy_previous
 fi

stage4/00-install-packages/00-packages

@@ -33,5 +33,3 @@ python-picraft python3-picraft
 python3-codebug-tether python3-codebug-i2c-tether
 python-sense-emu python3-sense-emu sense-emu-tools python-sense-emu-doc
 pprompt
-scratch2
-python3-thonny

stage4/00-install-packages/01-packages

@@ -1,19 +0,0 @@
-python-automationhat python3-automationhat
-python-blinkt python3-blinkt
-python-cap1xxx python3-cap1xxx
-python-drumhat python3-drumhat
-python-envirophat python3-envirophat
-python-explorerhat python3-explorerhat
-python-fourletterphat python3-fourletterphat
-python-microdotphat python3-microdotphat
-python-mote python3-mote
-python-motephat python3-motephat
-python-phatbeat python3-phatbeat
-python-pianohat python3-pianohat
-python-piglow python3-piglow
-python-rainbowhat python3-rainbowhat
-python-scrollphat python3-scrollphat
-python-scrollphathd python3-scrollphathd
-python-sn3218 python3-sn3218
-python-skywriter python3-skywriter
-python-touchphat python3-touchphat

stage4/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	copy_previous
 fi

stage5/EXPORT_NOOBS

@@ -1,2 +1,2 @@
-NOOBS_NAME="Raspbian"
+NOOBS_NAME="Raspbian with PIXEL"
 NOOBS_DESCRIPTION="A port of Debian jessie for the Raspberry Pi (full desktop version)"

stage5/prerun.sh

@@ -1,5 +1,4 @@
 #!/bin/bash -e
-
 if [ ! -d ${ROOTFS_DIR} ]; then
 	copy_previous
 fi