From b9b8df3d113288765e893e181e6c3ea54cab5f43 Mon Sep 17 00:00:00 2001 From: Joseph Carter Date: Sun, 4 Sep 2016 21:02:39 -0700 Subject: [PATCH 1/2] README.md: Working toward basic documentation (#10) * README.md: document config file Still pretty sparse, but as this bit is necessary to actually use pi-gen, here it is. * README.md: Overview of Raspbian's stages * README.md: Stages -> Raspbian Stages + heading Decided that the Stages heading probably made more sense as being retitled to refer specifically to Raspbian since it has no direct bearing on pi-gen itself which could produce any Debianish system for the pi, and also that there ought to be a Stage Anatomy section first talking about the nuts and bolts of what goes into these stages. Stage Anatomy is just a heading for now. * README.md: Make Config section less conversational Left the stage overview as it was since it's intended more as a plain language explanation than a technically complete look. I believe that kind of information is useful to have, but perhaps as README.md becomes longer it may be advisable to move it to another location. --- README.md | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/README.md b/README.md index b8ac6e5..9ff754f 100644 --- a/README.md +++ b/README.md @@ -6,3 +6,70 @@ #Dependencies `quilt kpartx realpath qemu-user-static debootstrap zerofree pxz` + +#Config + +Upon execution, `build.sh` will source the file `config` in the current +working directory. This bash shell fragment is intended to set needed +environment variables. + +The following environment variables are supported: + + * `IMG_NAME`, the name of the distribution to build (required) + * `APT_PROXY`, proxy/cache URL to be included in the build + +A simple example for building Raspbian: + +```bash +IMG_NAME='Raspbian' +``` + +#Stage Anatomy + + + +#Raspbian Stage Overview + +The build of Raspbian is divided up into several stages for logical clarity +and modularity. This causes some initial complexity, but it simplifies +maintenance and allows for more easy customization. + + - Stage 0, bootstrap. The primary purpose of this stage is to create a + usable filesystem. This is accomplished largely through the use of + `debootstrap`, which creates a minimal filesystem suitable for use as a + base.tgz on Debian systems. This stage also configures apt settings and + installs `raspberrypi-bootloader` which is missed by debootstrap. The + minimal core is installed but not configured, and the system will not quite + boot yet. + + - Stage 1, truly minimal system. This stage makes the system bootable by + installing system files like `/etc/fstab`, configures the bootloader, makes + the network operable, and installs packages like raspi-config. At this + stage the system should boot to a local console from which you have the + means to perform basic tasks needed to configure and install the system. + This is as minimal as a system can possibly get, and its arguably not + really usable yet in a traditional sense yet. Still, if you want minimal, + this is minimal and the rest you could reasonably do yourself as sysadmin. + + - State 2, lite system. This stage produces the Raspbian-Lite image. It + installs some optimized memory functions, sets timezone and charmap + defaults, installs fake-hwclock and ntp, wifi and bluetooth support, + dphys-swapfile, and other basics for managing the hardware. It also + creates necessary groups and gives the pi user access to sudo and the + standard console hardware permission groups. + + There are a few tools that may not make a whole lot of sense here for + development purposes on a minimal system such as basic python and lua + packages as well as the `build-essential` package. They are lumped right + in with more essential packages presently, though they need not be with + pi-gen. These are understandable for Raspbian's target audience, but if + you were looking for something between truly minimal and Raspbian-lite, + here's where you start trimming. + + - Stage 3, desktop system. Here's where you get the full desktop system + with X11 and LXDE, web browsers, git for development, Raspbian custom UI + enhancements, etc. This is a base desktop system, with some development + tools installed. + + - Stage 4, complete Raspbian system. More development tools, large packages + like LibreOffice, email, sonic-pi, wolfram-engine, etc. All the things. From abc3e45727d0ed1a7804ed68a8a1c1b096b8372c Mon Sep 17 00:00:00 2001 From: Joseph Carter Date: Wed, 7 Sep 2016 13:32:36 -0700 Subject: [PATCH 2/2] build.sh: Support comments in package files (#14) * build.sh: Support comments in package files This patch allows the use of hash comments inside patch files. It's a little ugly, but it strips comments and collapses all whitespace down to single space characters between package names. It handles comments anywhere in a line, as well. Was unsure if \ continuation of the long sed line or the inclusion of a couple of lines of comments explaining what the sed expressions are doing would be appreciated, so didn't include them in this patch. * build.sh: whitespace fix * build.sh: Use sed script for packages files Broke the sed expressions out of build.sh and put them their own documented sed script. This greatly improves readability and avoids build.sh getting messier. Broke the substitution command into two separate subs. The first just deletes comments, and the second collapses all whitespace into a single space. This too is easier to read, and catches a couple of edge cases that would result it not all whitespace being collapsed. The result may still have (one) leading and/or trailing space, which is acceptable. --- build.sh | 6 ++++-- scripts/remove-comments.sed | 11 +++++++++++ 2 files changed, 15 insertions(+), 2 deletions(-) create mode 100644 scripts/remove-comments.sed diff --git a/build.sh b/build.sh index 5b745e8..52f7a82 100755 --- a/build.sh +++ b/build.sh @@ -16,7 +16,8 @@ EOF fi if [ -f ${i}-packages-nr ]; then log "Begin ${SUB_STAGE_DIR}/${i}-packages-nr" - PACKAGES=`cat $i-packages-nr | tr '\n' ' '` + PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages-nr)" + PACKAGES="$(sed -e "$sed_expr_packages" < ${i}-packages-nr)" if [ -n "$PACKAGES" ]; then on_chroot sh -e - << EOF apt-get install --no-install-recommends -y $PACKAGES @@ -26,7 +27,7 @@ EOF fi if [ -f ${i}-packages ]; then log "Begin ${SUB_STAGE_DIR}/${i}-packages" - PACKAGES=`cat $i-packages | tr '\n' ' '` + PACKAGES="$(sed -f "${SCRIPT_DIR}/remove-comments.sed" < ${i}-packages)" if [ -n "$PACKAGES" ]; then on_chroot sh -e - << EOF apt-get install -y $PACKAGES @@ -77,6 +78,7 @@ EOF log "End ${SUB_STAGE_DIR}" } + run_stage(){ log "Begin ${STAGE_DIR}" STAGE=$(basename ${STAGE_DIR}) diff --git a/scripts/remove-comments.sed b/scripts/remove-comments.sed new file mode 100644 index 0000000..2a6889f --- /dev/null +++ b/scripts/remove-comments.sed @@ -0,0 +1,11 @@ +# Deletes comments and collapses whitespace in ##-packages files + +# Append (N)ext line to buffer +# if (!)not ($)buffer is EOF, (b)ranch to (:)label loop +:loop +N +$ !b loop + +# Buffer is "line1\nline2\n...lineN", del comments and collapse whitespace +s/#[^\n]*//g +s/[[:space:]]\{1,\}/ /g