Burrow Raspberry Pi Image
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

384 lines
15 KiB

  1. # Burrow Pi Image
  2. This is a fork of the [pi-gen](https://github.com/RPi-Distro/pi-gen) project, the "tool used to create the raspberrypi.org Raspbian images". `pi-gen` serves as a decent base becuase it:
  3. * Builds compact images that self-inflate on the device during the first boot
  4. * Provides a comfortable development workflow
  5. I modified `pi-gen` with the following features:
  6. * Rudefox Burrow installed
  7. * Auto-login
  8. * Read-only filesystem (as a security measure to prevent you from accidentally storing sensitive information on this low-security hardware)
  9. * Most user-space networking packages removed (kernel still included networking capabilities)
  10. ## Dependencies
  11. pi-gen runs on Debian based operating systems. Currently it is only supported on
  12. either Debian Buster or Ubuntu Xenial and is known to have issues building on
  13. earlier releases of these systems. On other Linux distributions it may be possible
  14. to use the Docker build described below.
  15. To install the required dependencies for pi-gen you should run:
  16. ```bash
  17. apt-get install coreutils quilt parted qemu-user-static debootstrap zerofree zip \
  18. dosfstools bsdtar libcap2-bin grep rsync xz-utils file git curl bc
  19. ```
  20. The file `depends` contains a list of tools needed. The format of this
  21. package is `<tool>[:<debian-package>]`.
  22. ## Config
  23. Upon execution, `build.sh` will source the file `config` in the current
  24. working directory. This bash shell fragment is intended to set needed
  25. environment variables.
  26. The following environment variables are supported:
  27. * `IMG_NAME` (Default: `RudefoxBurrow`)
  28. The name of the image to build with the current stage directories. Setting
  29. `IMG_NAME=Raspbian` is logical for an unmodified RPi-Distro/pi-gen build,
  30. but you should use something else for a customized version. Export files
  31. in stages may add suffixes to `IMG_NAME`.
  32. * `RUDEFOX_REPO_USERNAME` (Default: `anonymous`)
  33. If the value is set to `anonymous`, then Burrow is pulled from the publicly available Rudefox repository. Otherwise, the build attempts to login to the Rudefox private repository for access to SNAPSHOT builds using the provided `RUDEFOX_REPO_USERNAME` / `RUDEFOX_REPO_PASSWORD` credentials.
  34. * `RUDEFOX_BURROW_VERSION` (Default: `latest`)
  35. If the value is set to `latest`, then the latest version is pulled from the Rudefox repository and installed. Otherwise, the specified version is downloadedand installed.
  36. * `RELEASE` (Default: buster)
  37. The release version to build images against. Valid values are jessie, stretch
  38. buster, bullseye, and testing.
  39. * `APT_PROXY` (Default: unset)
  40. If you require the use of an apt proxy, set it here. This proxy setting
  41. will not be included in the image, making it safe to use an `apt-cacher` or
  42. similar package for development.
  43. If you have Docker installed, you can set up a local apt caching proxy to
  44. like speed up subsequent builds like this:
  45. docker-compose up -d
  46. echo 'APT_PROXY=' >> config
  47. * `BASE_DIR` (Default: location of `build.sh`)
  48. **CAUTION**: Currently, changing this value will probably break build.sh
  49. Top-level directory for `pi-gen`. Contains stage directories, build
  50. scripts, and by default both work and deployment directories.
  51. * `WORK_DIR` (Default: `"$BASE_DIR/work"`)
  52. Directory in which `pi-gen` builds the target system. This value can be
  53. changed if you have a suitably large, fast storage location for stages to
  54. be built and cached. Note, `WORK_DIR` stores a complete copy of the target
  55. system for each build stage, amounting to tens of gigabytes in the case of
  56. Raspbian.
  57. **CAUTION**: If your working directory is on an NTFS partition you probably won't be able to build. Make sure this is a proper Linux filesystem.
  58. * `DEPLOY_DIR` (Default: `"$BASE_DIR/deploy"`)
  59. Output directory for target system images and NOOBS bundles.
  60. * `DEPLOY_ZIP` (Default: `1`)
  61. Setting to `0` will deploy the actual image (`.img`) instead of a zipped image (`.zip`).
  62. * `USE_QEMU` (Default: `"0"`)
  63. Setting to '1' enables the QEMU mode - creating an image that can be mounted via QEMU for an emulated
  64. environment. These images include "-qemu" in the image file name.
  65. * `LOCALE_DEFAULT` (Default: "en_CA.UTF-8" )
  66. Default system locale.
  67. * `TARGET_HOSTNAME` (Default: "burrow" )
  68. Setting the hostname to the specified value.
  69. * `KEYBOARD_MODEL` (Default: "Generic 101-key PC" )
  70. Default keyboard model.
  71. To get the current value from a running system, run `debconf-show
  72. keyboard-configuration` and look at the
  73. `keyboard-configuration/model` value.
  74. * `KEYBOARD_KEYMAP` (Default: "us" )
  75. Default keyboard keymap.
  76. To get the current value from a running system, run `debconf-show
  77. keyboard-configuration` and look at the
  78. `keyboard-configuration/xkb-keymap` value.
  79. * `KEYBOARD_LAYOUT` (Default: "English (UK)" )
  80. Default keyboard layout.
  81. To get the current value from a running system, run `debconf-show
  82. keyboard-configuration` and look at the
  83. `keyboard-configuration/variant` value.
  84. * `TIMEZONE_DEFAULT` (Default: "Europe/London" )
  85. Default keyboard layout.
  86. To get the current value from a running system, look in
  87. `/etc/timezone`.
  88. * `FIRST_USER_NAME` (Default: "rudefox" )
  89. Username for the first user
  90. * `FIRST_USER_PASS` (Default: "burrow")
  91. Password for the first user
  92. * `WPA_ESSID`, `WPA_PASSWORD` and `WPA_COUNTRY` (Default: unset)
  93. If these are set, they are use to configure `wpa_supplicant.conf`, so that the Raspberry Pi can automatically connect to a wifi network on first boot. If `WPA_ESSID` is set and `WPA_PASSWORD` is unset an unprotected wifi network will be configured. If set, `WPA_PASSWORD` must be between 8 and 63 characters.
  94. * `ENABLE_SSH` (Default: `0`)
  95. Setting to `1` will enable ssh server for remote log in. Note that if you are using a common password such as the defaults there is a high risk of attackers taking over you Raspberry Pi.
  96. * `STAGE_LIST` (Default: `stage[0-2]`)
  97. The default setting builds only the "lite" image for Rudefox Burrow, which is all you need.
  98. If set, then instead of working through the numeric stages in order, this list will be followed. For example setting to `"stage0 stage1 mystage stage2"` will run the contents of `mystage` before stage2. Note that quotes are needed around the list. An absolute or relative path can be given for stages outside the pi-gen directory.
  99. A simple example for building RaspbianBurrow:
  100. ```bash
  101. APT_PROXY=
  102. TIMEZONE_DEFAULT="America/Toronto"
  103. ```
  104. The config file can also be specified on the command line as an argument the `build.sh` or `build-docker.sh` scripts.
  105. ```
  106. ./build.sh -c myconfig
  107. ```
  108. This is parsed after `config` so can be used to override values set there.
  109. ## How the build process works
  110. The following process is followed to build images:
  111. * Loop through all of the stage directories in alphanumeric order
  112. * Move on to the next directory if this stage directory contains a file called
  113. "SKIP"
  114. * Run the script ```prerun.sh``` which is generally just used to copy the build
  115. directory between stages.
  116. * In each stage directory loop through each subdirectory and then run each of the
  117. install scripts it contains, again in alphanumeric order. These need to be named
  118. with a two digit padded number at the beginning.
  119. There are a number of different files and directories which can be used to
  120. control different parts of the build process:
  121. - **00-run.sh** - A unix shell script. Needs to be made executable for it to run.
  122. - **00-run-chroot.sh** - A unix shell script which will be run in the chroot
  123. of the image build directory. Needs to be made executable for it to run.
  124. - **00-debconf** - Contents of this file are passed to debconf-set-selections
  125. to configure things like locale, etc.
  126. - **00-packages** - A list of packages to install. Can have more than one, space
  127. separated, per line.
  128. - **00-packages-nr** - As 00-packages, except these will be installed using
  129. the ```--no-install-recommends -y``` parameters to apt-get.
  130. - **00-patches** - A directory containing patch files to be applied, using quilt.
  131. If a file named 'EDIT' is present in the directory, the build process will
  132. be interrupted with a bash session, allowing an opportunity to create/revise
  133. the patches.
  134. * If the stage directory contains files called "EXPORT_NOOBS" or "EXPORT_IMAGE" then
  135. add this stage to a list of images to generate
  136. * Generate the images for any stages that have specified them
  137. It is recommended to examine build.sh for finer details.
  138. ## Docker Build
  139. Docker can be used to perform the build inside a container. This partially isolates
  140. the build from the host system, and allows using the script on non-debian based
  141. systems (e.g. Fedora Linux). The isolate is not complete due to the need to use
  142. some kernel level services for arm emulation (binfmt) and loop devices (losetup).
  143. To build:
  144. ```bash
  145. vi config # Edit your config file. See above.
  146. ./build-docker.sh
  147. ```
  148. If everything goes well, your finished image will be in the `deploy/` folder.
  149. You can then remove the build container with `docker rm -v pigen_work`
  150. If something breaks along the line, you can edit the corresponding scripts, and
  151. continue:
  152. ```bash
  153. CONTINUE=1 ./build-docker.sh
  154. ```
  155. To examine the container after a failure you can enter a shell within it using:
  156. ```bash
  157. sudo docker run -it --privileged --volumes-from=pigen_work pi-gen /bin/bash
  158. ```
  159. After successful build, the build container is by default removed. This may be undesired when making incremental changes to a customized build. To prevent the build script from remove the container add
  160. ```bash
  161. PRESERVE_CONTAINER=1 ./build-docker.sh
  162. ```
  163. There is a possibility that even when running from a docker container, the
  164. installation of `qemu-user-static` will silently fail when building the image
  165. because `binfmt-support` _must be enabled on the underlying kernel_. An easy
  166. fix is to ensure `binfmt-support` is installed on the host machine before
  167. starting the `./build-docker.sh` script (or using your own docker build
  168. solution).
  169. ## Stage Anatomy
  170. ### Raspbian Stage Overview
  171. The build of Raspbian is divided up into several stages for logical clarity
  172. and modularity. This causes some initial complexity, but it simplifies
  173. maintenance and allows for more easy customization.
  174. - **Stage 0** - bootstrap. The primary purpose of this stage is to create a
  175. usable filesystem. This is accomplished largely through the use of
  176. `debootstrap`, which creates a minimal filesystem suitable for use as a
  177. base.tgz on Debian systems. This stage also configures apt settings and
  178. installs `raspberrypi-bootloader` which is missed by debootstrap. The
  179. minimal core is installed but not configured, and the system will not quite
  180. boot yet.
  181. - **Stage 1** - truly minimal system. This stage makes the system bootable by
  182. installing system files like `/etc/fstab`, configures the bootloader, makes
  183. the network operable, and installs packages like raspi-config. At this
  184. stage the system should boot to a local console from which you have the
  185. means to perform basic tasks needed to configure and install the system.
  186. This is as minimal as a system can possibly get, and its arguably not
  187. really usable yet in a traditional sense yet. Still, if you want minimal,
  188. this is minimal and the rest you could reasonably do yourself as sysadmin.
  189. - **Stage 2** - lite system. This stage produces the Raspbian-Lite image. It
  190. installs some optimized memory functions, sets timezone and charmap
  191. defaults, installs fake-hwclock and ntp, wifi and bluetooth support,
  192. dphys-swapfile, and other basics for managing the hardware. It also
  193. creates necessary groups and gives the pi user access to sudo and the
  194. standard console hardware permission groups.
  195. There are a few tools that may not make a whole lot of sense here for
  196. development purposes on a minimal system such as basic Python and Lua
  197. packages as well as the `build-essential` package. They are lumped right
  198. in with more essential packages presently, though they need not be with
  199. pi-gen. These are understandable for Raspbian's target audience, but if
  200. you were looking for something between truly minimal and Raspbian-Lite,
  201. here's where you start trimming.
  202. - **Stage 3** - desktop system. Here's where you get the full desktop system
  203. with X11 and LXDE, web browsers, git for development, Raspbian custom UI
  204. enhancements, etc. This is a base desktop system, with some development
  205. tools installed.
  206. - **Stage 4** - Normal Raspbian image. System meant to fit on a 4GB card. This is the
  207. stage that installs most things that make Raspbian friendly to new
  208. users like system documentation.
  209. - **Stage 5** - The Raspbian Full image. More development
  210. tools, an email client, learning tools like Scratch, specialized packages
  211. like sonic-pi, office productivity, etc.
  212. ### Stage specification
  213. If you wish to build up to a specified stage (such as building up to stage 2
  214. for a lite system), place an empty file named `SKIP` in each of the `./stage`
  215. directories you wish not to include.
  216. Then add an empty file named `SKIP_IMAGES` to `./stage4` and `./stage5` (if building up to stage 2) or
  217. to `./stage2` (if building a minimal system).
  218. ```bash
  219. # Example for building a lite system
  220. echo "IMG_NAME='Raspbian'" > config
  221. touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
  222. touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
  223. sudo ./build.sh # or ./build-docker.sh
  224. ```
  225. If you wish to build further configurations upon (for example) the lite
  226. system, you can also delete the contents of `./stage3` and `./stage4` and
  227. replace with your own contents in the same format.
  228. ## Skipping stages to speed up development
  229. If you're working on a specific stage the recommended development process is as
  230. follows:
  231. * Add a file called SKIP_IMAGES into the directories containing EXPORT_* files
  232. (currently stage2, stage4 and stage5)
  233. * Add SKIP files to the stages you don't want to build. For example, if you're
  234. basing your image on the lite image you would add these to stages 3, 4 and 5.
  235. * Run build.sh to build all stages
  236. * Add SKIP files to the earlier successfully built stages
  237. * Modify the last stage
  238. * Rebuild just the last stage using ```sudo CLEAN=1 ./build.sh```
  239. * Once you're happy with the image you can remove the SKIP_IMAGES files and
  240. export your image to test
  241. # Troubleshooting
  242. ## `64 Bit Systems`
  243. Please note there is currently an issue when compiling with a 64 Bit OS. See https://github.com/RPi-Distro/pi-gen/issues/271
  244. ## `binfmt_misc`
  245. Linux is able execute binaries from other architectures, meaning that it should be
  246. possible to make use of `pi-gen` on an x86_64 system, even though it will be running
  247. ARM binaries. This requires support from the [`binfmt_misc`](https://en.wikipedia.org/wiki/Binfmt_misc)
  248. kernel module.
  249. You may see the following error:
  250. ```
  251. update-binfmts: warning: Couldn't load the binfmt_misc module.
  252. ```
  253. To resolve this, ensure that the following files are available (install them if necessary):
  254. ```
  255. /lib/modules/$(uname -r)/kernel/fs/binfmt_misc.ko
  256. /usr/bin/qemu-arm-static
  257. ```
  258. You may also need to load the module by hand - run `modprobe binfmt_misc`.