Building Linux-based BusyBox Distro on M-series Macs

In the world of embedded systems and operating system development, relying on heavy virtualization tools like Docker Desktop or VMware can sometimes obscure what is actually happening under the hood. I wanted to build a Linux kernel from scratch (upstream) and a minimal root filesystem using BusyBox, all on my Apple Silicon.

In this post, we will explore how to use the Lima project to create a native Linux build factory, compile Torvalds' upstream kernel, and create a bootable "Demo OS" that runs on the QEMU test bench.

The Stack

Before diving into commands, let's briefly define our toolkit:

• Lima (Linux Machines): A lightweight virtualization tool for macOS. It acts as our "Factory," giving us a Fedora environment to compile code that macOS cannot build natively.
• Linux Kernel (Upstream): The core operating system, pulled directly from Linus Torvalds' source tree.
• BusyBox: Often called the "Swiss Army Knife of Embedded Linux," this provides all our userland tools (ls, sh, mount) in a single binary.
• QEMU: An emulator and virtualizer. We use it as our "Test Bench" to boot our custom kernel on the Mac.

The ones we'll use in the host machine (Mac OS) will be Lima and QEMU. Let's install them using homebrew:

$ brew install qemu lima

Setting Up the Factory (Lima)

We need a Linux environment to compile the kernel. I chose Fedora for its robust package management.

First, create a Fedora instance using Lima:

$ limactl start template://fedora --name=kernel-builder --cpus=4 --memory=8
$ limactl shell kernel-builder

Once inside the shell, we need to install the development tools for building Linux kernel and BusyBox. We also need specific static libraries for BusyBox later.

$ sudo dnf update -y

# Install standard build tools
$ sudo dnf group install development-tools
$ sudo dnf group install c-development

# Install kernel dependencies and static libraries for BusyBox
$ sudo dnf install -y ncurses-devel openssl-devel elfutils-libelf-devel dwarves bc cpio perl glibc-static libxcrypt-static

Building the Kernel (Linux)

We will use the upstream source directly from git. This ensures we are working with the latest features. Because, why not?

# Go the VM's not-mounted $HOME directory.
$ cd ~

# Clone the Linux repo.
$ git clone https://github.com/torvalds/linux.git
$ cd linux

# Build the config first, then the kernel. It'll take some time.
$ make defconfig
$ make -j$(nproc)

After compilation, our kernel is ready at arch/arm64/boot/Image.

Building the Userland (BusyBox)

A kernel is useless without applications. We will build a minimal OS using BusyBox version 1.36 (stable). I'm particularly using this version since with others I had issues compiling on ARM.

# Go to the VM's not-mounted $HOME directory.
$ cd ~

# Clone the BusyBox repo.
$ git clone https://git.busybox.net/busybox/
$ cd busybox

# Checkout to the 1.36-stable branch.
$ git checkout origin/1_36_stable

# Build default config.
$ make defconfig

There are some gotchas in building BusyBox on Fedora:

1. Static Linking: We must tell BusyBox to include all libraries inside the binary so it can run standalone.
2. Missing tc Support: The Traffic Control (tc) applet fails to build against modern kernel headers.
3. The Menuconfig Loop: Running make menuconfig might fail due to missing helpers, so we will edit the config directly. If you can achieve having menuconfig, let me know to update this blog post as well!

Run the following to fix these issues and build:

# Enable Static Linking
$ sed -i 's/^# CONFIG_STATIC is not set/CONFIG_STATIC=y/' .config

# Disable the broken 'tc' applet
$ sed -i 's/^CONFIG_TC=y/# CONFIG_TC is not set/' .config

# Build and Install
$ make -j$(nproc)
$ make install

This creates an _install directory containing our OS. We need to package this into a cpio archive. Note that BusyBox creates a linuxrc link, but modern kernels look for /init. We will create our own init script:

$ cd _install
$ mkdir -p dev proc sys

# Create the startup script
$ cat > init <<EOF
#!/bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
echo "-----------------------------------"
echo "Success! Your Kernel and Demo OS are working."
echo "-----------------------------------"
exec /bin/sh
EOF

$ chmod +x init

# Compress into a disk image
$ find . -print0 | cpio --null -ov --format=newc | gzip > ~/initramfs.cpio.gz

The Test Bench (QEMU)

Now we move the artifacts from our Linux "Factory" back to our Mac "Test Bench."

Exit the Lima shell (exit) and run the following on your Mac terminal:

# Copy files from the Lima instance
$ limactl copy kernel-builder:~/linux/arch/arm64/boot/Image .
$ limactl copy kernel-builder:~/initramfs.cpio.gz .

Finally, we boot our custom kernel. We use -accel hvf to leverage Apple's Hypervisor Framework for native speeds.

$ qemu-system-aarch64 \
  -M virt \
  -accel hvf \
  -cpu host \
  -nographic \
  -kernel Image \
  -initrd initramfs.cpio.gz \
  -append "console=ttyAMA0"

If successful, you will be greeted by your custom boot message and a # shell prompt. You have just built and booted a Linux distribution from scratch on Apple Silicon.