Using SWUpdate with Qt

In a previous article about software update, we gave a general overview of different update strategies. In this new article, we will take a closer look at SWUpdate and demonstrate how it can be used in conjunction with Qt to implement a partition swapping mechanism.

Yocto Image

In this article, we will create a Yocto image for the Toradex Verdin iMX8M Plus using the meta-embear layer. To create the image, we will need to follow these steps:

$ mkdir boot2qt && cd boot2qt
$ repo init -u git://code.qt.io/yocto/boot2qt-manifest.git -m v6.3.0.xml
$ repo sync
$ cd sources
$ git clone https://github.com/embear-engineering/meta-embear.git
$ git clone https://github.com/sbabic/meta-swupdate.git -b honister
$ cd ..
$ MACHINE=verdin-imx8mp . setup-environment.sh

Add meta-embear and meta-swupdate to conf/bblayers.conf:

BBLAYERS ?= " \
  ${BSPDIR}/sources/poky/meta \
  ${BSPDIR}/sources/poky/meta-poky \
  ${BSPDIR}/sources/meta-freescale \
  ${BSPDIR}/sources/meta-freescale-3rdparty \
  ${BSPDIR}/sources/meta-toradex-bsp-common \
  ${BSPDIR}/sources/meta-toradex-nxp \
  ${BSPDIR}/sources/meta-openembedded/meta-oe \
  ${BSPDIR}/sources/meta-openembedded/meta-python \
  ${BSPDIR}/sources/meta-openembedded/meta-networking \
  ${BSPDIR}/sources/meta-openembedded/meta-initramfs \
  ${BSPDIR}/sources/meta-openembedded/meta-multimedia \
  ${BSPDIR}/sources/meta-boot2qt/meta-boot2qt \
  ${BSPDIR}/sources/meta-boot2qt/meta-boot2qt-distro \
  ${BSPDIR}/sources/meta-mingw \
  ${BSPDIR}/sources/meta-qt6 \
  ${BSPDIR}/sources/meta-swupdate \
  ${BSPDIR}/sources/meta-embear \
  "

Now we can build the embear-boot2qt-image:

$ bitbake embear-boot2qt-image

There is no Toradex Easy Installer image output, this has been disabled in meta-embear because we need to change the configuration anyway. We need to create our own bundle (because we want multiple partitions). We start as follows:

$ cd && mkdir embear-boot2qt && cd embear-boot2qt
$ cp ~/boot2qt/build-verdin-imx8mp/tmp/deploy/images/verdin-imx8mp/embear-boot2qt-image-verdin-imx8mp.tar.zst .
$ cp ~/boot2qt/build-verdin-imx8mp/tmp/deploy/images/verdin-imx8mp/imx-boot .

Now we create a minimal image.json file that looks like this:

{
    "config_format": "4",
    "autoinstall": false,
    "name": "Embear Boot2Qt Image",
    "description": "Demo image for open source Boot2Qt",
    "version": "Qt 6.3.0",
    "release_date": "2022-12-02",
    "supported_product_ids": [
        "0058",
        "0061",
        "0063",
        "0064",
        "0066"
    ],
    "blockdevs": [
        {
            "name": "emmc",
            "partitions": [
                {
                    "partition_size_nominal": 4048,
                    "want_maximised": false,
                    "content": {
                        "label": "RFS1",
                        "filesystem_type": "ext4",
                        "mkfs_options": "-E nodiscard",
                        "filename": "embear-boot2qt-image-verdin-imx8mp.tar.zst",
                        "uncompressed_size": 1741.73828125
                    }
                },
                {
                    "partition_size_nominal": 4048,
                    "want_maximised": false,
                    "content": {
                        "label": "RFS2",
                        "filesystem_type": "ext4",
                        "mkfs_options": "-E nodiscard"
                    }
                },
                {
                    "partition_size_nominal": 2048,
                    "want_maximised": true,
                    "content": {
                        "label": "config",
                        "filesystem_type": "ext4",
                        "mkfs_options": "-E nodiscard"
                    }
                }
            ]
        },
        {
            "name": "emmc-boot0",
            "erase": true,
            "content": {
                "filesystem_type": "raw",
                "rawfiles": [
                    {
                        "filename": "imx-boot",
                        "dd_options": "seek=0"
                    }
                ]
            }
        }
    ]
}

We can now copy the whole folder to a USB stick and install the image using the Toradex Easy Installer. The resulting image can be used as embear-initial.tar.zst in the SWUpdate section. If we don't want to build the image ourselves, we can use the following pre-built images for the Verdin iMX8MP from Toradex.

• embear-boot2qt-image-verdin-imx8mp.tar.zst (needs to be renamed to embear-boot2qt-image-verdin-imx8mp.tar.zst for the Toradex Easy Installer)
• imx-boot

SWUpdate

This article will focus on the partition swap update scenario supported by SWUpdate. Although SWUpdate supports several update scenarios, we will only discuss this particular one. The documentation for SWUpdate can be found here: swupdate

SWUpdate expects a cpio archive containing a sw-description file as the first file in the archive. Here is an example of such a file:

{
    version = "1.0.0";
    description = "Embear Firmware Update Demo";
    hardware-compatibility: [ "1.0", "1.2", "1.3"];
    files: (
        {
            filename = "image.tar.zst";
            type = "archive";
            compressed = "zstd";
            device = "/dev/update";
            filesystem = "ext4";
            path = "/";
        }
    );

    scripts: (
        {
            filename = "update.sh";
            type = "shellscript";
        }
    );
}

The files and scripts referenced in the sw-description must also be part of the cpio archive. In this scenario we use a tar.zst file to store the rootfs and install it on /dev/update which is an ext4 partition. The update.sh script is run by SWUpdate to perform some pre- and post-installation steps.

#!/bin/sh

if [ $# -lt 1 ]; then
    exit 0;
fi

function get_current_root_device
{
    PARTUUID=$(cat /proc/cmdline | sed 's/root=PARTUUID=\([^ ]*\).*/\1/')
    CURRENT_ROOT=$(readlink -f /dev/disk/by-partuuid/$PARTUUID)
}

function get_update_part
{
    CURRENT_PART="${CURRENT_ROOT: -1}"
    if [ $CURRENT_PART = "1" ]; then
        UPDATE_PART="2";
    else
        UPDATE_PART="1";
    fi
}

function get_update_device
{
    UPDATE_ROOT=${CURRENT_ROOT%?}${UPDATE_PART}
}

function format_update_device
{
    umount -q $UPDATE_ROOT
    mkfs.ext4 -q $UPDATE_ROOT -L RFS${UPDATE_PART} -E nodiscard
}

if [ $1 == "preinst" ]; then
    # get the current root device
    get_current_root_device

    # get the device to be updated
    get_update_part
    get_update_device

    # format the device to be updated
    format_update_device

    # create a symlink for the update process
    ln -sf $UPDATE_ROOT /dev/update
fi

if [ $1 == "postinst" ]; then
    get_current_root_device

    get_update_part

    mmcdev=${CURRENT_ROOT::-2}

    # eMMC needs to have reliable write on
    parted $mmcdev set $UPDATE_PART boot on &> /dev/null
    parted $mmcdev set $CURRENT_PART boot off &> /dev/null
fi

The files are based on what Variscite describes in their Wiki.

The preinst step will figure out which partition is currently active and then create a link /dev/update that points to the currently inactive root partition.

The postinst step will then use parted to mark the previously inactive partition as active and the active partition as inactive. We use the MSDOS boot partition flag to do this. We need to make sure that updating this flag is atomic and doesn't corrupt the partition table. This should be the case because the partition table fits into the block size that the eMMC can update atomically. In this example it may happen that two partitions have the boot partition flag set. We can do this because U-Boot's distroboot will take the first active partition in this case. Note, however, that this must be verified. Alternatively, we would have to use a tool or mechanism that swaps the boot partition at once (e.g. fdisk).

Scripts

There are some scripts available on Github. We can use these scripts to quickly create new images with some additional binaries or config files without using Yocto.

Script Name	Description
create-tar.sh	Create a tar.zstd from different overlays
create-swu.sh	Create a cpio archive which includes a swupdate description
create-qt-image.sh	Create an ostree image for qtota, it is not used in this article
qtostree	qtostree script from qtota with some modifications, it is not used in this article

Creating a new image

We use the create-tar script to generate a new tar file from a base directory structure and some overlays that are copied over the base directory. This allows us to quickly modify an existing image. The output is a tar.zstd file. Here is an example:

git clone git@github.com:embear-engineering/sw-update-scripts.git
cd sw-update-scripts
wget https://files.embear.ch/embear-initial.tar.zst
wget https://files.embear.ch/overlay_base.tar.zst
wget https://files.embear.ch/overlay_swupdate.tar.zst
mkdir rootfs
tar -xf embear-initial -C rootfs
mkdir overlay_base
tar -xf overlay_base.tar.zst -C overlay_base
mkdir overlay_swupdate
tar -xf overlay_swupdate -C overlay_swupdate
fakeroot ./create-tar.sh embear-image.tar.zst rootfs overlay_base overlay_swupdate

Now we can use the script create-swu.sh to create a swu file:

./create-swu.sh embear-image 2.0.0

This will create a file swupdate/embear-image_2.0.0.swu which we can install using SWUpdate. If we want to create a new package, we simply add a new overlay to create-tar.sh and repeat the create-swu.sh step with a different version number. To install the image, we can use SWupdate's built-in web server and open a browser with the URL http://<board ip>:8080.

SWUpdate Qt integration

The image build previously includes the sample Qt integration from SWUpdate. The sample integration uses the SWUpdate API via the client library. This means that swupdate runs as a daemon and the application communicates with it via a unix socket. However, the library wraps the protocol so we don't have to worry about it.

Code

The code is for demonstration purposes only and doesn't do any proper error handling. However, this chapter will try to explain what happens.

C++

SwUpdate::update()

bool SwUpdate::update(const QString file)
{
    updateFuture = QtConcurrent::run([file]() -> bool {
        struct swupdate_request req;
        int rc;

        /* This Posix signal needs to be ingored else we would stop the application */
        signal(SIGPIPE, [](int /*sig*/) {
            qDebug() << "Swupdate broke update pipe";
        });

        QUrl fileUrl(file);
        fd = open(fileUrl.path().toStdString().c_str(), O_RDONLY);
        if (fd < 0) {
            return false;
        }

        swupdate_prepare_req(&req);

        rc = swupdate_async_start(readimage, NULL, end, &req, sizeof(req));
        if (rc) {
            return false;
        }
        return true;
    });

    if (updateFuture.isFinished() && !updateFuture.result())
        return false;

    updateWatcher.setFuture(updateFuture);

    QObject::connect(&updateWatcher, &QFutureWatcher<bool>::finished, this, &SwUpdate::updateDone);

    return true;

}

The update method of SwUpdate is called when we want to perform an update. SWupdate is started with swupdate_async_start. The SWUpdate client library will call readimage to read a new set of bytes from the file. We have to ignore the SIGPIPE signal, otherwise the Qt event loop would stop running. We start the update in a separate process. This process will inform the Qt Event Loop via Future when swupdate_async_start returns.

SwUpdate::checkProgress()

void SwUpdate::checkProgress()
{
    progressFuture = QtConcurrent::run([this]() -> QString {
        struct progress_msg msg;

        progressPid = gettid();
        signal(SIGUSR1, [](int /*sig*/) {
            // Closing the filehandler will make read return
            close(progressFd);
        });
        while (progressFd <= 0 && !this->stop) {
            progressFd = progress_ipc_connect(false);
            QThread::msleep(500);
        }

        if (progressFd <= 0)
            return QString();

        // This is blocking, use Posix signals to interrupt
        if (progress_ipc_receive(&progressFd, &msg) <= 0)
            return QString();

        // Convert status to JSON message for QML (could also be done by using Qt JSON)
        switch (msg.status) {
        case IDLE:
            return QString("{\"status\": \"idle\"}");
        case RUN:
            return QString("{\"status\": \"run\", \"progress\": ") + QString::number(msg.cur_percent) + QString("}");
        case START:
            return QString("{\"status\": \"start\", \"progress\": ") + QString::number(msg.cur_percent) + QString("}");
        case SUCCESS:
            return QString("{\"status\": \"success\"}");
        case FAILURE:
            return QString("{\"status\": \"failure\", \"info\": ") + QString(msg.info) + QString("}");
        case DOWNLOAD:
            return QString("{\"status\": \"download\", \"progress\": ") + QString::number(msg.dwl_percent) + QString("}");
        case DONE:
            return QString("{\"status\": \"done\"}");
        case SUBPROCESS:
            return QString("{\"status\": \"subprocess\", \"info\": ") + QString(msg.info) + QString("}");
        case PROGRESS:
            return QString("{\"status\": \"progress\", \"progress\": ") + QString::number(msg.cur_percent) + QString("}");
        }

        return QString();
    });

    this->progressWatcher.setFuture(this->progressFuture);
}

The checkProgress method of SwUpdate is used to receive the asynchronous messages from SwUpdate. These messages are sent whenever an update is started, even if it is started from the command line or web server. To allow the process to be stopped when we close the application, we use Posix signals. This is necessary because a read used by progress_ipc_receive can't be interrupted by the main process. So we send a SIGUSR1 to the process and close the file handler in this callback. This will make the read return immediately.

QML

In QML we just show the status of what checkProgress returns and change the string and progress bar based on that:

main.qml

SwUpdate {
    id: softwareUpdate
    onProgress: {
        console.log(progress);
        var _progress = JSON.parse(progress);

        if (_progress.status === "start")
            state.text = "Current State: Update Started (loading file)";
        else if (_progress.status === "progress") {
            state.text = "Current State: Update Running";
            progressBar.value = _progress.progress;
        }
        else if (_progress.status === "done")
            state.text = "Current State: Update Done";
        else if (_progress.status === "failure")
            state.text = "Current State: Update Failed (" + progress + ")";
    }
}

The software update is started if we have selected and accepted a file in the file dialog:

main.qml

MyFileDialog {
    x: (parent.width/2)-300
    y: parent.y + 300;
    id: fileDialog
    title: "Please choose an update file"
    currentFolder: "file:///media"
    font.pointSize: 15
    onAccepted: {
        console.log("You chose: " + fileDialog.selectedFile)
        softwareUpdate.update(fileDialog.selectedFile)
    }
    onRejected: {
        console.log("Canceled")
    }
}

There is a copy of FileDialog and FolderDialog in the repository. This is due to some issues with EGLFS. It seems EGLFS has problems if clipping is enabled. This was the case in FolderBreadcrumBar. Therefore, we create a copy of several standard QML files with disabled clipping.

Demo

A demo on how to use SWUpdate and how the qt integration looks was shown in a Qt Developer Conference talk:

Conclusion

SWUpdate simplifies the update process by handling tasks such as binary verification, configuration parsing, and script execution, allowing us to focus on the essential aspects of the update. While other solutions like RAUC share this feature, they may offer additional functionalities. A notable advantage of SWUpdate is its integrated web interface, which facilitates direct updates.