To Infinity and Beyond – Using Qt to Display Gigapixel Images

Hubble Space Telescope project offers many fascinating images like the one above. Some of these come in large resolutions like this one above GOODS South field, sporting a resolution of 31813 x 19425 pixels and a size of 928 MB.
While the image itself would be worth talking about in great length, in this post I would like to show a way how to handle images like these in a Qt / QtQuick application. This shall serve as an example how to deal with large image data as provided by satellites, microscopes and alike.

Example of a zoomable tiled image. For demonstration purposes reloading of the individual tiles is highlighted with an opacity animation making them flash as they are reloaded.

Since the overall image is (or could be) too large to be loaded and kept in memory at full resolution it is split into tiles of a more manageable size. These tiles can be quickly loaded at the desired resolution and deleted, or kept in smaller resolution, when not needed. The following sample application is meant to demonstrate how to organize these tiles asynchronously while the user is zooming and panning.
At the heart of the sample application lies an implementation of QAbstractTableModel that provides the image data as individual tiles arranged in rows and columns. The models contents get displayed in the overhauled TableView that recently has been (re-) added to QtQuick.

I used ImageMagick to split the large image into small tiles of equal size where each tile’s row and column position is part of the filename. For the image above this results in a folder with more than 2.000 individual images.

convert heic0602a.jpg -crop 512X512 -set filename:tile "%[fx:page.y/512]_%[fx:page.x/512]" tiles/tile_%[filename:tile].jpg

The table model used to manage these tiles reads the tiles from the directory and creates a ImageTile for each entry. The ImageTile class looks like this:

class ImageTile : public QObject
{
    Q_OBJECT
    Q_PROPERTY(int row READ getRow CONSTANT)
    Q_PROPERTY(int column READ getColumn CONSTANT)
    Q_PROPERTY(QString fileName READ getFileName CONSTANT)

public:
    explicit ImageTile(int row, int column, QString fileName, QObject *parent = nullptr);
    Q_INVOKABLE int getRow() { return row; }
    Q_INVOKABLE int getColumn() { return column; }
    Q_INVOKABLE QString getFileName() { return fileName; }
    QImage getImage() { return image; }
    void setImage(QImage image);
    void clearImage();

signals:
    void imageChanged();

private:
    QString fileName;
    QImage image;
    QRect window;
    int row{0};
    int column{0};
};

The table model needs to be aware which images need to be loaded in which resolution. Therefore it needs to know what part of the image is displayed at the moment and at which resolution. To achieve this the QtQuick frontend informs the model about the current geometry and scale whenever the visible window of the image has changed.

void ImageTilesModel::setGeometry(QRect visibleWindow, qreal scale)
{
    qDebug() << Q_FUNC_INFO << visibleWindow << scale;
    this->visibleWindow = visibleWindow;
    this->scale = scale;
    updateTiles();
}

This starts the update process in which each tile is checked whether or not it is inside the visible window. For each tile within a worker thread is started that will load the tile in the background. The worker threads are organized in a QThreadPool that takes care or running them one after another.

void ImageTilesModel::updateTiles()
{
    qDebug() << Q_FUNC_INFO;
    workerPool.clear();
    QSize scaledTileSize = scale * tileSize;
    for(auto&&row: tiles) {
        for(auto&&tile: row) {
            QRect tileRect(tile->getColumn() * tileSize.width(),
                           tile->getRow() * tileSize.height(),
                           tileSize.width(),
                           tileSize.height()
                           );
            if(tileRect.intersects(visibleWindow)) {
                if(tile->getImage().size().width() < scaledTileSize.width()) {
                    ImageTileWorker* worker = new ImageTileWorker(tile, scaledTileSize);
                    worker->setAutoDelete(true);
                    workerPool.start(worker);
                }
            }
        }
    }
}

The worker class itself is small since it only has one job, which is to load the image and scale it to the desired size.

class ImageTileWorker : public QObject, public QRunnable
{
    Q_OBJECT
public:
    explicit ImageTileWorker(ImageTile* tile, QSize requestedSize, QObject *parent = nullptr);
    void run() override;

signals:

private:
    QSize requestedSize;
    ImageTile* tile = nullptr;
};

void ImageTileWorker::run() {
    qDebug() << Q_FUNC_INFO;
    QImage image(tile->getFileName());
    image = image.scaled(requestedSize.width(), requestedSize.height());
    tile->setImage(image);
}

The tiles informs the table model, that its content has changed. The table model then informs the TableView that a specific tile has changed using the dataChanged signal. Note that the TableView’s contentHeight and contentWidth properties are set manually instead of letting the TableView figure it out on its own as recommended here.
Also note that the individuals cells width and height have to be set through their implicit properties. Setting them explicitly will be ignored as documented here.
The opacity animation in the delegate is used for debugging and demonstration only in order to visualize when a tile has changed.

TableView {
            id: table
            property int tileHeight: tilesModel.tileSize.height
            property int tileWidth: tilesModel.tileSize.width
            height: rows * tileHeight
            width: columns * tileWidth
            contentHeight: rows * tileHeight
            contentWidth: columns * tileWidth
            model: tilesModel
            interactive: false
            delegate: Item {
                //explicit width and height are ignored, see:
                //https://doc.qt.io/qt-5/qml-qtquick-tableview.html#delegate-prop
                implicitHeight: table.tileHeight
                implicitWidth: table.tileWidth
                Image {
                    id: img
                    anchors.fill: parent
                    source: "image://tileProvider/" + tile.row + "," + tile.column + "," + Math.random()
                    onSourceChanged: PropertyAnimation { target: img; property: "opacity"; from: 0; to: 1}
                }
            }
        }

This triggers an update in the TableView and loads the new image from a custom QQuickImageProvider:

QImage ImageTileProvider::requestImage(const QString &id, QSize *size, const QSize &requestedSize)
{
    qDebug() << Q_FUNC_INFO << id << size << requestedSize;
    QImage image;
    QStringList positions = id.split(",");
    if(positions.size() < 2)
    {
        qCritical() << "bad image id" << id;
    }
    else
    {
        int row = positions.at(0).toInt();
        int column = positions.at(1).toInt();
        image = model->getImage(row, column);
    }
    size->setWidth(image.width());
    size->setHeight(image.height());
    return image;
}

That’s it for now. Above code is meant as an example and source of inspiration. It should not be mistaken for production ready, as there is little error handling. Feel free to contact me with any questions or ideas for applications. The complete code needs some revision before it will be on my GitHub account.