rpm-ostree and bootc#

rpm-ostree and bootc are complementary system management technologies that form the foundation of image-based Linux operating systems like Bluefin. rpm-ostree provides the storage layer with an immutable base system using OSTree commits, while bootc serves as the primary upgrade mechanism for atomic updates and rollbacks using OCI/Docker container images. Together, these tools enable transactional system updates that eliminate the package-by-package mutation model of traditional Linux distributions.

bootc implements transactional, in-place OS updates using the standard OCI container image format, applying the successful container layering model to bootable host systems. It depends on ostree as a storage backend but replaces ostree's HTTP transport with OCI container fetching, enabling systems to consume updates from standard container registries. This architecture represents a fundamental paradigm shift from mutable package-based distributions to immutable image-based delivery, providing stronger consistency guarantees and simplified rollback capabilities.

In Bluefin specifically, the system underwent drastic refactoring in 2025 to fully embrace bootc as the primary update path, with rpm-ostree serving as a fallback mechanism only when local package layering is enabled. This design ensures the system image remains pristine with no package-based degradation over time, making upgrades more predictable and reliable.

Architecture and Relationship#

rpm-ostree and bootc work in a complementary manner within image-based operating systems. rpm-ostree provides the underlying immutable filesystem layer with content-addressed storage, while bootc handles delivery and update orchestration using modern container standards. The OS is immutable by default using composefs with /usr read-only, /etc for configuration, and /var for data.

A key architectural distinction: bootc upgrade does not support layered packages, so rpm-ostree upgrade is used only when local package layering is enabled (LockLayering=false in /etc/rpm-ostreed.conf). This design decision reinforces the immutable system philosophy by discouraging client-side mutations.

OSTree Foundation#

OSTree provides three key components: a git-like repository for OS data with full rootfs checkout, a bootloader integration layer, and a transport layer for pulling content. bootc leverages the first two components but replaces the OSTree HTTP transport with OCI container fetching via skopeo, enabling integration with standard container registries and authentication mechanisms.

Core Concepts#

Immutability Model#

Local package layering is locked by default via LockLayering=true in /etc/rpm-ostreed.conf, which prevents mutation of the base OSTree commit including package overlays, overrides, and initramfs changes. This locked configuration ensures the system operates in a truly immutable mode where all changes come from upstream container images rather than client-side modifications.

If the system becomes modified through package layering, running rpm-ostree reset and rebooting always restores the system to pure image mode. This reset capability provides a reliable escape hatch from configuration drift. Composefs enables the read-only root filesystem that enforces immutability at the kernel level.

Filesystem Layout#

The deployed system operates with a clear separation between immutable system content and mutable state:

• Physical root: The actual host root filesystem is mounted at /sysroot, with the deployed system running in an equivalent of a chroot
• /usr: Contains all immutable OS content and is read-only when composefs is enabled
• /etc: Holds mutable persistent configuration state with OSTree 3-way merge semantics that reconcile image defaults with local modifications across updates
• /var: Contains persistent application data that survives across all deployments, with exactly one /var shared across bootloader entries
• /opt and others: Read-only by default with composefs, requiring explicit configuration for mutability

Image-Based Updates#

Unlike traditional package managers that modify systems piece by piece, system updates are delivered as complete new images, not individual packages. Updates are checked automatically every 6 hours and applied on reboot, keeping the running system stable while staging changes for the next boot.

The A/B style upgrade system provides safety by maintaining both current and previous deployments. Staged updates allow downloading and preparing updates without affecting the running system, enabling validation before commit and instant rollback if issues arise.

bootc Commands and Operations#

bootc upgrade#

bootc upgrade queries the container image source and queues an updated image for application on the next boot. Key flags modify this behavior:

# Standard upgrade (stages update, requires reboot)
bootc upgrade

# Upgrade and immediately reboot to apply
bootc upgrade --apply

# Download-only mode (prepare without auto-applying)
bootc upgrade --download-only

# Apply previously downloaded update
bootc upgrade --from-downloaded

# Check for updates without side effects
bootc upgrade --check

bootc switch#

bootc switch changes the tracked container image while preserving /etc and /var state, enabling blue/green deployments and variant rebasing. Both system-wide Flatpaks and user Flatpak data/configs are preserved during rebase.

# Switch to a different variant
sudo bootc switch ghcr.io/ublue-os/bluefin-dx:latest

# Switch to a specific date tag
sudo bootc switch ghcr.io/ublue-os/bluefin:stable-43.20260228

bootc status#

Displays the current system state including the booted deployment, staged deployments (if any), container image reference and version, and download-only status in verbose mode.

bootc status

bootc rollback#

bootc rollback swaps the bootloader ordering to the previous boot entry, providing quick recovery from problematic updates. The previous deployment is retained automatically for this purpose.

bootc rollback

bootc install#

bootc install bridges OCI container images and bootable systems and must be run from within the container being installed. It supports three modes:

• to-disk: Direct installation to a block device
• to-filesystem: Installation to an existing filesystem (for external installers)
• to-existing-root: Convert an existing Linux system to bootc

Options include filesystem type selection, LUKS encryption bound to TPM2, kernel arguments, and SSH key injection.

rpm-ostree Commands#

rpm-ostree status#

Lists current deployments, shows pinned deployments, and displays layered packages if any. Use -v for verbose output.

rpm-ostree status
rpm-ostree status -v

rpm-ostree reset#

Removes package layering and restores the system to pure image mode. Requires a reboot to take effect.

rpm-ostree reset
sudo reboot

Deployment Pinning#

ostree admin pin pins a specific deployment to prevent automatic garbage collection. Pinned deployments are important for preventing the /boot partition from filling, but should be managed carefully.

# Pin current deployment (index 0)
sudo ostree admin pin 0

# List deployments to find index
rpm-ostree status -v

# Unpin a deployment by index
sudo ostree admin pin --unpin <index>

Bluefin Implementation#

Update Mechanism#

Bluefin underwent drastic refactoring in 2025 to fully embrace bootc, establishing bootc as the primary update mechanism with rpm-ostree as a fallback. ujust update uses bootc upgrade by default, falling back to rpm-ostree upgrade if package layering is detected.

The uupd service coordinates system and Flatpak updates, replacing or supplementing rpm-ostreed-automatic and Flatpak timers. This unified service ensures consistent update orchestration across the system.

Update Streams#

Bluefin provides multiple update streams allowing users to select their preferred cadence:

Users can switch between streams and pin to specific dates or versions using container image tags.

Three-Layer Package Management#

Bluefin implements a three-layer package management strategy:

1. System: Immutable base OS delivered as complete images via bootc, updated automatically every 6 hours and applied on reboot
2. Flatpak: GUI applications updated daily at 4:00 AM via systemd timers, no reboots required
3. Homebrew: CLI tools managed independently within user space

This layered approach separates system stability from application flexibility, allowing users to install and update applications without affecting the base system image.

Bluefin Variants#

All Bluefin variants use bootc as the primary update mechanism:

• Bluefin: Standard desktop variant based on Fedora Silverblue
• Bluefin DX: Developer Experience variant with approximately 67 additional packages for development, including Docker, Podman, Incus, LXC, QEMU, and libvirt
• Bluefin LTS: Based on CentOS Stream 10 rather than Fedora Silverblue, built natively on bootc from the ground up for long-term stability

Usage Examples#

Manual System Update#

ujust update

This command upgrades the system using bootc upgrade by default, falling back to rpm-ostree upgrade if package layering is detected, and updates all Flatpaks.

Toggle Automatic Updates#

ujust toggle-updates enable
ujust toggle-updates disable

Rebase to Different Variant#

# Interactive rebase helper
ujust rebase-helper

# Manual rebase to specific variant
sudo bootc switch ghcr.io/ublue-os/bluefin-dx:latest

# Rebase to NVIDIA variant
sudo bootc switch ghcr.io/ublue-os/bluefin-nvidia-open:stable

Pin Current Deployment#

sudo ostree admin pin 0

Unpin Deployment#

# List deployments with verbose output
rpm-ostree status -v

# Unpin by index number
sudo ostree admin pin --unpin <index>

Reset to Pure Image Mode#

rpm-ostree reset
sudo reboot

Monitor Update Logs#

# Follow update service logs
journalctl -fexu uupd.service

# Check update timer status
systemctl status uupd.timer

# List all update-related timers
systemctl list-timers | grep update

Customize Update Schedule#

# Edit the uupd timer
sudo systemctl edit uupd.timer

# Example: Change to run every 2 hours
# [Timer]
# OnCalendar=
# OnCalendar=hourly

# Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart uupd.timer

Troubleshooting#

Common Issues#

Boot failures can result from several causes:

1. Disk space exhaustion: If /boot partition fills from multiple pinned deployments, unpin unnecessary deployments or boot from live USB to free space
2. Package layering preventing bootc upgrade: System falls back to rpm-ostree; check /etc/rpm-ostreed.conf for LockLayering setting
3. SELinux context problems: Try booting in permissive mode or relabel with sudo restorecon -Rv /sysroot/ostree
4. /etc becoming read-only: Unmerge systemd-confext or overlay extensions

Solutions#

For package layering issues:

rpm-ostree reset
sudo reboot

To free /boot space:

# List all deployments
rpm-ostree status -v

# Unpin old deployments
sudo ostree admin pin --unpin <index>

Check LockLayering status:

cat /etc/rpm-ostreed.conf | grep LockLayering

Factory reset (WARNING: erases all user data):

ujust powerwash

Best Practices#

1. Prefer bootc for updates when using the default immutable configuration with LockLayering=true
2. Avoid local package layering unless absolutely necessary; use Flatpak for GUI applications and Homebrew for CLI tools instead
3. Use Distrobox or Dev Containers for distribution-specific development requirements rather than layering packages
4. Keep deployments pinned only when necessary to avoid filling /boot partition
5. Use drop-in files for timer customization via systemctl edit to avoid overwriting package-managed units
6. Monitor logs for update services to diagnose issues early: journalctl -fexu uupd.service
7. Prefer system-wide Flatpak installations over --user installations when appropriate for better integration

Technical Comparison: rpm-ostree vs bootc#

Related Topics#

• OSTree: Underlying git-like storage technology for bootable filesystems
• OCI Container Images: Standard image format used by bootc for system delivery
• Flatpak: Application layer that runs atop the immutable system
• composefs: Filesystem technology providing kernel-level immutability
• Distrobox: Containerized development environments for distribution-specific tooling
• ujust: Command runner providing system administration utilities in Bluefin
• Fedora Silverblue: Immutable Fedora desktop that serves as Bluefin's foundation

Relevant Code Files#

Key Documentation Resources#

• bootc Official Documentation - Comprehensive bootc reference
• bootc Relationships - How bootc interacts with OSTree and other tools
• bootc Filesystem - Filesystem layout and immutability model
• bootc Upgrades - Update and rollback mechanisms
• Bluefin Administration - Bluefin-specific system management guide
• Bluefin 2025 Blog Post - Details on Bluefin's bootc transition