Fedora Modularization: OSTree Prototype

Using rpm-ostree to deliver a “regular” system

The first idea, which I think had been banging around in several people’s heads for a while, actually came up more formally in an Environments & Stacks meeting on Apr 16. The idea, in essence, is “can we use rpm-ostree (an “implementation” of OSTree) to layer components on to an installation of Fedora.”

In some ways, the use case of adding desktop components in layers is really the original use case for OSTree. From talking to Colin a long time ago, he had originally started and used OSTree to allow him to work on Gnome, basically to use it as a way write some code, test it, then roll back to the stable version to write some more code. What we want here is similar but, really, a way to run a “production install” with the layering/rollback ability of OSTree for sets of components.

The Prototypes

Prototype-1: Normal Workstation & Server

  1. Use rpm-ostree-toolbox to create a compose tree from the official Fedora 21 Workstation kickstart file. It is likely that Brent Baude’s articles1 2 on RHT Developer Blog will be helpful to this work.
  2. Create a VM and install the ostree-based Fedora Workstation
  3. Ensure that normal operation performs correctly, aside from installing new things
  4. Repeat steps for Fedora Server

Prototype-2: Normal Workstation & Server update

  1. Using the VM and compose tree from the “Normal Workstation” use case, introduce an updated rpm in to the tree
  2. Ensure that “rpm-ostree upgrade” (or “atomic host upgrade“) successfully update the system with the new rpm.
  3. Ensure that the rpm can be reverted by executing a “rpm-ostree rollback” (or “atomic host rollback”)
  4. Repeat steps for Fedora Server

Prototype-3: Investigate location of user files

In order for this to “feel” like a normal user system, a user must have the freedom, with some constraints, to add “content” to the places they expect to on the system as well as have the applications they use recognize those locations as “where things go.” For example, I often symlink my “Downloads” directory (in Gnome) to my mounted “projects” directory so that it can grow with my “projects allocation” (and be reused across installs) rather than with my “home dir allocation.” However, if you do that, you need to ensure that Firefox default downloads directory follows the symlink when downloading, the Files app keeps “Downloads” in the “pick list,” etc. As a result, if we move home-dirs to somewhere else, we need to ensure the user experience is the same, or has easily documented differences. I would expect we want to have a similar experience for /opt.

Prototype-4: Investigate using dnf to switch compose-trees

  1. Create a plugin for dnf that front-ends “rpm-ostree rebase”
  2. Create an alternate compose-tree with a significant component change. For example, tuned or a different version of Gnome
  3. Attempt to rebase to the new compose tree using dnf
  4. Attempt to rebase back to the old compose tree using dnf

Prototype-5: Investigate using dnf to create a new compose tree

In order to execute on this prototype in a reasonable way, we will need to declare a couple of tenets which, arguably, invalidate the test, but are still a good prototype while we devise a prototype that will test the tenets.

First off, we are just going to be writing the new compose-tree to disk with some mechanism to verify its quality. In a later protoype we can worry about moving the compose-tree to “someplace” which could host a rebase to that tree.

Second, the ability of the existing compose-tree to meet the dependency graph of the new rpm may prove problematic. While the compose-tree installed on the local system should have an rpm database that can be used for the dependency walk, the rpm coming from an external repository may have new dependencies, or, perhaps more likely, new versions of existing dependencies. For this prototype, it is recommended that we just carefully select the rpms to avoid this problem.

  1. Write a dnf plugin to front-end “rpm-ostree-toolbox.” However, the input should be the existing compose-tree from the user’s box and an rpm from a normal repo
  2. The plugin should generate a new compose-tree including the existing components, the rpm selected, and dependencies walking the new rpm’s dependency tree

Prototype-6: Use dnf to “host” compose-trees

  1. Leveraging a dnf plugin, likely the same one as from Prototype-5, create and manage a location on disk to manage ostrees.
  2. Using the dnf plugin and the compose-tree from Prototype-5, setup the compose-tree to be a target for rebasing of the local system
  3. Use the new compose-tree to rebase
  4. Rebase back to the old compose-tree

Prototype-7: Update existing compose-tree and add new rpm

The need for this prototype is to address the second tenet in Prototype-5. We may discover in the work to do Prototype-5 that the composition of rpms in to the new compose-tree just as easily uses the upstream repository directly as using the locally installed tree. If so, then this prototype is unnecessary or “marked complete” based on those results.

  1. Leveraging the work from Prototypes 5 & 6, identify an rpm that has changed or updated dependencies in the upstream repository
  2. As part of the update to the compose, layer in the changed rpm dependencies and the new rpm and its dependencies
  3. Host the compose-tree per Prototype-6
  4. Rebase to the new compose-tree
  5. Rebase back to the old compose-tree

Conclusion

In order to make this more workable, I have created a github repo with the prototypes identified above as sub-directories. Each sub-directory will contain a markdown file of the description of the prototype as well as Behave features and steps to test the efficacy of the prototype. If you would please file issues there for comments or changes to the prototypes I think this will be a better “living document.” Also, depending on when you read this, there might be lots more prototypes and/or results!

4 thoughts on “Fedora Modularization: OSTree Prototype

  1. Steps 4-7 seem to be experimenting with the question “how do a create a locally modified version of the system.” While this could be useful for operating system development, is it really a practical answer to the needs of the user to install new content?

    The idea of xdg-app (see https://lists.fedoraproject.org/pipermail/desktop/2015-May/012362.html) is that we can layer packaged applications on top of the system in a way that is isolated from the base system both in a security sense and in terms of dependencies.

    For developers we need something a bit different: the traditional Fedora model is that you install -devel packages and modules as needed until the thing you are compiling or developing works. Usually this is a scattershot process as you find more things you need, and having to reboot to a different ostree repeatedly is pretty much a non-starter. This implies that instead the developer should be installing things in an isolated environment. A raft of questions then arise: is this isolated environment something that is existing and well established? in addition to Docker there are language-specific solutions like virtualenv. does the isolated environment instead look more like a standard Fedora environment where you can freely ‘dnf install’ new packages? does the isolated environment inherit anything from the running system, or does it start from scratch? And so forth.

    • I’ve been working more on the xdg-app fedora runtime, in particular on how to create apps from the existing packages. Hopefully I can post something more detailed about this in a few days.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s