Identifying Operating Systems in GNOME Boxes

One secret sauce of GNOME Boxes is libosinfo. It basically is an umbrella for three components: libosinfo, osinfo-db-tools, and osinfo-db.

libosinfo offers programmatic means to query for information about OSes. osinfo-db-tools is a set of tools that help manipulate and extract information from OS images (such as ISO files). osinfo-db is a database of operating system information describing requirements for virtualized installations as well as virtual drivers and devices that work with each OS in the database.

For a given image file, Boxes will use libosinfo to detect which operating system it contains and assign the appropriate configuration preferences for it.

When an operating system’s data was not present in osinfo-db, we used to treat the image file as an “unknown OS”, which would get set up with the Boxes default configuration preferences. This usually works, but when it fails, it gets really hard for a regular user to use the virtual machine. Think of a wrong virtual video driver causing the guest’s graphic session to hang, or a virtual tablet device getting on the way of gestures being passed to the guest, etc…

For this very reason, we decided to add an extra page in our VM creation assistant that will only get visible for those “unknown” OSes. This page allows you to specify which operating system you intend to install. This way Boxes can use libosinfo to set up your VM with the right configuration preferences for the chosen OS.

screenshot

If you can’t find your OS in the list, try picking a similar one. For instance, if Linux Mint isn’t on the list, you are probably better off telling Boxes it is a Ubuntu or Debian based OS.

Not choosing anything on this page is also fine, in this case, Boxes will go with its defaults.

New operating systems can be added to the osinfo-db by creating merge requests in their GitLab repository.

This (with some more polishing) and other changes will be available in GNOME Boxes 3.38. If you can’t wait, give a go to our *unstable* nightly Flatpak.

Try the GNOME Nightly VM images with GNOME Boxes

It was a long time overdue but we now have bootable VM images for GNOME again. These VMs are good for testing and documenting new features before they reach distros.

To provide the best experience in terms of performance and host-guest integration, we landed in BoxesDevel (Nightly GNOME Boxes) an option to create GNOME VMs with the correct device drivers and configurations assigned to it. You know…the Boxes way.

Installing GNOME Boxes (Nightly)

1. Set up our nightlies Flatpak repository:

flatpak remote-add --if-not-exists gnome-nightly https://nightly.gnome.org/gnome-nightly.flatpakrepo

2. Install Boxes

flatpak install gnome-nightly org.gnome.BoxesDevel

Testing the GNOME VM image

1. Download a recent VM snapshot (linked on the unstable release announcements). It is a qcow2 file.

2. Open the new VM dialog in Boxes and click on the “GNOME Nightly” entry in the Featured Downloads section. It will open a file chooser.

Screenshot

3. After selecting the qcow2 file downloaded in step one, you can continue to Create a VM. Once the creation is over, you will be able to start the VM by clicking in it on the icon view.

Future developments

We haven’t reached a consensus yet on how we are going to distribute/store/host these VM images, that’s why we have the extra-step before, requiring to pick the file in a file chooser.

In the near future, we will host the images and you will be able to download them directly from GNOME Boxes.

Also, the latest image as of today (3.35.91) doesn’t come with spice-vdagent. It should be included in the next builds, allowing for a maximum host-guest integration like dragging and dropping files from host to guest, automatic resolution, etc…

This is just the beginning. Stay tuned!

Trip report: Flock to Fedora 2019 + Fedora Flatpaks

This was my first time at Flock to Fedora, and it was a blast! The conference took place from August 8th to August 11th in the astonishing city of Budapest.

It is very convenient to host the conference at the same place where people are accommodated. The whole infrastructure and conference organization was top-notch. Nice social events and great comfort during the talks/workshops.

At the very beginning, it was pleasant to watch Matthew Miller’s “The State of Fedora”, especially the emphasis on Silverblue being “the future of Fedora Workstation”, and the overview of all the other teams building fantastic things on top of Fedora. The “Facebook Loves Fedora” talk was definitely the one we talked the most about during the breaks. Long story short, Facebook’s IT is supporting Fedora Workstations for its employees and they have a quite appealing story of their adoption. All recorded Flock talks are planned to be published in the Fedora Project YouTube channel, so I encourage you to watch specifically this quick one (25 minutes) once it is out.

Debarshi Ray’s “Toolbox” talk was well received by the audience, and the post-talk corridor convo was productive. People seemed curious and optimistic about the solutions we have for “making their workflow-breakage less painful”. 🙂 Unfortunately Rishi’s talk was scheduled at the same time slot as Christian Schaller’s “Fedora Workstation update and roadmap”. It is great having talks recorded for this very reason.

“Fedora IoT” by Peter Robinson was a nice surprise. Peter brought an Exxon Mobil representative to talk about their use and challenges while using Fedora technologies in IoT devices. These folks have a very interesting set of problems to solve, and I would love FOSS to be the go-to option in this market (any market, really!). I am personally interested in home/domestic automation with open hardware tech, and I can see how the “Fedora IoT” efforts can have a beneficial impact on the enterprise but also in STEM education.

To start the second day, Denise Dumas presented a very reassuring keynote talk on “Red Hat + IBM” and how that impacts the Fedora project. Once again,  it was very satisfying to hear “Fedora is RHEL’s upstream” being emphasized. The Red Hat commitment to upstream communities is something we hear a lot internally, but I feel we rarely express that to the outside world (to the point that a significant amount of people eventually question that).

My colleagues Jiri Eischmann and Tomas Popela had a talk on Silverblue. It gathered an interested audience that engaged in Q&A with us afterwards. Some of the questions were positive feedback that we should take, and some others were useful questions that enabled us to clarify some common misunderstandings and lack of knowledge about Silverblue, ostree, containers, Flatpak, and all things. 🙂

At the end of the day I presented a “Fedora Flatpaks” talk. You can watch its recording below.

After the talk, I was approached by a couple of packagers interested in converting their RPMed apps into Flatpaks. Win-Win!

The river cruise and dinner in the Danube is now history! Check out all the pics https://photos.app.goo.gl/rz1VZqdw6MKeasuc8

I am looking forward to seeing you all again in DevConf.CZ and/or Flock to Fedora 2020.

DevConf.CZ 2019

Last month I attended DevConf CZ for the third time. The conference has been growing a lot in the last years and it has been attracting a wider variety of people. It is a free-admission conference in the lovely Brno, Czech Republic, the place that I now call home. If you haven’t attended it yet, you should definitely consider it for next year.

This year I had a talk titled “Running virtual machines in the Flatpak sandbox”, where I described the process of Flatpaking GNOME Boxes. There’s a video available on YouTube.

We had a small Desktop track on Friday afternoon with a good amount of talks about Flatpak. The audience was engaged and interested in the topics. Besides of my talk, Owen Taylor spoke about the state of Flatpaks in Fedora and Jiří Janoušek spoke about the Linux fragmentation and how Flatpak can tackle the problem. Last but not least, Kalev Lamber told us what’s new in gnome-software.

Contributing to Boxes

I have to admit that Boxes is a bit late for the Flatpak party, but that’s not a problem. The technical difficulties of getting a virtualization hypervisor to run inside the flatpak sandbox are mostly overcomed. This way, contributing to Boxes has never been easier.

In the following sections I will describe the step-by-step process of making your first code contribution to GNOME Boxes.

Get GNOME Builder

Builder makes it very easy to download and build GNOME applications with just a couple of clicks. It will also make your life easier while writing the code.

Download Builder

Download and build Boxes

GNOME Builder: cloning a project and building it

That’s it! Now that you have the project built and can run it, we can start looking into fixing bugs.

Finding an issue to hack

You can have an overview of the ongoing work in the project by browsing our kanban board. We also have issues tagged as Newcomers if you are making your first contribution and want to start hacking on something easy.

Create a GitLab account and fork the project

Visit gitlab.gnome.org and create an account. GitLab will pop up a banner asking you to add your SSH keys to your profile, or you can go directly to edit your profile.

After your profile has been properly setup, it is time to fork the project!

Go to the Boxes project page and click the Fork button. This will create your own copy of the git repository under your personal namespace in GitLab.

Finally, get your fork URL and add to your local git repository as a remote:

git remote add fork $project_url

Making changes and submitting your code

After building Boxes and finding an issue to work, it is time to dive into the codebase. Edit the files and press the GNOME Builder “play” button to see your changes take effect.

Since the migration to GitLab, we have adopted the merge request workflow.

You need to:

1. Create a git branch and commit your changes

git checkout -b $descriptive-branch-name

Do your work, and commit your changes. Take a look at our commit message guidelines.

2. Push your changes for the world to see!

git push fork

A message with a link to create a merge request will be printed in your terminal. Click it, describe your changes, and Submit!

3. Follow up on the feedback

Me and other developers will review your work and recommend changes if necessary. We will iterate over and over until your contributions are ready to be merged.

4. Celebrate your first contribution!

Further reading

The steps described above are based on the GNOME Newcomers initiative. We have a detailed step-by-step process of making contributions and you should definitely check it out. It has pointers about documentation, tips about finding the right approach to dive into the code base, and examples.

Let’s do it!

Boxes + Flatpak

TL;DR: You can now install GNOME Boxes directly from Flathub! 🎉

INSTALL

Why are we so excited about this?

It might seem at first sight that Boxes is a simple application, and that is partially true if you ignore the deep stack under the hood responsible for making virtualization simple™. The various modules (some of them gigantic such as qemu, libvirt, freerdp…) need to be setup in perfect harmony for us to boot a whole operating system with its essential functionalities.

The GNU/Linux distribution model has historically delegated to downstream packagers the responsibility of integrating dependencies in order to provide an application to their end users. This model has worked for some for many decades, but it has fundamental flaws that “trickle down” making the upstream developers’ life miserable.

Don’t get me wrong, bugs are mostly our fault, but a significant amount of bug reports I receive consist of issues I cannot reproduce in my development environment. A combinatorial explosion of package versions, build flags, and/or pivotal architecture differences between distros.

Therefore this is the first and foremost benefit we get from shipping Boxes as a Flatpak.

Another difficulty we face during our development cycle is having the ability of having designers, translators, and marketing folk being able to run our latest snapshot or a specific work-in-progress tree. With the GitLab continuous integration combined with Flatpak we can spin bundles at any moment, and they can be installed within a couple of clicks, alongside other versions of the same app! This is The Future!

Having our apps widely available is another concern we have. Many distributions which stick to the package model also support Flatpak. Besides that, there are new players which are essentially different. Container-based desktop operating systems are a thing now too.

Software is never done…

There are indeed downsides of running Boxes in a Flatpak in comparison with a well crafted build of dependencies bottom-up. But these are issues that can be solved and are going to be prioritized in our TODO list.

Running Boxes in a Flatpak TODAY you won’t have yet:

  • A bridged network between host and guests.
  • The ability of installing an image from a physical device.
  • Access to your system’s libvirtd (we run our own libvirtd in the sandbox).
  • Shared folders.
  • USB redirection.
  • $YOUR_ISSUE_HERE

The list above is far from complete, and I would like to count on you to experiment with the Flatpak and report issues you might encounter. Use the label “flatpak” to collect points which you can later exchange for beers or other beverages of your choice. 😉