News

Overview and timeline

At cloudscale, several popular Linux distributions are available for new servers. Thanks to so-called custom images, even more individual setups are possible: as a customer, you upload a hard disk image, which can be used to fill the root volume of your new virtual servers initially, including any required tools and settings. This is where the security vulnerability was found in OpenStack, the open source project our cloud offer is based on. Uploading specially crafted images in QCOW2 format made it possible to access data in systems that play a part in our import process of custom images.

As an immediate measure, we had temporarily deactivated the import of QCOW2 images. Specifically, this means that all images were treated as "raw". Images that actually were in QCOW2 format could therefore not be used by virtual servers to boot. While we were aware that this causes additional work for some customers, as they need to convert images into raw image format before importing them, we decided to implement this temporary step in the short term in the interest of security for us and our customers.

Timeline:

• December 2020: Introduction of custom images, support for raw format
• May 2022: Additional support for images in QCOW2 format
• February 2023: Specification of image format no longer required for import
• 2024-07-02: OSSA-2024-001 / CVE-2024-32498 security vulnerability disclosed
• 2024-07-03: Import of images in QCOW2 format blocked as an immediate measure
• 2024-07-31: Import of images in QCOW2 format possible again, adaptation of API

Background of the vulnerability

Image files in QCOW2 format have several special features, e.g. the fact that they tend to require less space. In addition, they can not only contain the actual data of a data carrier, but also references to externally located files. The core of the OpenStack security vulnerability was inadequate checks in some OpenStack components that processed references of this kind and their target files when processing QCOW2 images. As custom images are always saved in raw format at cloudscale, the vulnerability affected in particular automatic recognition of the format to be imported and the conversion from QCOW2 to raw. However, according to our analysis, there are no indications that the vulnerability was actually exploited at cloudscale.

The conversion of images where OpenStack uses qemu-img convert has in the meantime been patched so that it is no longer possible to exploit it even using specially crafted images. This means that we can once again enable the import of QCOW2 images here at cloudscale. However, in the OpenStack developer community, there is not (yet) a reliable solution for automatic format recognition using qemu-img info, which is why at cloudscale we will in future no longer attempt automatic image format recognition.

Use of QCOW2 images at cloudscale

Importing QCOW2 images is possible at cloudscale again effective immediately, but the format has to be indicated during imports. When importing via API, the attribute source_format is used for this purpose. After being declared deprecated in February 2023, this has now once again been officially introduced following recent incidents. While the attribute can be left out for images in raw format, source_format is required for QCOW2 images. We are aware that this change in API behavior is not completely non-breaking, but the purpose of the selected solution is to minimize the required adaptations.

The current versions of our modules/plug-ins support the source_format attribute, which means they can be used to import QCOW2 images. You will find answers to all the important questions relating to this topic in the FAQs below.

We are delighted that, after the exclusion of QCOW2, which became necessary at short notice, we can once again enable the import of custom images in this format. Unfortunately, in the process, it was not possible to avoid the reintroduction of an old attribute in the relevant API call. It was important to us not to leave any known vulnerabilities open, while causing as little work as possible in terms of adaptation. We would ask for your understanding where adaptations are nonetheless necessary.

Considerate and pragmatic.
Your cloudscale team

FAQs

What will happen to existing custom images?
Nothing. The change only applies to the import process. Existing images are not affected.

What will change for imports using the web-based control panel (control.cloudscale.ch)?
You now need to specify the format of the image in the "Source Format" drop-down.

My import is showing the error message "Import could not be processed", what do I need to do?
You probably tried to import a file as "QCOW2" that is not a file of this kind. Check the format and the URL.

I only import raw images. Am I affected?
No. However, ensure that the source_format field is either not transmitted or contains "raw". An error will occur for other values.

I use Terraform, what do I need to do?
In order to be able to import QCOW2 images, you need to use the "cloudscale" provider from version v4.4.0 onwards and specify the import_source_format for new images. You can continue to use existing images with no change, the format should not be specified retrospectively.

I use cloudscale_ch.cloud for Ansible, what do I need to do?
Ensure that the correct source_format is transmitted. You do not need to update the Ansible collection.

I use cloudscale-go-sdk in my Go application, what do I need to do?
If you use it to import custom images, you need to ensure that the correct source_format is transmitted. The dependency does not need to be updated.

I use cloudscale-python-sdk in my Python application, what do I need to do?
If you use it to import custom images, you need to ensure that the correct source_format is transmitted. The dependency does not need to be updated.