Accessing the RAM of a QEMU Emulated System from another Process

February 4, 2021

Sometimes it may be of interest to expose the RAM of a hardware platform emulated in QEMU to an outside process, e.g., for monitoring, testing, or co-simulation.

The QEMU monitor allows to inspect the memory but is not necessarily the most practical tool when you need to access it from another process, e.g., a C program or an external simulator, maybe Questasim simulating a device written in SystemVerilog.

In this post I will show how to make the memory of an x86_64 emulated PC system visible to other processes. There are some examples online, I could not find any that would expose the main memory, only parts, like NVDIMMs (Non Volatile DIMMs). The QEMU documentation is very complete, however, it is not always easy to find what you are looking for.

I wrote this post to provide a demonstration on how to make the main memory of a system emulated in QEMU accessible to other processes because I could not find a complete example online.

TL DR : See section TL DR

Introduction

Let’s say we are launching QEMU with the following command :

qemu -M pc -nographic -m 512m -smp cpus=2 -kernel ./bzImage_5.9 -drive file=./rootfs-target.img,if=ide -append "console=ttyS0 root=/dev/sda rw panic=1 earlyprintk=serial,ttyS0,115200"

This launches a machine of type “pc”, without graphics, 512M of RAM, two CPUs, a provided Linux 5.9 kernel as well as a single IDE drive. The -append string is passed to the Linux kernel and serves to indicate the serial console and the location of the rootfs.

The 512M of main memory specified with -m 512m uses the default QEMU memory backend. However, QEMU has alternative memory backends.

memory-backend-ram – The default backend
memory-backend-file – Memory backed by a file, typcially used for non volatile memories (to save the contents when powering off the machine)
memory-backend-memfd – Anonymous memory file backend

The QEMU Documentation suggests to use memory-backend-memfd to share the memory with external processes. But I prefer to use the memory-backend-file with the /dev/shm virtual filesystem. The result will be the same, both use shared memory, but with the file backend I can give the file a name (instead of it being anonymous). This makes sharing the memory easier because it can be done by file name instead of file descriptor.

The journey to shared memory

The QEMU Documentation gives the following command and options

“Creates a memory file backend object, which can be used to back the guest RAM with huge pages.”

So now, how should we adapt the QEMU launch command to make our memory use this backend ?

There are some examples of this command being used, for example with NVDIMMs, as well as an unanswered question on stackexchange.

With this info I tried running QEMU with the following command

qemu -M pc -nographic -m 512m -object memory-backend-file,id=mem,size=512M,mem-path=/dev/shm/qemu-ram,share=on -smp cpus=2 -kernel ./bzImage_5.9 -drive file=./rootfs-target.img,if=ide -append "console=ttyS0 root=/dev/sda rw panic=1 earlyprintk=serial,ttyS0,115200"

This launches QEMU and creates the file /dev/shm/qemu-ram however, the /dev/shm/qemu-ram file is blank and does not reflect the RAM contents, (e.g., running shasum on the file always returns the same value).

So I went back to check if the memory region was used in QEMU. I opened a QEMU console (ctrl-a c) and with

(qemu) info mtree

I get

Where I can see that the main RAM (512M) is called pc.ram and is visible in the system memory region (0x0000'0000-0x1fff'ffff). However, the object mem I added is nowhere to be found (you can switch back from the QEMU console with the same (ctrl-a c) command).

The first thing that comes in mind is renaming my object pc.ram

qemu -M pc -nographic -m 512m -object memory-backend-file,id=pc.ram,size=512M,mem-path=/dev/shm/qemu-ram,share=on -smp cpus=2 -kernel ./bzImage_5.9 -drive file=./rootfs-target.img,if=ide -append "console=ttyS0 root=/dev/sda rw panic=1 earlyprintk=serial,ttyS0,115200"

When running QEMU the following error appears

The main memory already exists, with id pc.ram, so a new object with this id cannot be added.

The problem still remains… The solution was nowhere to be found in the QEMU documentation, after searching the web for some time I came across the –machine memory-backend option in some posts in the QEMU development mailing list. Later I found a post that indicates that the option is currently undocumented and suggests it should be.

With this extra information I was able to create the following QEMU command

qemu -M pc -nographic -m 512m -object memory-backend-file,id=pc.ram,size=512M,mem-path=/dev/shm/qemu-ram,share=on -machine memory-backend=pc.ram -smp cpus=2 -kernel ./bzImage_5.9 -drive file=./rootfs-target.img,if=ide -append "console=ttyS0 root=/dev/sda rw panic=1 earlyprintk=serial,ttyS0,115200"

This does launch without errors and creates the file /dev/shm/qemu-ram, running shasum on the file gives different results each time meaning the file is being updated, a good sign !

Sharing the memory – Demonstration

The solution was to add the machine memory backend option highlighted above to the QEMU command so that it would use the file backed memory object created. Note that the id can be something else than pc.ram but needs to be the same in both the object and machine options. It will appear with the chosen id in the QEMU memory tree.

The QEMU launch command with highlighted extra options for the memory backend is

As a demonstration to access and modify the RAM of the emulated PC from another process, I’ve written a small utility shmem2 (based on devmem2) to read and write shared memory files. Compile with

gcc -m32 -o shmem2 shmem2.c -lrt

Usage of this utility is shown below

To test that it works let’s just look at physical address 0x0000'0000 of the RAM from inside the emulated system with devmem2 (my emulated pc is running a Linux 5.9 kernel with a Ubuntu rootfs with the devmem2 utility installed, but you can compile it from source if needed or maybe already have access to physical memory if you are emulating a “bare-metal” platform).

The RAM holds the value 0xF000FF53. Now let’s open the shared memory file with shmem2 at the same physical address 0x0000'0000

We can see that we read the correct value ! Now let’s write something back. It is usually not advisable to write random values to random addresses of RAM, unless maybe you are trying to simulate data corruption. Here I do it for the demonstration.

Let’s read the memory at physical address 0x0000'0000 from inside the emulated machine again.

The RAM contents have indeed changed ! We have successfully demonstrated sharing the memory of a machine emulated in QEMU with an external process, here shmem2.

Mapping the shared memory in a C program

You may be interested in mapping the RAM of the emulated system in C. For example, to code a library, maybe that you can call from python, to read and write the RAM of the emulated system.

All you have to do is open the shared memory file and map it

int schmid = 0;
if ((shmid = shm_open("/qemu-ram", O_RDWR, 0)) < 0) { perror("shm_open(qemu-ram) failed"); exit(1); }
volatile uint8_t *qemu_ram = mmap(NULL, 512*1024*1024 /* 512 M */, PROT_READ | PROT_WRITE, MAP_SHARED, shmid, 0); if (qemu_ram == MAP_FAILED) { perror("DDR Memory map failed"); exit(1); }

Now your C program has access to the whole RAM of the emulated system through the *qemu_ram pointer.

This can be done for other QEMU emulated systems as well, not only the “pc” machine shown here, this applies to embedded systems and can really come in handy, especially if some devices are externally simulated and require to access the main RAM.

Other use cases can be to monitor the RAM or to edit it from outside QEMU for testing purposes, or maybe to access the RAM from a simulated RTL (e.g., VHDL, Verilog) design through the SysteVerilog DPI.

Conclusion

Sharing the memory of an emulated machine in QEMU is relatively simple once you know the options to use. I hope that this feature will be useful to you and that this post saves you some time where I had to find the solution by trial and error.

TL DR

Step 1) Add

-object memory-backend-file,id=mem,size=512M,mem-path=/dev/shm/qemu-ram,share=on -machine memory-backend=mem

To your QEMU command

Note : Choose the correct size, the same as specified by -m, here -m 512m, therefore size=512M.

Step 2) Open /dev/shm/qemu-ram from your external process

Step 3) Profit !