The Linux Kernel is under constant development and improvement. Everyday patches are submitted to the Linux Kernel Mailing List (LKML). Some of these patches get accepted and merged into the mainline Linux kernel and become available to the user, other patches never do.
Sometimes it is useful to get patches from the LKML, for example if you are developing in the kernel or simply because you want to stay at the bleeding edge. Another reason could be that you need patches that were proposed to the LKML but were never merged. This can happen when developing with exotic hardware, for example a driver could have been submitted but never merged because of some reason (e.g., code doesn’t follow the kernel guidelines etc.), however this code might still be of interest to you.
In this post we will explore how we can incorporate code from the LKML into our kernel.
Getting the Linux Kernel source code
Source code for the Linux kernel is available from https://www.kernel.org it can be downloaded or cloned with git. When we go the to git repositories page https://git.kernel.org one might feel overwhelmed by the number of repositories (there are hundreds of them).
The main repository is shown in red in the image above. Once you click on the repository the links to clone it will be visible.
It is possible to clone the repository with these links. Here I used the “https” link.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Exploring the Linux Kernel Mailing List
There are several ways to explore the LKML.
- https://patchwork.kernel.org
- http://vger.kernel.org/vger-lists.html
- https://lore.kernel.org
- https://lkml.org
There is also LKML FAQ which contains a treasure trove of information. Personally I like to use patchwork to look at new patches submitted to the LKML, and use https://lore.kernel.org to search for specific terms.
Apply a patch or a patch series from the LKML to your kernel
In most cases we don’t receive the emails with the patches directly, either because we don’t subscribe to the mailing list or simply because we don’t have an email client installed on our development machine. So we will retrieve the patch or series directly from on of the LKML archives from https://lore.kernel.org.
When we identified a patch or a patch series that is of interest for us, there are several ways to apply them locally. For example this series of patches to the kernel that was written to fix some issues with the PCIe endpoint test driver. First find the patches you are interested in on https://lore.kernel.org. Then on the thread summary you will find a link to a mbox.gz file as shown below
This file (t.mbox.gz) can be downloaded, let’s create a directory “patches” in our clone of the Linux kernel, download the file (copy the link on the page or manually download it), and extract it.
The t.mbox file is a “mailbox” file, in plain text, one can open it with a text viewer (e.g., less) or editor (vi/emacs). It contains the mails exchanged in the LKML thread. Patches from these email exchanges can be applied with git am. Let’s first create a local development branch to which we will apply the patches, then we will use git am to apply patches from this series.
git will complain that the patch is empty. This is normal if the patch series comes with a “cover letter” (patch 0/X), this email (the cover letter) is used to explain a patch series (multiple patches that go together) and doesn’t contain a patch. So don’t worry about the error message and just skip this “patch” (message) that isn’t a patch “git am -i –skip”.
Once you skip the first message, the next one will be presented to you with an interactive prompt as shown above. You can view the patch with “v”, edit it, or apply it. You can even accept all patches in the series.
Let’s review (view) them one-by-one and apply them if they are appropriate. Here we will apply them all. Then we might encounter more “Patch is empty” messages, these are from the emails when people reply to the mailing list without submitting a patch, but rather responding in text, e.g., commenting or replying to the author of the patch. Use “git am -i –skip” until you’ve gone through all the messages. Now if you check the git log with git log you can see that the patches have been applied.
Depending on the version of the Linux kernel for which the patches have been developed and the version you currently work on there might be issues and patches might require edits. Some patches also have dependencies on other patches or require to be applied in a given order. This is often documented in the patch itself or the patch series cover letter. Patches in a given series are usually ordered so that they can be applied in that order. In our case here we were able to directly apply them. We can now compile the kernel with the new patches !
Apply a patch or a patch series from the LKML to your kernel with b4
Developers of the Linux kernel have created a tool called b4 https://pypi.org/project/b4/ the tool’s description is the following :
“This is a helper utility to work with patches and pull requests made available via a public-inbox archive like lore.kernel.org. It is written to make it easier to participate in a patch-based workflows, like those used in the Linux kernel development.”
“The name “b4” was chosen for ease of typing and because B-4 was the precursor to Lore and Data in the Star Trek universe.”
The documentation is available here : https://b4.docs.kernel.org/en/latest/ it can be install through pip :
# Make sure you have pip installed
pip install b4
With b4 the process is a bit simpler and cleaner. First, identify the patch or patch series you are interested in. Once you found it on https://lore.kernel.org get the thread ID as shown below.
Then you can use b4 to download the patches. The nice thing with b4 is that it will automatically split the patches, cover letter and messages. It will also show information such as “Reviewed-by”, “Tested-by” etc.
# The -o "output" specifies the patches directory
b4 am -o patches 20230215032155.74993-1-damien.lemoal@opensource.wdc.com
We can see that in contrast to directly downloading the mailbox as per the previous method, here the patches are listed, the cover letter taken separately and we also have a nice list of patches with the extra information such as who reviewed the patches. “b4” also informs us on how to apply the patches.
Now we can use git am to apply the patches, with the following command we can interactively apply/edit/skip patches
git am -i patches/20230215_damien_lemoal_pci_endpoint_fixes_and_improvements.mbx
The advantages is that now we don’t have plenty of error messages related to “empty patches” because b4 cleaned the mailbox file for us. Therefore the git am command will only apply the patches and ignore the cover letter and messages. If we now check with git log we can see that the patches have been applied.
Conclusion
Applying patches from the Linux Kernel Mailing List may look mysterious at first but with the proper tools it is not that complicated. Here we downloaded the mailboxes from a website, if you are a kernel developer you will probably use your own mailbox and local client e.g., mutt where you can save a conversation as .mbox file. Nevertheless, “b4” is a very useful tool and it has plenty of other features that help with handling patches in the LKML.
With this feel free to follow LKML of your interest and try out the latests patches as soon as they are submitted ! This allows you to test and play around with very new and experimental features or simply use/test drivers way before they become available in the mainline version of Linux.
Happy Hacking !
Further Reading
- “b4” documentation : https://b4.docs.kernel.org/en/latest/
- “Doing more with lore and b4” presentation by Konstantin Ryabitsev, Linux Foundation (wayback machine backup link)
- “Doing more with lore and b4” YouTube video by Konstantin Ryabitsev, Linux Foundation
- Linux Weekly News (LWN)
- https://en.wikipedia.org/wiki/Linux_kernel_mailing_list
- https://en.wikipedia.org/wiki/LWN.net
Tux image in post header is by Iwan Gabovitch source : https://commons.wikimedia.org/wiki/File:Tux_Mono.svg