All Posts By


State of KernelCI and new blog

By Blog

KernelCI officially launched as a Linux Foundation project at the end of October 2019. Now that we’re several months into the project, it’s time to launch a blog so everyone can keep track of what’s going on.

To get the blog started, here’s a bit of what we’ve been up to since the project launch.

Setup / Administration

To get things off to an official start, we spent some time with the member companies getting to know each other better and establishing our basic governance structure.  We now have a governing board, made up of one representative from each of the member companies: BayLibre, Civil Infrastructure Platform, Collabora,, Google, Microsoft and Red Hat.  The governing board meets weekly for planning, discussion, budgeting decisions and to set priorities for the technical steering committee.

Recent GB accomplishments

Getting Technical

The technical steering committee (TSC) is where technical decisions are made and software implementation details are discussed during a weekly call as well as on our mailing list.  Both the weekly call and mailing list are open to the public (details below.)

Some highlights of recent accomplishments

  • improvements to frontend to view test results (previously only boot reports were available in the UI)
  • building the kernel with clang (including latest version: clang-10)
  • kcidb
    • prototype of a new schema and datastore/schema using Google BigQuery
    • combined results from and Red Hat CKI
    • prototype of new email reports based on kcidb
    • prototype of new web UI using kcidb based on Grafana

And some active areas of development and future work ( help wanted! )

  • increasing test coverage by enabling more test suites to run (kselftest, KUnit, LTP, xfstests, etc…)
  • increased build capacity: thanks to generous donations from Google and Microsoft, we are migrating to using Kubernetes to manage our kernel build capacity using VMs from Google Cloud Platform (GCP) and Microsoft Azure.  We expect to roll this out over the next month or two.
  • modernization of CI/CD pipelines (e.g. container-centric tools like Tekton)
  • re-factoring and upgrading codebase and tooling to python3
  • improving tools to become more generic and reusable in other projects doing Kernel CI
  • modernizing the UI / UX

These are just a few highlights, and we have many more ideas, but things move slow since we are a very small team of active developers.  If you have any interest in helping in any of these areas, please don’t hesitate to reach out.

Next steps

Building on all the great progress above, we are now gearing up for more collaboration and discussions as part of the testing and fuzzing microconference at Linux Plumbers Conference this August.  The KernelCI project will be actively participating in the microconference as we aim to bring our tools to more developers, listen to feedback on some of our new ideas and directions, and make decisions on future directions.

Getting involved

Even if you cannot be involved directly at  Linux Plumbers, there are many other ways to be involved in the discussion.



The New Stack: KernelCI Joins Linux Foundation to Accelerate Linux Kernel Hardware Testing

By News

Testing software can present difficulties enough on its own, but when you’re talking about the Linux kernel — that piece of software that directly interfaces with hardware — there are innumerable more complexities added to the scenario. One of those complexities, of course, is the number of devices Linux is intended to run on, from a variety of desktops, laptops and phones, to Raspberry Pis and other devices on the edge. Each device presents a new environment in which the Linux kernel needs to be tested, and that’s where KernelCI comes into the equation.

Read More »

ZDNet: Automated testing comes to the Linux kernel: KernelCI

By News

At the recent Linux Kernel Plumbers get-together in Lisbon, Portugal, one of the hottest topics was how to bring better and automated testing to the Linux kernel. There, the top Linux developers united their efforts behind one testing framework: KernelCI. Now, at Open Source Summit Europe in Lyon, France, to help give KernelCI the resources it needs to be successful, it became a Linux Foundation project.

Read More

TechStartUps: KernelCI testing platform becomes part of Linux Foundation project, now supported with funding from Microsoft, Google, Red Hat,, and others

By News

The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced that KernelCI testing platform is becoming a Linux Foundation project. The project by Civil Infrastructure Platform,, Google, Microsoft, and Red Hat, which now join early collaborators BayLibre and Collabora to support Linux testing on the largest variety of hardware platforms. With the new level of support from tech heavyweights, KernelCI will now be able to accelerate its work to test Linux on the largest variety of hardware platforms.

Read More

The Register: Annoyed by too many kernel testing projects? Good news. Linux Foundation anoints chosen one – KernelCI

By News

“We have six or seven projects which are doing kernel testing, so much so that we’re getting really annoyed,” kernel maintainer Greg Kroah-Hartman told a crowd today.

“We have six or seven projects which are doing kernel testing, so much so that we’re getting really annoyed,” kernel maintainer Greg Kroah-Hartman told a crowd today.

Read More

Distributed Linux Testing Platform KernelCI Secures Funding and Long-Term Sustainability as New Linux Foundation Project

By News

Civil Infrastructure Platform,, Google, Microsoft, and Red Hat join early collaborators BayLibre and Collabora to support Linux testing on the largest variety of hardware platforms 

Lyon, France, Open Source Summit Europe, October 28, 2019 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced the KernelCI testing platform is becoming a Linux Foundation project underwritten by BayLibre, Civil Infrastructure Platform, Collabora,, Google, Microsoft, Red Hat. As a Linux Foundation project, KernelCI will accelerate its work to test Linux on the largest variety of hardware platforms.

The Linux kernel is developed by a large, collaborative open source community working together to continuously improve the software. Conversely, Linux kernel testing is often fragmented since it is largely done in private silos with little collaboration on the testing software or methodologies. KernelCI is a community-based open source distributed test automation system focused on upstream kernel development. It was originally started in 2014 as a side project by a few engineers who were doing the testing at home and in their spare time. A variety of hardware labs contributed to the work over time, but until now there was no sustainable structure in place for open governance and contribution, or expanded access for the developers to hardware.

“Testing is traditionally done only on the most common hardware. But because Linux runs on more hardware than any other operating system, it’s important to also test it on all that hardware. The Linux Foundation’s support is enabling us to expand the great work we started five years ago and sets us up for a bright future with a growing community,” said Kevin Hilman, co-founder of embedded Linux consultancy BayLibre and co-founder of the KernelCI project.

“KernelCI represents the passion and diligence we see among developers all over the world. The individuals who have given their nights and weekends to this work deserve our gratitude and support, and we’re excited to be able to host this work at the Linux Foundation,” said Jim Zemlin, executive director at the Linux Foundation.

The primary goal of KernelCI is to use an open testing philosophy to improve the quality, stability and long-term maintenance of the Linux kernel. Expected improvements to the platform under the Linux Foundation include improved LTS kernel testing and validation; consolidation of existing testing initiatives; quality-of-life improvements to the current service; expanded compute resources; and increased pool of hardware to be tested. In the long-term, members expect to modernize the architecture; test software beyond the Linux kernel; and define testing standards and engage in cross-project collaboration.

For more information or to contribute to this work, please visit:


Civil Infrastructure Platform
“The Civil Infrastructure Platform (CIP) aims to maintain Linux for decades and more. In times of ever-changing, dynamic software like the Linux kernel, this very challenging task is unthinkable without advanced testing strategies and infrastructure that can also navigate the intricacies of real hardware,” said Prof. Dr. Wolfgang Mauerer, Senior Research Scientist at Siemens Corporate Technology and member of the CIP Technical Steering Committee. “The CIP project has long since relied on KernelCI for distributed, heterogeneous testing, and we are excited to see the development of KernelCI advance as a Linux Foundation project.”

“Provided how crucial Linux has become to society, achieving comprehensive test coverage of the Linux kernel is essential. By applying the open source philosophy to testing, the KernelCI distributed architecture enables the whole kernel community to collaborate around a single upstream CI system. Becoming part of the Linux Foundation shines the light on the progress we have made with KernelCI and enables the project to become integral part of the Linux kernel development workflow,” said Guy Lunardi, VP Business Development at Open Source software consultancy and leading KernelCI contributor Collabora.
“At, we believe the latest software is the most secure software. That is why it is important to have a unified continuous integration project dedicated to the Linux kernel, which focuses on a broad range of architectures and hardware.” said  Tyler Baker, CTO, “This enables us and our customers to run the latest software with confidence. As co-founders of KernelCI, we are excited to help expand the reach of the project through continued contributions and guidance.”

“At Microsoft, we believe in making Linux the best platform it can be for customers,” said KY Srinivasan, general manager, Azure. “By securing funding and long-term sustainability for the KernelCI Testing platform, we’re collaborating together to harden Linux for the broader Linux community.”

Red Hat
“Red Hat invests in upstream projects like KernelCI to further scale the fundamental open source advantage offered by the ‘Bazaar.’ By facilitating engagement between Linux communities and partners earlier and more frequently, we’re helping to enable rapid innovation while bringing the benefits of open technologies to the enterprise through the thousands of projects that comprise Red Hat Enterprise Linux.”

About the Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see its trademark usage page: Linux is a registered trademark of Linus Torvalds.


Media Contact
Jennifer Cloer
reTHINKit Media