We introduced Android’s Private Compute Core in Android 12 Beta. Today, we’re excited to announce a new suite of services that provide a privacy-preserving bridge between Private Compute Core and the cloud.

Recap: What is Private Compute Core?

Android’s Private Compute Core is an open source, secure environment that is isolated from the rest of the operating system and apps. With each new Android release we’ll add more privacy-preserving features to the Private Compute Core. Today, these include:

  • Live Caption, which adds captions to any media using Google’s on-device speech recognition
  • Now Playing, which recognizes music playing nearby and displays the song title and artist name on your device’s lock screen
  • Smart Reply, which suggests relevant responses based on the conversation you’re having in messaging apps

For these features to be private, they must:

  1. Keep the information on your device private. Android ensures that the sensitive data processed in the Private Compute Core is not shared to any apps without you taking an action. For instance, until you tap a Smart Reply, the OS keeps your reply hidden from both your keyboard and the app you’re typing into.
  2. Let your device use the cloud (to download new song catalogs or speech-recognition models) without compromising your privacy. This is where Private Compute Services comes in.

Introducing Android’s Private Compute Services

Machine learning features often improve by updating models, and Private Compute Services helps features get these updates over a private path. Android prevents any feature inside the Private Compute Core from having direct access to the network. Instead, features communicate over a small set of purposeful open-source APIs to Private Compute Services, which strips out identifying information and uses a set of privacy technologies, including Federated Learning, Federated Analytics, and Private information retrieval.

We will publicly publish the source code for Private Compute Services, so it can be audited by security researchers and other teams outside of Google. This means it can go through the same rigorous security programs that ensure the safety of the Android platform.

We’re enthusiastic about the potential for machine learning to power more helpful features inside Android, and Android’s Private Compute Core will help users benefit from these features while strengthening privacy protections via the new Private Compute Services. Android is the first open source mobile OS to include this kind of externally verifiable privacy; Private Compute Services helps the Android OS continue to innovate in machine learning, while also maintaining the highest standards of privacy and security.

One of the main challenges of evaluating Rust for use within the Android platform was ensuring we could provide sufficient interoperability with our existing codebase. If Rust is to meet its goals of improving security, stability, and quality Android-wide, we need to be able to use Rust anywhere in the codebase that native code is required. To accomplish this, we need to provide the majority of functionality platform developers use. As we discussed previously, we have too much C++ to consider ignoring it, rewriting all of it is infeasible, and rewriting older code would likely be counterproductive as the bugs in that code have largely been fixed. This means interoperability is the most practical way forward.

Before introducing Rust into the Android Open Source Project (AOSP), we needed to demonstrate that Rust interoperability with C and C++ is sufficient for practical, convenient, and safe use within Android. Adding a new language has costs; we needed to demonstrate that Rust would be able to scale across the codebase and meet its potential in order to justify those costs. This post will cover the analysis we did more than a year ago while we evaluated Rust for use in Android. We also present a follow-up analysis with some insights into how the original analysis has held up as Android projects have adopted Rust.

Language interoperability in Android

Existing language interoperability in Android focuses on well defined foreign-function interface (FFI) boundaries, which is where code written in one programming language calls into code written in a different language. Rust support will likewise focus on the FFI boundary as this is consistent with how AOSP projects are developed, how code is shared, and how dependencies are managed. For Rust interoperability with C, the C application binary interface (ABI) is already sufficient.

Interoperability with C++ is more challenging and is the focus of this post. While both Rust and C++ support using the C ABI, it is not sufficient for idiomatic usage of either language. Simply enumerating the features of each language results in an unsurprising conclusion: many concepts are not easily translatable, nor do we necessarily want them to be. After all, we’re introducing Rust because many features and characteristics of C++ make it difficult to write safe and correct code. Therefore, our goal is not to consider all language features, but rather to analyze how Android uses C++ and ensure that interop is convenient for the vast majority of our use cases.

We analyzed code and interfaces in the Android platform specifically, not codebases in general. While this means our specific conclusions may not be accurate for other codebases, we hope the methodology can help others to make a more informed decision about introducing Rust into their large codebase. Our colleagues on the Chrome browser team have done a similar analysis, which you can find here.

This analysis was not originally intended to be published outside of Google: our goal was to make a data-driven decision on whether or not Rust was a good choice for systems development in Android. While the analysis is intended to be accurate and actionable, it was never intended to be comprehensive, and we’ve pointed out a couple of areas where it could be more complete. However, we also note that initial investigations into these areas showed that they would not significantly impact the results, which is why we decided to not invest the additional effort.

Methodology

Exported functions from Rust and C++ libraries are where we consider interop to be essential. Our goals are simple:

  • Rust must be able to call functions from C++ libraries and vice versa.
  • FFI should require a minimum of boilerplate.
  • FFI should not require deep expertise.

While making Rust functions callable from C++ is a goal, this analysis focuses on making C++ functions available to Rust so that new Rust code can be added while taking advantage of existing implementations in C++. To that end, we look at exported C++ functions and consider existing and planned compatibility with Rust via the C ABI and compatibility libraries. Types are extracted by running objdump on shared libraries to find external C++ functions they use1 and running c++filt to parse the C++ types. This gives functions and their arguments. It does not consider return values, but a preliminary analysis2 of those revealed that they would not significantly affect the results.

We then classify each of these types into one of the following buckets:

Supported by bindgen

These are generally simple types involving primitives (including pointers and references to them). For these types, Rust’s existing FFI will handle them correctly, and Android’s build system will auto-generate the bindings.

Supported by cxx compat crate

These are handled by the cxx crate. This currently includes std::string, std::vector, and C++ methods (including pointers/references to these types). Users simply have to define the types and functions they want to share across languages and cxx will generate the code to do that safely.

Native support

These types are not directly supported, but the interfaces that use them have been manually reworked to add Rust support. Specifically, this includes types used by AIDL and protobufs.

We have also implemented a native interface for StatsD as the existing C++ interface relies on method overloading, which is not well supported by bindgen and cxx3. Usage of this system does not show up in the analysis because the C++ API does not use any unique types.

Potential addition to cxx

This is currently common data structures such as std::optional and std::chrono::duration and custom string and vector implementations.

These can either be supported natively by a future contribution to cxx, or by using its ExternType facilities. We have only included types in this category that we believe are relatively straightforward to implement and have a reasonable chance of being accepted into the cxx project.

We don’t need/intend to support

Some types are exposed in today’s C++ APIs that are either an implicit part of the API, not an API we expect to want to use from Rust, or are language specific. Examples of types we do not intend to support include:

  • Mutexes – we expect that locking will take place in one language or the other, rather than needing to pass mutexes between languages, as per our coarse-grained philosophy.
  • native_handle – this is a JNI interface type, so it is inappropriate for use in Rust/C++ communication.
  • std::locale& – Android uses a separate locale system from C++ locales. This type primarily appears in output due to e.g., cout usage, which would be inappropriate to use in Rust.

Overall, this category represents types that we do not believe a Rust developer should be using.

HIDL

Android is in the process of deprecating HIDL and migrating to AIDL for HALs for new services.We’re also migrating some existing implementations to stable AIDL. Our current plan is to not support HIDL, preferring to migrate to stable AIDL instead. These types thus currently fall into the “We don’t need/intend to support” bucket above, but we break them out to be more specific. If there is sufficient demand for HIDL support, we may revisit this decision later.

Other

This contains all types that do not fit into any of the above buckets. It is currently mostly std::string being passed by value, which is not supported by cxx.

Top C++ libraries

One of the primary reasons for supporting interop is to allow reuse of existing code. With this in mind, we determined the most commonly used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed all of the external C++ functions used by these libraries and their arguments to determine how well they would interoperate with Rust.

Overall, 81% of types are in the first three categories (which we currently fully support) and 87% are in the first four categories (which includes those we believe we can easily support). Almost all of the remaining types are those we believe we do not need to support.

Mainline modules

In addition to analyzing popular C++ libraries, we also examined Mainline modules. Supporting this context is critical as Android is migrating some of its core functionality to Mainline, including much of the native code we hope to augment with Rust. Additionally, their modularity presents an opportunity for interop support.

We analyzed 64 binaries and libraries in 21 modules. For each analyzed library we examined their used C++ functions and analyzed the types of their arguments to determine how well they would interoperate with Rust in the same way we did above for the top 10 libraries.

Here 88% of types are in the first three categories and 90% in the first four, with almost all of the remaining being types we do not need to handle.

Analysis of Rust/C++ Interop in AOSP

With almost a year of Rust development in AOSP behind us, and more than a hundred thousand lines of code written in Rust, we can now examine how our original analysis has held up based on how C/C++ code is currently called from Rust in AOSP.4

The results largely match what we expected from our analysis with bindgen handling the majority of interop needs. Extensive use of AIDL by the new Keystore2 service results in the primary difference between our original analysis and actual Rust usage in the “Native Support” category.

A few current examples of interop are:

  • Cxx in Bluetooth – While Rust is intended to be the primary language for Bluetooth, migrating from the existing C/C++ implementation will happen in stages. Using cxx allows the Bluetooth team to more easily serve legacy protocols like HIDL until they are phased out by using the existing C++ support to incrementally migrate their service.
  • AIDL in keystore – Keystore implements AIDL services and interacts with apps and other services over AIDL. Providing this functionality would be difficult to support with tools like cxx or bindgen, but the native AIDL support is simple and ergonomic to use.
  • Manually-written wrappers in profcollectd – While our goal is to provide seamless interop for most use cases, we also want to demonstrate that, even when auto-generated interop solutions are not an option, manually creating them can be simple and straightforward. Profcollectd is a small daemon that only exists on non-production engineering builds. Instead of using cxx it uses some small manually-written C wrappers around C++ libraries that it then passes to bindgen.

Conclusion

Bindgen and cxx provide the vast majority of Rust/C++ interoperability needed by Android. For some of the exceptions, such as AIDL, the native version provides convenient interop between Rust and other languages. Manually written wrappers can be used to handle the few remaining types and functions not supported by other options as well as to create ergonomic Rust APIs. Overall, we believe interoperability between Rust and C++ is already largely sufficient for convenient use of Rust within Android.

If you are considering how Rust could integrate into your C++ project, we recommend doing a similar analysis of your codebase. When addressing interop gaps, we recommend that you consider upstreaming support to existing compat libraries like cxx.

Acknowledgements

Our first attempt at quantifying Rust/C++ interop involved analyzing the potential mismatches between the languages. This led to a lot of interesting information, but was difficult to draw actionable conclusions from. Rather than enumerating all the potential places where interop could occur, Stephen Hines suggested that we instead consider how code is currently shared between C/C++ projects as a reasonable proxy for where we’ll also likely want interop for Rust. This provided us with actionable information that was straightforward to prioritize and implement. Looking back, the data from our real-world Rust usage has reinforced that the initial methodology was sound. Thanks Stephen!

Also, thanks to:

  • Andrei Homescu and Stephen Crane for contributing AIDL support to AOSP.
  • Ivan Lozano for contributing protobuf support to AOSP.
  • David Tolnay for publishing cxx and accepting our contributions.
  • The many authors and contributors to bindgen.
  • Jeff Vander Stoep and Adrian Taylor for contributions to this post.


  1. We used undefined symbols of function type as reported by objdump to perform this analysis. This means that any header-only functions will be absent from our analysis, and internal (non-API) functions which are called by header-only functions may appear in it. 

  2. We extracted return values by parsing DWARF symbols, which give the return types of functions. 

  3. Even without automated binding generation, manually implementing the bindings is straightforward. 

  4. In the case of handwritten C/C++ wrappers, we analyzed the functions they call, not the wrappers themselves. For all uses of our native AIDL library, we analyzed the types used in the C++ version of the library. 

Integrating security into your app development lifecycle can save a lot of time, money, and risk. That’s why we’ve launched Security by Design on Google Play Academy to help developers identify, mitigate, and proactively protect against security threats.

The Android ecosystem, including Google Play, has many built-in security features that help protect developers and users. The course Introduction to app security best practices takes these protections one step further by helping you take advantage of additional security features to build into your app. For example, Jetpack Security helps developers properly encrypt their data at rest and provides only safe and well known algorithms for encrypting Files and SharedPreferences. The SafetyNet Attestation API is a solution to help identify potentially dangerous patterns in usage. There are several common design vulnerabilities that are important to look out for, including using shared or improper file storage, using insecure protocols, unprotected components such as Activities, and more. The course also provides methods to test your app in order to help you keep it safe after launch. Finally, you can set up a Vulnerability Disclosure Program (VDP) to engage security researchers to help.

In the next course, you can learn how to integrate security at every stage of the development process by adopting the Security Development Lifecycle (SDL). The SDL is an industry standard process and in this course you’ll learn the fundamentals of setting up a program, getting executive sponsorship and integration into your development lifecycle.

Threat modeling is part of the Security Development Lifecycle, and in this course you will learn to think like an attacker to identify, categorize, and address threats. By doing so early in the design phase of development, you can identify potential threats and start planning for how to mitigate them at a much lower cost and create a more secure product for your users.

Improving your app’s security is a never ending process. Sign up for the Security by Design module where in a few short courses, you will learn how to integrate security into your app development lifecycle, model potential threats, and app security best practices into your app, as well as avoid potential design pitfalls.

Integrating security into your app development lifecycle can save a lot of time, money, and risk. That’s why we’ve launched Security by Design on Google Play Academy to help developers identify, mitigate, and proactively protect against security threats.

The Android ecosystem, including Google Play, has many built-in security features that help protect developers and users. The course Introduction to app security best practices takes these protections one step further by helping you take advantage of additional security features to build into your app. For example, Jetpack Security helps developers properly encrypt their data at rest and provides only safe and well known algorithms for encrypting Files and SharedPreferences. The SafetyNet Attestation API is a solution to help identify potentially dangerous patterns in usage. There are several common design vulnerabilities that are important to look out for, including using shared or improper file storage, using insecure protocols, unprotected components such as Activities, and more. The course also provides methods to test your app in order to help you keep it safe after launch. Finally, you can set up a Vulnerability Disclosure Program (VDP) to engage security researchers to help.

In the next course, you can learn how to integrate security at every stage of the development process by adopting the Security Development Lifecycle (SDL). The SDL is an industry standard process and in this course you’ll learn the fundamentals of setting up a program, getting executive sponsorship and integration into your development lifecycle.

Threat modeling is part of the Security Development Lifecycle, and in this course you will learn to think like an attacker to identify, categorize, and address threats. By doing so early in the design phase of development, you can identify potential threats and start planning for how to mitigate them at a much lower cost and create a more secure product for your users.

Improving your app’s security is a never ending process. Sign up for the Security by Design module where in a few short courses, you will learn how to integrate security into your app development lifecycle, model potential threats, and app security best practices into your app, as well as avoid potential design pitfalls.

The Android team has been working on introducing the Rust programming language into the Android Open Source Project (AOSP) since 2019 as a memory-safe alternative for platform native code development. As with any large project, introducing a new language requires careful consideration. For Android, one important area was assessing how to best fit Rust into Android’s build system. Currently this means the Soong build system (where the Rust support resides), but these design decisions and considerations are equally applicable for Bazel when AOSP migrates to that build system. This post discusses some of the key design considerations and resulting decisions we made in integrating Rust support into Android’s build system.

Rust integration into large projects

A RustConf 2019 meeting on Rust usage within large organizations highlighted several challenges, such as the risk that eschewing Cargo in favor of using the Rust Compiler, rustc, directly (see next section) may remove organizations from the wider Rust community. We share this same concern. When changes to imported third-party crates might be beneficial to the wider community, our goal is to upstream those changes. Likewise when crates developed for Android could benefit the wider Rust community, we hope to release them as independent crates. We believe that the success of Rust within Android is dependent on minimizing any divergence between Android and the Rust community at large, and hope that the Rust community will benefit from Android’s involvement.

No nested build systems

Rust provides Cargo as the default build system and package manager, collecting dependencies and invoking rustc (the Rust compiler) to build the target crate (Rust package). Soong takes this role instead in Android and calls rustc directly for several reasons:

  • In Cargo, C dependencies are handled independently in an ad-hoc manner via build.rs scripts. Soong already provides a mechanism for building C libraries and defining them as dependencies, and Android carefully controls the compiler version and global compilation flags to ensure libraries are built a particular way. Relying on Cargo would introduce a second non-Soong mechanism for defining/building C libraries that would not be constrained by the carefully selected compilation controls implemented in Soong. This could also lead to multiple different versions of the same library, negatively impacting memory/disk usage.
  • Calling compilers directly through Soong provides the stability and control Android requires for the variety of build configurations it supports (for example, specifying where target-specific dependencies are and which compilation flags to use). While it would technically be possible to achieve the necessary level of control over rustc indirectly through Cargo, Soong would have no understanding of how the Cargo.toml (the Cargo build file) would influence the commands Cargo emits to rustc. Paired with the fact that Cargo evolves independently, this would severely restrict Soong’s ability to precisely control how build artifacts are created.
  • Builds which are self-contained and insensitive to the host configuration, known as hermetic builds, are necessary for Android to produce reproducible builds. Cargo, which relies on build.rs scripts, doesn’t yet provide hermeticity guarantees.
  • Incremental builds are important to maintain engineering productivity; building Android takes a considerable amount of resources. Cargo was not designed for integration into existing build systems and does not expose its compilation units. Each Cargo invocation builds the entire crate dependency graph for a given Cargo.toml, rebuilding crates multiple times across projects1. This is too coarse for integration into Soong’s incremental build support, which expects smaller compilation units. This support is necessary to scale up Rust usage within Android.

    Using the Rust compiler directly allows us to avoid these issues and is consistent with how we compile all other code in AOSP. It provides the most control over the build process and eases integration into Android’s existing build system. Unfortunately, avoiding it introduces several challenges and influences many other build system decisions because Cargo usage is so deeply ingrained in the Rust crate ecosystem.

    No build.rs scripts

    A build.rs script compiles to a Rust binary which Cargo builds and executes during a build to handle pre-build tasks, commonly setting up the build environment, or building libraries in other languages (for example C/C++). This is analogous to configure scripts used for other languages.

    Avoiding build.rs scripts somewhat flows naturally from not relying on Cargo since supporting these would require replicating Cargo behavior and assumptions. Beyond this however, there are good reasons for AOSP to avoid build scripts as well:

    • build.rs scripts can execute arbitrary code on the build host. From a security perspective, this introduces an additional burden when adding or updating third-party code as the build.rs script needs careful scrutiny.
    • Third-party build.rs scripts may not be hermetic or reproducible in potentially subtle ways. It is also common for build.rs files to access files outside the build directory (such as /usr/lib). When they are not hermetic, we would need to either carry a local patch or work with upstream to resolve the issue.
    • The most common task for build.rs is to build C libraries which Rust code depends on. We already support this through Soong.
    • Android likewise avoids running build scripts while building for other languages, instead, simply using them to inform the structure of the Android.bp file.

For instances in third-party code where a build script is used only to compile C dependencies, we either use existing cc_library Soong definitions (such as boringssl for quiche) or create new definitions for crate-specific code.

When the build.rs is used to generate source, we try to replicate the core functionality in a Soong rust_binary module for use as a custom source generator. In other cases where Soong can provide the information without source generation, we may carry a small patch that leverages this information.

Why proc_macro but not build.rs?

Why do we support proc_macros, which are compiler plug-ins that execute code on the host within the compiler context, but not build.rs scripts?

While build.rs code is written as one-off code to handle building a single crate, proc_macros define reusable functionality within the compiler which can become widely relied upon across the Rust community. As a result popular proc_macros are generally better maintained and more scrutinized upstream, which makes the code review process more manageable. They are also more readily sandboxed as part of the build process since they are less likely to have dependencies external to the compiler.

proc_macros are also a language feature rather than a method for building code. These are relied upon by source code, are unavoidable for third-party dependencies, and are useful enough to define and use within our platform code. While we can avoid build.rs by leveraging our build system, the same can’t be said of proc_macros.

There is also precedence for compiler plugin support within the Android build system. For example see Soong’s java_plugin modules.

Generated source as crates

Unlike C/C++ compilers, rustc only accepts a single source file representing an entry point to a binary or library. It expects that the source tree is structured such that all required source files can be automatically discovered. This means that generated source either needs to be placed in the source tree or provided through an include directive in source:

include!("/path/to/hello.rs");

The Rust community depends on build.rs scripts alongside assumptions about the Cargo build environment to get around this limitation. When building, the cargo command sets an OUT_DIR environment variable which build.rs scripts are expected to place generated source code in. This source can then be included via:

include!(concat!(env!("OUT_DIR"), "/hello.rs"));

This presents a challenge for Soong as outputs for each module are placed in their own out/ directory2; there is no single OUT_DIR where dependencies output their generated source.

For platform code, we prefer to package generated source into a crate that can be imported. There are a few reasons to favor this approach:

  • Prevent generated source file names from colliding.
  • Reduce boilerplate code checked-in throughout the tree and which needs to be maintained. Any boilerplate necessary to make the generated source compile into a crate can be centrally maintained.
  • Avoid implicit3 interactions between generated code and the surrounding crate.
  • Reduce pressure on memory and disk by dynamically liking commonly used generated sources.

    As a result, all of Android’s Rust source generation module types produce code that can be compiled and used as a crate.

    We still support third-party crates without modification by copying all the generated source dependencies for a module into a single per-module directory similar to Cargo. Soong then sets the OUT_DIR environment variable to that directory when compiling the module so the generated source can be found. However we discourage use of this mechanism in platform code unless absolutely necessary for the reasons described above.

    Dynamic linkage by default

    By default, the Rust ecosystem assumes that crates will be statically linked into binaries. The usual benefits of dynamic libraries are upgrades (whether for security or functionality) and decreased memory usage. Rust’s lack of a stable binary interface and usage of cross-crate information flow prevents upgrading libraries without upgrading all dependent code. Even when the same crate is used by two different programs on the system, it is unlikely to be provided by the same shared object4 due to the precision with which Rust identifies its crates. This makes Rust binaries more portable but also results in larger disk and memory footprints.

    This is problematic for Android devices where resources like memory and disk usage must be carefully managed because statically linking all crates into Rust binaries would result in excessive code duplication (especially in the standard library). However, our situation is also different from the standard host environment: we build Android using global decisions about dependencies. This means that nearly every crate is shareable between all users of that crate. Thus, we opt to link crates dynamically by default for device targets. This reduces the overall memory footprint of Rust in Android by allowing crates to be reused across multiple binaries which depend on them.

    Since this is unusual in the Rust community, not all third-party crates support dynamic compilation. Sometimes we must carry small patches while we work with upstream maintainers to add support.

    Current Status of Build Support

    We support building all output types supported by rustc (rlibs, dylibs, proc_macros, cdylibs, staticlibs, and executables). Rust modules can automatically request the appropriate crate linkage for a given dependency (rlib vs dylib). C and C++ modules can depend on Rust cdylib or staticlib producing modules the same way as they would for a C or C++ library.

    In addition to being able to build Rust code, Android’s build system also provides support for protobuf and gRPC and AIDL generated crates. First-class bindgen support makes interfacing with existing C code simple and we have support modules using cxx for tighter integration with C++ code.

    The Rust community produces great tooling for developers, such as the language server rust-analyzer. We have integrated support for rust-analyzer into the build system so that any IDE which supports it can provide code completion and goto definitions for Android modules.

    Source-based code coverage builds are supported to provide platform developers high level signals on how well their code is covered by tests. Benchmarks are supported as their own module type, leveraging the criterion crate to provide performance metrics. In order to maintain a consistent style and level of code quality, a default set of clippy lints and rustc lints are enabled by default. Additionally, HWASAN/ASAN fuzzers are supported, with the HWASAN rustc support added to upstream.

    In the near future, we plan to add documentation to source.android.com on how to define and use Rust modules in Soong. We expect Android’s support for Rust to continue evolving alongside the Rust ecosystem and hope to continue to participate in discussions around how Rust can be integrated into existing build systems.

    Thank you to Matthew Maurer, Jeff Vander Stoep, Joel Galenson, Manish Goregaokar, and Tyler Mandry for their contributions to this post.

    Notes


    1. This can be mitigated to some extent with workspaces, but requires a very specific directory arrangement that AOSP does not conform to. 

    2. This presents no problem for C/C++ and similar languages as the path to the generated source is provided directly to the compiler. 

    3. Since include! works by textual inclusion, it may reference values from the enclosing namespace, modify the namespace, or use constructs like #![foo]. These implicit interactions can be difficult to maintain. Macros should be preferred if interaction with the rest of the crate is truly required.  

    4. While libstd would usually be shareable for the same compiler revision, most other libraries would end up with several copies for Cargo-built Rust binaries, since each build would attempt to use a minimum feature set and may select different dependency versions for the library in question. Since information propagates across crate boundaries, you cannot simply produce a “most general” instance of that library. 

Correctness of code in the Android platform is a top priority for the security, stability, and quality of each Android release. Memory safety bugs in C and C++ continue to be the most-difficult-to-address source of incorrectness. We invest a great deal of effort and resources into detecting, fixing, and mitigating this class of bugs, and these efforts are effective in preventing a large number of bugs from making it into Android releases. Yet in spite of these efforts, memory safety bugs continue to be a top contributor of stability issues, and consistently represent ~70% of Android’s high severity security vulnerabilities.

In addition to ongoing and upcoming efforts to improve detection of memory bugs, we are ramping up efforts to prevent them in the first place. Memory-safe languages are the most cost-effective means for preventing memory bugs. In addition to memory-safe languages like Kotlin and Java, we’re excited to announce that the Android Open Source Project (AOSP) now supports the Rust programming language for developing the OS itself.

Systems programming

Managed languages like Java and Kotlin are the best option for Android app development. These languages are designed for ease of use, portability, and safety. The Android Runtime (ART) manages memory on behalf of the developer. The Android OS uses Java extensively, effectively protecting large portions of the Android platform from memory bugs. Unfortunately, for the lower layers of the OS, Java and Kotlin are not an option.

Lower levels of the OS require systems programming languages like C, C++, and Rust. These languages are designed with control and predictability as goals. They provide access to low level system resources and hardware. They are light on resources and have more predictable performance characteristics.

For C and C++, the developer is responsible for managing memory lifetime. Unfortunately, it’s easy to make mistakes when doing this, especially in complex and multithreaded codebases.

Rust provides memory safety guarantees by using a combination of compile-time checks to enforce object lifetime/ownership and runtime checks to ensure that memory accesses are valid. This safety is achieved while providing equivalent performance to C and C++.

The limits of sandboxing

C and C++ languages don’t provide these same safety guarantees and require robust isolation. All Android processes are sandboxed and we follow the Rule of 2 to decide if functionality necessitates additional isolation and deprivileging. The Rule of 2 is simple: given three options, developers may only select two of the following three options.

For Android, this means that if code is written in C/C++ and parses untrustworthy input, it should be contained within a tightly constrained and unprivileged sandbox. While adherence to the Rule of 2 has been effective in reducing the severity and reachability of security vulnerabilities, it does come with limitations. Sandboxing is expensive: the new processes it requires consume additional overhead and introduce latency due to IPC and additional memory usage. Sandboxing doesn’t eliminate vulnerabilities from the code and its efficacy is reduced by high bug density, allowing attackers to chain multiple vulnerabilities together.

Memory-safe languages like Rust help us overcome these limitations in two ways:

  1. Lowers the density of bugs within our code, which increases the effectiveness of our current sandboxing.
  2. Reduces our sandboxing needs, allowing introduction of new features that are both safer and lighter on resources.

But what about all that existing C++?

Of course, introducing a new programming language does nothing to address bugs in our existing C/C++ code. Even if we redirected the efforts of every software engineer on the Android team, rewriting tens of millions of lines of code is simply not feasible.

The above analysis of the age of memory safety bugs in Android (measured from when they were first introduced) demonstrates why our memory-safe language efforts are best focused on new development and not on rewriting mature C/C++ code. Most of our memory bugs occur in new or recently modified code, with about 50% being less than a year old.

The comparative rarity of older memory bugs may come as a surprise to some, but we’ve found that old code is not where we most urgently need improvement. Software bugs are found and fixed over time, so we would expect the number of bugs in code that is being maintained but not actively developed to go down over time. Just as reducing the number and density of bugs improves the effectiveness of sandboxing, it also improves the effectiveness of bug detection.

Limitations of detection

Bug detection via robust testing, sanitization, and fuzzing is crucial for improving the quality and correctness of all software, including software written in Rust. A key limitation for the most effective memory safety detection techniques is that the erroneous state must actually be triggered in instrumented code in order to be detected. Even in code bases with excellent test/fuzz coverage, this results in a lot of bugs going undetected.

Another limitation is that bug detection is scaling faster than bug fixing. In some projects, bugs that are being detected are not always getting fixed. Bug fixing is a long and costly process.

Each of these steps is costly, and missing any one of them can result in the bug going unpatched for some or all users. For complex C/C++ code bases, often there are only a handful of people capable of developing and reviewing the fix, and even with a high amount of effort spent on fixing bugs, sometimes the fixes are incorrect.

Bug detection is most effective when bugs are relatively rare and dangerous bugs can be given the urgency and priority that they merit. Our ability to reap the benefits of improvements in bug detection require that we prioritize preventing the introduction of new bugs.

Prioritizing prevention

Rust modernizes a range of other language aspects, which results in improved correctness of code:

  • Memory safety – enforces memory safety through a combination of compiler and run-time checks.
  • Data concurrency – prevents data races. The ease with which this allows users to write efficient, thread-safe code has given rise to Rust’s Fearless Concurrency slogan.
  • More expressive type system – helps prevent logical programming bugs (e.g. newtype wrappers, enum variants with contents).
  • References and variables are immutable by default – assist the developer in following the security principle of least privilege, marking a reference or variable mutable only when they actually intend it to be so. While C++ has const, it tends to be used infrequently and inconsistently. In comparison, the Rust compiler assists in avoiding stray mutability annotations by offering warnings for mutable values which are never mutated.
  • Better error handling in standard libraries – wrap potentially failing calls in Result, which causes the compiler to require that users check for failures even for functions which do not return a needed value. This protects against bugs like the Rage Against the Cage vulnerability which resulted from an unhandled error. By making it easy to propagate errors via the ? operator and optimizing Result for low overhead, Rust encourages users to write their fallible functions in the same style and receive the same protection.
  • Initialization – requires that all variables be initialized before use. Uninitialized memory vulnerabilities have historically been the root cause of 3-5% of security vulnerabilities on Android. In Android 11, we started auto initializing memory in C/C++ to reduce this problem. However, initializing to zero is not always safe, particularly for things like return values, where this could become a new source of faulty error handling. Rust requires every variable be initialized to a legal member of its type before use, avoiding the issue of unintentionally initializing to an unsafe value. Similar to Clang for C/C++, the Rust compiler is aware of the initialization requirement, and avoids any potential performance overhead of double initialization.
  • Safer integer handling – Overflow sanitization is on for Rust debug builds by default, encouraging programmers to specify a wrapping_add if they truly intend a calculation to overflow or saturating_add if they don’t. We intend to enable overflow sanitization for all builds in Android. Further, all integer type conversions are explicit casts: developers can not accidentally cast during a function call when assigning to a variable or when attempting to do arithmetic with other types.

Where we go from here

Adding a new language to the Android platform is a large undertaking. There are toolchains and dependencies that need to be maintained, test infrastructure and tooling that must be updated, and developers that need to be trained. For the past 18 months we have been adding Rust support to the Android Open Source Project, and we have a few early adopter projects that we will be sharing in the coming months. Scaling this to more of the OS is a multi-year project. Stay tuned, we will be posting more updates on this blog.

Java is a registered trademark of Oracle and/or its affiliates.

Thanks Matthew Maurer, Bram Bonne, and Lars Bergstrom for contributions to this post. Special thanks to our colleagues, Adrian Taylor for his insight into the age of memory vulnerabilities, and to Chris Palmer for his work on “The Rule of 2” and “The limits of Sandboxing”.

Google Keyboard (a.k.a Gboard) has a critical mission to provide frictionless input on Android to empower users to communicate accurately and express themselves effortlessly. In order to accomplish this mission, Gboard must also protect users’ private and sensitive data. Nothing users type is sent to Google servers. We recently launched privacy-preserving input by further advancing the latest federated technologies. In Android 11, Gboard also launched the contextual input suggestion experience by integrating on-device smarts into the user’s daily communication in a privacy-preserving way.

Before Android 11, input suggestions were surfaced to users in several different places. In Android 11, Gboard launched a consistent and coordinated approach to access contextual input suggestions. For the first time, we’ve brought Smart Replies to the keyboard suggestions – powered by system intelligence running entirely on device. The smart input suggestions are rendered with a transparent layer on top of Gboard’s suggestion strip. This structure maintains the trust boundaries between the Android platform and Gboard, meaning sensitive personal content cannot be not accessed by Gboard. The suggestions are only sent to the app after the user taps to accept them.

For instance, when a user receives the message “Have a virtual coffee at 5pm?” in Whatsapp, on-device system intelligence predicts smart text and emoji replies “Sounds great!” and “👍”. Android system intelligence can see the incoming message but Gboard cannot. In Android 11, these Smart Replies are rendered by the Android platform on Gboard’s suggestion strip as a transparent layer. The suggested reply is generated by the system intelligence. When the user taps the suggestion, Android platform sends it to the input field directly. If the user doesn’t tap the suggestion, gBoard and the app cannot see it. In this way, Android and Gboard surface the best of Google smarts whilst keeping users’ data private: none of their data goes to any app, including the keyboard, unless they’ve tapped a suggestion.

Additionally, federated learning has enabled Gboard to train intelligent input models across many devices while keeping everything individual users type on their device. Today, the emoji is as common as punctuation – and have become the way for our users to express themselves in messaging. Our users want a way to have fresh and diversified emojis to better express their thoughts in messaging apps. Recently, we launched new on-device transformer models that are fine-tuned with federated learning in Gboard, to produce more contextual emoji predictions for English, Spanish and Portuguese.

Furthermore, following the success of privacy-preserving machine learning techniques, Gboard continues to leverage federated analytics to understand how Gboard is used from decentralized data. What we’ve learned from privacy-preserving analysis has let us make better decisions in our product.

When a user shares an emoji in a conversation, their phone keeps an ongoing count of which emojis are used. Later, when the phone is idle, plugged in, and connected to WiFi, Google’s federated analytics server invites the device to join a “round” of federated analytics data computation with hundreds of other participating phones. Every device involved in one round will compute the emoji share frequency, encrypt the result and send it a federated analytics server. Although the server can’t decrypt the data individually, the final tally of total emoji counts can be decrypted when combining encrypted data across devices. The aggregated data shows that the most popular emoji is 😂 in Whatsapp, 😭 in Roblox(gaming), and ✔ in Google Docs. Emoji 😷 moved up from 119th to 42nd in terms of frequency during COVID-19.

Gboard always has a strong commitment to Google’s Privacy Principles. Gboard strives to build privacy-preserving effortless input products for users to freely express their thoughts in 900+ languages while safeguarding user data. We will keep pushing the state of the art in smart input technologies on Android while safeguarding user data. Stay tuned!

Posted by Kylie McRoberts, Program Manager and Alec Guertin, Security Engineer

Android graphic

Google’s Android Security & Privacy team has launched the Android Partner Vulnerability Initiative (APVI) to manage security issues specific to Android OEMs. The APVI is designed to drive remediation and provide transparency to users about issues we have discovered at Google that affect device models shipped by Android partners.

Another layer of security

Android incorporates industry-leading security features and every day we work with developers and device implementers to keep the Android platform and ecosystem safe. As part of that effort, we have a range of existing programs to enable security researchers to report security issues they have found. For example, you can report vulnerabilities in Android code via the Android Security Rewards Program (ASR), and vulnerabilities in popular third-party Android apps through the Google Play Security Rewards Program. Google releases ASR reports in Android Open Source Project (AOSP) based code through the Android Security Bulletins (ASB). These reports are issues that could impact all Android based devices. All Android partners must adopt ASB changes in order to declare the current month’s Android security patch level (SPL). But until recently, we didn’t have a clear way to process Google-discovered security issues outside of AOSP code that are unique to a much smaller set of specific Android OEMs. The APVI aims to close this gap, adding another layer of security for this targeted set of Android OEMs.

Improving Android OEM device security

The APVI covers Google-discovered issues that could potentially affect the security posture of an Android device or its user and is aligned to ISO/IEC 29147:2018 Information technology — Security techniques — Vulnerability disclosure recommendations. The initiative covers a wide range of issues impacting device code that is not serviced or maintained by Google (these are handled by the Android Security Bulletins).

Protecting Android users

The APVI has already processed a number of security issues, improving user protection against permissions bypasses, execution of code in the kernel, credential leaks and generation of unencrypted backups. Below are a few examples of what we’ve found, the impact and OEM remediation efforts.

Permission Bypass

In some versions of a third-party pre-installed over-the-air (OTA) update solution, a custom system service in the Android framework exposed privileged APIs directly to the OTA app. The service ran as the system user and did not require any permissions to access, instead checking for knowledge of a hardcoded password. The operations available varied across versions, but always allowed access to sensitive APIs, such as silently installing/uninstalling APKs, enabling/disabling apps and granting app permissions. This service appeared in the code base for many device builds across many OEMs, however it wasn’t always registered or exposed to apps. We’ve worked with impacted OEMs to make them aware of this security issue and provided guidance on how to remove or disable the affected code.

Credential Leak

A popular web browser pre-installed on many devices included a built-in password manager for sites visited by the user. The interface for this feature was exposed to WebView through JavaScript loaded in the context of each web page. A malicious site could have accessed the full contents of the user’s credential store. The credentials are encrypted at rest, but used a weak algorithm (DES) and a known, hardcoded key. This issue was reported to the developer and updates for the app were issued to users.

Overly-Privileged Apps

The checkUidPermission method in the PackageManagerService class was modified in the framework code for some devices to allow special permissions access to some apps. In one version, the method granted apps with the shared user ID com.google.uid.shared any permission they requested and apps signed with the same key as the com.google.android.gsf package any permission in their manifest. Another version of the modification allowed apps matching a list of package names and signatures to pass runtime permission checks even if the permission was not in their manifest. These issues have been fixed by the OEMs.

More information

Keep an eye out at https://bugs.chromium.org/p/apvi/ for future disclosures of Google-discovered security issues under this program, or find more information there on issues that have already been disclosed.

Acknowledgements: Scott Roberts, Shailesh Saini and Łukasz Siewierski, Android Security and Privacy Team


[Cross-posted from the Android Developers Blog]

As phones become faster and smarter, they play increasingly important roles in our lives, functioning as our extended memory, our connection to the world at large, and often the primary interface for communication with friends, family, and wider communities. It is only natural that as part of this evolution, we’ve come to entrust our phones with our most private information, and in many ways treat them as extensions of our digital and physical identities.

This trust is paramount to the Android Security team. The team focuses on ensuring that Android devices respect the privacy and sensitivity of user data. A fundamental aspect of this work centers around the lockscreen, which acts as the proverbial front door to our devices. After all, the lockscreen ensures that only the intended user(s) of a device can access their private data.

This blog post outlines recent improvements around how users interact with the lockscreen on Android devices and more generally with authentication. In particular, we focus on two categories of authentication that present both immense potential as well as potentially immense risk if not designed well: biometrics and environmental modalities.

The tiered authentication model

Before getting into the details of lockscreen and authentication improvements, we first want to establish some context to help relate these improvements to each other. A good way to envision these changes is to fit them into the framework of the tiered authentication model, a conceptual classification of all the different authentication modalities on Android, how they relate to each other, and how they are constrained based on this classification.

The model itself is fairly simple, classifying authentication modalities into three buckets of decreasing levels of security and commensurately increasing constraints. The primary tier is the least constrained in the sense that users only need to re-enter a primary modality under certain situations (for example, after each boot or every 72 hours) in order to use its capability. The secondary and tertiary tiers are more constrained because they cannot be set up and used without having a primary modality enrolled first and they have more constraints further restricting their capabilities.

  1. Primary Tier – Knowledge Factor: The first tier consists of modalities that rely on knowledge factors, or something the user knows, for example, a PIN, pattern, or password. Good high-entropy knowledge factors, such as complex passwords that are hard to guess, offer the highest potential guarantee of identity.

    Knowledge factors are especially useful on Android becauses devices offer hardware backed brute-force protection with exponential-backoff, meaning Android devices prevent attackers from repeatedly guessing a PIN, pattern, or password by having hardware backed timeouts after every 5 incorrect attempts. Knowledge factors also confer additional benefits to all users that use them, such as File Based Encryption (FBE) and encrypted device backup.

  1. Secondary Tier – Biometrics: The second tier consists primarily of biometrics, or something the user is. Face or fingerprint based authentications are examples of secondary authentication modalities. Biometrics offer a more convenient but potentially less secure way of confirming your identity with a device.

We will delve into Android biometrics in the next section.

  1. The Tertiary Tier – Environmental: The last tier includes modalities that rely on something the user has. This could either be a physical token, such as with Smart Lock’s Trusted Devices where a phone can be unlocked when paired with a safelisted bluetooth device. Or it could be something inherent to the physical environment around the device, such as with Smart Lock’s Trusted Places where a phone can be unlocked when it is taken to a safelisted location.

    Improvements to tertiary authentication

    While both Trusted Places and Trusted Devices (and tertiary modalities in general) offer convenient ways to get access to the contents of your device, the fundamental issue they share is that they are ultimately a poor proxy for user identity. For example, an attacker could unlock a misplaced phone that uses Trusted Place simply by driving it past the user’s home, or with moderate amount of effort, spoofing a GPS signal using off-the-shelf Software Defined Radios and some mild scripting. Similarly with Trusted Device, access to a safelisted bluetooth device also gives access to all data on the user’s phone.

    Because of this, a major improvement has been made to the environmental tier in Android 10. The Tertiary tier was switched from an active unlock mechanism into an extending unlock mechanism instead. In this new mode, a tertiary tier modality can no longer unlock a locked device. Instead, if the device is first unlocked using either a primary or secondary modality, it can continue to keep it in the unlocked state for a maximum of four hours.

A closer look at Android biometrics

Biometric implementations come with a wide variety of security characteristics, so we rely on the following two key factors to determine the security of a particular implementation:

  1. Architectural security: The resilience of a biometric pipeline against kernel or platform compromise. A pipeline is considered secure if kernel and platform compromises don’t grant the ability to either read raw biometric data, or inject synthetic data into the pipeline to influence an authentication decision.
  2. Spoofability: Is measured using the Spoof Acceptance Rate (SAR). SAR is a metric first introduced in Android P, and is intended to measure how resilient a biometric is against a dedicated attacker. Read more about SAR and its measurement in Measuring Biometric Unlock Security.

We use these two factors to classify biometrics into one of three different classes in decreasing order of security:

  • Class 3 (formerly Strong)
  • Class 2 (formerly Weak)
  • Class 1 (formerly Convenience)

Each class comes with an associated set of constraints that aim to balance their ease of use with the level of security they offer.

These constraints reflect the length of time before a biometric falls back to primary authentication, and the allowed application integration. For example, a Class 3 biometric enjoys the longest timeouts and offers all integration options for apps, while a Class 1 biometric has the shortest timeouts and no options for app integration. You can see a summary of the details in the table below, or the full details in the Android Android Compatibility Definition Document (CDD).

1 App integration means exposing an API to apps (e.g., via integration with BiometricPrompt/BiometricManager, androidx.biometric, or FIDO2 APIs)

2 Keystore integration means integrating Keystore, e.g., to release app auth-bound keys

Benefits and caveats

Biometrics provide convenience to users while maintaining a high level of security. Because users need to set up a primary authentication modality in order to use biometrics, it helps boost the lockscreen adoption (we see an average of 20% higher lockscreen adoption on devices that offer biometrics versus those that do not). This allows more users to benefit from the security features that the lockscreen provides: gates unauthorized access to sensitive user data and also confers other advantages of a primary authentication modality to these users, such as encrypted backups. Finally, biometrics also help reduce shoulder surfing attacks in which an attacker tries to reproduce a PIN, pattern, or password after observing a user entering the credential.

However, it is important that users understand the trade-offs involved with the use of biometrics. Primary among these is that no biometric system is foolproof. This is true not just on Android, but across all operating systems, form-factors, and technologies. For example, a face biometric implementation might be fooled by family members who resemble the user or a 3D mask of the user. A fingerprint biometric implementation could potentially be bypassed by a spoof made from latent fingerprints of the user. Although anti-spoofing or Presentation Attack Detection (PAD) technologies have been actively developed to mitigate such spoofing attacks, they are mitigations, not preventions.

One effort that Android has made to mitigate the potential risk of using biometrics is the lockdown mode introduced in Android P. Android users can use this feature to temporarily disable biometrics, together with Smart Lock (for example, Trusted Places and Trusted Devices) as well as notifications on the lock screen, when they feel the need to do so.

To use the lockdown mode, users first need to set up a primary authentication modality and then enable it in settings. The exact setting where the lockdown mode can be enabled varies by device models, and on a Google Pixel 4 device it is under Settings > Display > Lock screen > Show lockdown option. Once enabled, users can trigger the lockdown mode by holding the power button and then clicking the Lockdown icon on the power menu. A device in lockdown mode will return to the non-lockdown state after a primary authentication modality (such as a PIN, pattern, or password) is used to unlock the device.

BiometricPrompt – New APIs

In order for developers to benefit from the security guarantee provided by Android biometrics and to easily integrate biometric authentication into their apps to better protect sensitive user data, we introduced the BiometricPrompt APIs in Android P.

There are several benefits of using the BiometricPrompt APIs. Most importantly, these APIs allow app developers to target biometrics in a modality-agnostic way across different Android devices (that is, BiometricPrompt can be used as a single integration point for various biometric modalities supported on devices), while controlling the security guarantees that the authentication needs to provide (such as requiring Class 3 or Class 2 biometrics, with device credential as a fallback). In this way, it helps protect app data with a second layer of defenses (in addition to the lockscreen) and in turn respects the sensitivity of user data. Furthermore, BiometricPrompt provides a persistent UI with customization options for certain information (for example, title and description), offering a consistent user experience across biometric modalities and across Android devices.

As shown in the following architecture diagram, apps can integrate with biometrics on Android devices through either the framework API or the support library (that is, androidx.biometric for backward compatibility). One thing to note is that FingerprintManager is deprecated because developers are encouraged to migrate to BiometricPrompt for modality-agnostic authentications.

Improvements to BiometricPrompt

Android 10 introduced the BiometricManager class that developers can use to query the availability of biometric authentication and included fingerprint and face authentication integration for BiometricPrompt.

In Android 11, we introduce new features such as the BiometricManager.Authenticators interface which allows developers to specify the authentication types accepted by their apps, as well as additional support for auth-per-use keys within the BiometricPrompt class.

More details can be found in the Android 11 preview and Android Biometrics documentation. Read more about BiometricPrompt API usage in our blog post Using BiometricPrompt with CryptoObject: How and Why and our codelab Login with Biometrics on Android.

Trust is very important when it comes to the relationship between a user and their smartphone. While phone functionality and design can enhance the user experience, security is fundamental and foundational to our relationship with our phones.There are multiple ways to build trust around the security capabilities that a device provides and we continue to invest in verifiable ways to do just that.

Pixel 4a ioXt certification

Today we are happy to announce that the Pixel 4/4 XL and the newly launched Pixel 4a are the first Android smartphones to go through ioXt certification against the Android Profile.

The Internet of Secure Things Alliance (ioXt) manages a security compliance assessment program for connected devices. ioXt has over 200 members across various industries, including Google, Amazon, Facebook, T-Mobile, Comcast, Zigbee Alliance, Z-Wave Alliance, Legrand, Resideo, Schneider Electric, and many others. With so many companies involved, ioXt covers a wide range of device types, including smart lighting, smart speakers, webcams, and Android smartphones.

The core focus of ioXt is “to set security standards that bring security, upgradability and transparency to the market and directly into the hands of consumers.” This is accomplished by assessing devices against a baseline set of requirements and relying on publicly available evidence. The goal of ioXt’s approach is to enable users, enterprises, regulators, and other stakeholders to understand the security in connected products to drive better awareness towards how these products are protecting the security and privacy of users.

ioXt’s baseline security requirements are tailored for product classes, and the ioXt Android Profile enables smartphone manufacturers to differentiate security capabilities, including biometric authentication strength, security update frequency, length of security support lifetime commitment, vulnerability disclosure program quality, and preloaded app risk minimization.

We believe that using a widely known industry consortium standard for Pixel certification provides increased trust in the security claims we make to our users. NCC Group has published an audit report that can be downloaded here. The report documents the evaluation of Pixel 4/4 XL and Pixel 4a against the ioXt Android Profile.

Security by Default is one of the most important criteria used in the ioXt Android profile. Security by Default rates devices by cumulatively scoring the risk for all preloads on a particular device. For this particular measurement, we worked with a team of university experts from the University of Cambridge, University of Strathclyde, and Johannes Kepler University in Linz to create a formula that considers the risk of platform signed apps, pregranted permissions on preloaded apps, and apps communicating using cleartext traffic.

Screenshot of the presentation of the Android Device Security Database at the Android Security Symposium 2020

In partnership with those teams, Google created Uraniborg, an open source tool that collects necessary attributes from the device and runs it through this formula to come up with a raw score. NCC Group leveraged Uraniborg to conduct the assessment for the ioXt Security by Default category.

As part of our ongoing certification efforts, we look forward to submitting future Pixel smartphones through the ioXt standard, and we encourage the Android device ecosystem to participate in similar transparency efforts for their devices.

Acknowledgements: This post leveraged contributions from Sudhi Herle, Billy Lau and Sam Schumacher