@javierm on this too; please @ others as you see fit.

I would assume that user data under /etc should be pretty minimal. User data under /usr and /srv should be nonexistent. And our /tmp is tmpfs, so that's covered.

Guess: only /var is a real problem?

Guess: only /var is a real problem?

I have the same guess. Is ~/.var useful for this? Should it be preferred whenever possible?

Adding @vponcova , she is SME for storage code in Anaconda.

I have the same guess. Is ~/.var useful for this? Should it be preferred whenever possible?

Well ~/.var is a flatpak invention, so afaik it's not used by anything other than flatpak currently, but certainly it's intended to be usable for other things or it would have been named differently. (Just stay out of ~/.var/app if you're not flatpak.)

Guess: only /var is a real problem?

Maybe also /run. I see we are already using tmpfs for /run/user/1000, but the stuff in toplevel /run looks sensitive too. afaik Arch uses tmpfs for all of /run.

I have no clue what to do about /var.

Fedora uses a separate tmpfs for /run, and for each /run/user/* which are also a reduced size.

We could ask for a security risk assessment from some experts? I'm curious about the relative risk of leaks into /var, /etc, and swap. But for all I know security experts would just say, well you aren't encrypting them now by default, so each of those you do either encrypt or put on volatile media is an improvement.

Fedora uses a separate tmpfs for /run, and for each /run/user/* which are also a reduced size.

I see. I ran mount to check, but misread the output.

We could ask for a security risk assessment from some experts? I'm curious about the relative risk of leaks into /var, /etc, and swap. But for all I know security experts would just say, well you aren't encrypting them now by default, so each of those you do either encrypt or put on volatile media is an improvement.

Well if we get rid of disk-based swap by default, then swap definitely does not matter.

I think the problem with unencrypted /etc would be integrity, rather than confidentiality. E.g. an attacker could corrupt a systemd unit under /etc/systemd to cause it to execute something naughty.

/var is full of sensitive system logs, and confidentiality seems like a serious problem there.

I think the problem with unencrypted /etc would be integrity, rather than confidentiality

Encryption does not provide integrity per se; and provides no detection of corruption. Full volume encryption providing confidentiality is what makes it impossible to perform the specific modifications to insert malware; but injecting corruption and producing unpredictable consequences is still possible.

The problem with /etc is trust. The act of unlocking (decrypting) it requires authentication, and should mean anything written to the unlocked volume via kernel code can be trusted; but could still be corrupted. A checksum regime is really needed to fully trust it.

Re: A random key, sealed by the TPM, to encrypt sysroot:

While sysroot is unlocked merely by booting the system, it makes it difficult for an attacker to inject malware. It shouldn't be possible for them to break out of graphical or console, whether sysroot is encrypted. And they wouldn't be able to boot from external media and gain access to sysroot.

Difficulties remain:

• No TPM support in the installer, and no time frame for it.
• Quite a lot of systems don't have them.
• Key management. If sysroot is going to be encrypted or integrity verifiable, there needs to be some way of managing its key(s): sealed in TPM, or maybe in encrypted ~/.
• Motherboard/TPM dies, keys are lost.
• Double encryption makes it easier to lose data on ~/ (due to lost key).

Needs critique and research:

• What about booting a read-only sysroot? User login unlocks ~/, revealing a sysroot key, enabling unlock and mount of the real /etc and /var. (Yes, this has its own warts, certainly at least networking related.)
• That sysroot key can be used for the real /etc and /var, which are encrypted. A separate key is used for encrypting ~/.
• IMA
• fs-verity, optional IMA
• authenticated filesystem, HMAC(SHA-256) Btrfs

And they wouldn't be able to boot from external media and gain access to sysroot.

Why not?

Quite a lot of systems don't have them.

Do new systems have them?

And they wouldn't be able to boot from external media and gain access to sysroot.

Why not?

PCR measurement won't match, the encryption key won't be unsealed, and thus sysroot can't be unlocked.

Quite a lot of systems don't have them.

Do new systems have them?

Mixed.

@pjones @walters
@catanzaro writes in #82, related to system encryption:

TPM availability uncertain

I wonder about this too, but have found it difficult to qualify. Many manufacturer specs don't list it. Windows 10 hardware certification requires a TPM 2.0 to be present and enabled. Yet there's common consumer hardware with TPM that isn't supported by the kernel. Supporting Macs generally is getting harder, but the older ones don't have a TPM, and the latest ones have a T2 chip which is not a TPM substitute.

integrity protection (for the system)

Dilemma: Protect against targeted attacks like the evil maid installing a keylogger, versus dislike of two passphrases (sysroot and login) or depending on TPM being present.

This paper discusses IMA, dm-verity, fs-verity, and authenticated Btrfs (proposed).

Also relevant is ostree. I'm not sure to what degree rpm-ostree/flatpaks provide integrity; checksum only or also signature verficiation; on-the-fly during all reads or on-demand; does it include layered packages and configuration files?

Also relevant is ostree. I'm not sure to what degree rpm-ostree/flatpaks provide integrity;

https://github.com/ostreedev/ostree/issues/1997

checksum only or also signature verficiation;

You can do that, but if someone has compromised your OS, then you can't trust the thing verifying the signature.

Dilemma: Protect against targeted attacks like the evil maid installing a keylogger,

See https://github.com/coreos/rpm-ostree/issues/1883#issuecomment-537629143 for an interesting link on this topic. The TL;DR is mjg59 is thinking about how to use one's phone to verify a TPM-based attestation state from a laptop for example.

Upstream dm-crypt/cryptsetup folks are working on TPM 2.0 and yubikey support, but there's no ETA. That seems to be a prerequisite for installer support.

Re: "encrypt my data" checkbox (full disk encryption, minus /boot) in the automatic/default path of the installer

• A guest, James, brought up at today's meeting that once this is checked, there is no recourse for recovery (if the passphrase is forgotten or the encoding of the passphrase somehow changes).
• I note from dm-crypt list the recovery issue comes up regularly; and periodically it does result in complete data loss, and as a result the developers are sensitive about any/all changes that can make this worse.
• This includes their very strong recommendation that user passphrases only use printable ASCII.
• LUKS supports multiple keyslots. It's technically possible for the installer to automatically and always create a recovery key. If the user doesn't want one, they can merely not save/screenshot the recovery key, and then it's effectively lost forever.

RFE: Include TPM version info in LVFS
RFE: always create a recovery key when enabling disk encryption

Update, regarding Anaconda's ability to set a recovery password:

From the Anaconda RFE: We currently support creating a backup passphrase using volume key -- https://pagure.io/volume_key -- but only in kickstart using the --backuppassphrase and --escrowcert options. Backup keys are then stored in the /root directory encrypted by the provided certificate.

Assume a short term decision to encrypt by default sooner than later. Do this by enabling the "encrypt my data" option in Anaconda by default. (Reminder: this uses LUKS encryption for all of sysroot and home, and uses the plymouth workflow.) The following questions arise:

• should a recovery key be set by default?
• should the recovery key be displayed to the user in the installer? Or could this be done by g-i-s?
• does 'volume key' help? i'm not sure what work is implied by integrating it, or some other implementation, into either the installer or g-i-s?
• is it adequate to only document how to set and test a recovery key, using cryptsetup?

Thoughts?

At the next meeting: Discuss a short term encryption plan. What will it take to get to yes? What mitigations are necessary for the working group to agree to enable "encrypt my data" in the installer by default? Once we define and agree to those mitigations, it'll be possible to estimate whether and when they can be implemented.

Attempt to summarize this issue:

Problem is that, once we have encrypted the user's home directory in #82, if we don't encrypt / as well the user will be vulnerable to various data leakage and evil maid attacks. Considering the hard requirement to not have a plymouth password prompt and therefore not use LUKS full-disk encryption, and the additional requirement to avoid double-encrypting user data, this is a tricky problem. I think the closest we have to a reasonable plan is https://pagure.io/fedora-workstation/issue/82#comment-642602 (posted in #82).

It seems that working group might be revisiting the decision to enable full disk encryption. With that in mind, I've compiled a list of what I see as being the main issues with it. This could also serve as a useful reference when evaluating other options.

Full disk encryption issues

1. It requires an extra password:
   • Users have to remember two passwords rather than one. People don't like passwords and they're a burden. If you have more than one, the chance of failure (ie. forgetting one) dramatically increases.
   • Awkward relationship with the user session password - which is exposed.
2. Lack of password recovery - a major risk with disk encryption is someone losing the password and being locked out
3. Implications for the boot experience
   • Users have to enter two passwords, or enable autologin. Autologin has its own issues: no opportunity to select a user or session type, possibly a rough boot progress experience.
   • Problems with password entry during early boot:
     • No input methods (difficult to even indicate the current keyboard layout)
     • No accessibility tools
     • Input devices not always available (eg. USB, Thunderbolt)
   • Impacts on boot UX:
     • The current UX goal is to get the user to a nice personalised experience (pretty background, avatar, etc) as quickly and seamlessly as possible. Various work has been done in this area recently, such a skipping the boot menu when possible. More work is planned for F33. Encryption password is a backward step in this regard.
     • What's the impact of autologin on progress transitions from boot to the user session? Needs to be accounted for.
     • Requiring users to operate in an environment that isn't a full desktop session is not nice.
4. Implications for multiuser systems - is it practical to have full disk encryption on a multiuser system? If not, how are users expected to know?
5. Implications for OEM installations - if encryption is important for user-installed systems, that would imply that it's important for preinstalled systems too.
6. Desktop integration. The desktop ought to report disk encryption in the proper place, and provide appropriate guidance as needed. Examples of places where this might be needed: system information panel in settings, user settings (when adding users, changing passwords).

(Moving this system specific discussion over from home encryption.)
@lennart

Again, keep /var a separate discussion, bind to the TPM, as soon as we have support for that.

A few challenges:

• In issue #54, the competition for free space, mainly between /var and /home, is a regular problem that needs solving. It can be solved with the "one big file system" layout. Except, as soon as /var must be LUKS encrypted it means partitioning [1], and we're back to square one. Ideas? Maybe /var could use fscrypt since none of the files themselves are secrets, just their contents are?
• Workstation is currently supported on Macs, aarch64, and non-Windows 10 certified hardware, and they don't tend to have TPMs. And some TPM 2.0 chips aren't supported by the kernel. What's the fallback plan? Yubikey? Plymouth: two passphrases, maintenance?
• The idea of encrypting /var with /home is consistent with (a) factory reset/stateless systems where there'd be a generic /etc /var, and also (b) with CoreOS/rpm-ostree/Silverblue systems where /var/home is bind mounted to /home. All that's needed is a way to get to login window faster, and delay (or restart) services that depend on a custom /var. The main idea is to get to a GNOME stack faster, as a way to supplant plymouth as the interface for unlocking things.

Is a generic/stateless /var really a dead end idea? If it is, about the best way forward for system encryption is fscrypt, near as I can tell.

[1] I mean it in a generic sense. It could be partitions, LVM LVs, or yes even loop mounted files, treating var similarly to the per user home --storage=luks concept.

(Moving this system specific discussion over from home encryption.)
In issue #54, the competition for free space, mainly between /var and /home, is a regular problem that needs solving. It can be solved with the "one big file system" layout. Except, as soon as /var must be LUKS encrypted it means partitioning [1], and we're back to square one. Ideas? Maybe /var could use fscrypt since none of the files themselves are secrets, just their contents are?

Using fscrypt for /var seems reasonable?

Alternatively, we could go all-in on user flatpaks and have gnome-software just entirely stop installing system flatpaks.

Workstation is currently supported on Macs, aarch64, and non-Windows 10 certified hardware, and they don't tend to have TPMs. And some TPM 2.0 chips aren't supported by the kernel. What's the fallback plan? Yubikey? Plymouth: two passphrases, maintenance?

I wonder.

Naive fallback proposal: unencrypted. Our goal is to provide good defaults for most users, not to implement something for every possible scenario. This would suffice if we expect TPM 2.0 to be common on a large majority of new x86_64 hardware.

But I wonder. I don't think my desktop has a TPM, and I built it just two years ago. How would I know? If it doesn't come as part of the motherboard, how many desktops are really going to have them? We target desktops too, not just laptops....

Is a generic/stateless /var really a dead end idea? If it is, about the best way forward for system encryption is fscrypt, near as I can tell.

It seems very difficult. I wouldn't want to be responsible for making everything that needs /var not use it until after the user has logged in. journald?

Using fscrypt for /var seems reasonable?
Alternatively, we could go all-in on user flatpaks and have gnome-software just entirely stop installing system flatpaks.

Maybe. I'm concerned about app duplication wasting space. Perhaps clarification is needed:

• "general purpose, shared, non-confidential" apps that would be system flatpaks, such as the default flatpaks in Silverblue. Encrypting them isn't necessary. And surely it's not desirable to duplicate them per user. I have only a few flatpaks installed and /var/lib/flatpak is 5.3G.
• "user specific, not shared, confidential" apps that are user flatpaks. Encryption might be indicated.

If that sounds correct, then yes, go all-in on user flatpaks. For what it's worth, if /home is shared (no separate LUKS per user), and if it's Btrfs or XFS, it's possible to optimize space consumption using reflinks or snapshots of flatpaks. Such optimization is slightly off topic here, and more related to #54 and a separate flatpak/Software discussion.

Two fscrypt notes: It's good enough confidentiality from casual snoopers, but due to the significant metadata leakage, it's certain that flatpaks can be identified by a sophisticated attacker merely by file sizes and date stamps. If apps need to be confidential, they need to be on a LUKS volume. Second, fscrypt isn't supported by the installer yet, and it implies a TPM/yubikey sealed key. So no matter what it's a bit down the road.

Naive fallback proposal: unencrypted. Our goal is to provide good defaults for most users, not to implement something for every possible scenario. This would suffice if we expect TPM 2.0 to be common on a large majority of new x86_64 hardware.

Yep. Seeing as we have time before such TPM/yubikey support appears, there's time to investigate generic solutions that don't depend on either; but also to assess the prevalence of TPM 2.0 and the receptiveness of users to adding on a yubikey instead if they want better security (or opt into a plymouth two passphrase regime).

Is a generic/stateless /var really a dead end idea? If it is, about the best way forward for system encryption is fscrypt, near as I can tell.

It seems very difficult. I wouldn't want to be responsible for making everything that needs /var not use it until after the user has logged in. journald?

journald already starts early boot using /run/log/journal, and then switches and flushes once /var/log/journal becomes available. So just delay that? A massive journal with a ton of debugging enabled might hit 2MB.

The stateless system, factory reset concept is the idea of other people, including @lennart, Android, Chromium OS, CoreOS (then and now), iOS, and even Windows has an explicit reset option. I read that blog entry back in 2014, and skimmed it again just a couple weeks ago and it suggests a generic /var and /etc. Not my idea! :D

My expectation is quite a lot of services that depend on custom/local information would have their startup delayed, expressly to get to a login window quickly (and thus obviate plymouth and all those issues and limitations). If this idea is really not workable, OK fine.

For what it's worth, the implied UX of delayed startup following login exists already in the real world. This is how macOS behaves with full disk encryption enabled, since 2011. Initially, it did surprise users to see a nearly instant login window, followed by a noticeable delay before the desktop appears. They got over it.

In issue #54, the competition for free space, mainly between /var and /home, is a regular problem that needs solving. It can be solved with the "one big file system" layout. Except, as soon as /var must be LUKS encrypted it means partitioning [1], and we're back to square one. Ideas? Maybe /var could use fscrypt since none of the files themselves are secrets, just their contents are?

Is this really such a big issue? disks aren#t tiny anymore. At least I personally always ran stuff with /var split out and never felt the need to resize /var. I mean, aren't you sure you are making something big of something small?

Workstation is currently supported on Macs, aarch64, and non-Windows 10 certified hardware, and they don't tend to have TPMs. And some TPM 2.0 chips aren't supported by the kernel. What's the fallback plan? Yubikey? Plymouth: two passphrases, maintenance?

Maybe just not encrypt there, or only with user provided pw. i.e. when tpm is around default to encryption (maybe with opt-out), and if no tpm is around default to no encryption (with opt-in by user pw)

As I mentioned many times before: fscrypt is really not a good choice for dir trees with a lot of metadata (i.e. with many small files), because it leaks it all and allows removal of files without having crypto key. /var as a whole carries a ton of metadata, it's a typical UNIX tree with gazillion of things in them. It sucks using fscrypt for something like that. I mean, sure, stuff in /var is typically less sensitive than data in /home, but if one argues like that then maybe there's no point in encrypting /var at all...

ChromeOS and many other devices just allocate a fixed size to the vendor supplied OS image and to the local data, and sticks to it. Not sure why we should not be able to do that too.

But I wonder. I don't think my desktop has a TPM, and I built it just two years ago. How would I know? If it doesn't come as part of the motherboard, how many desktops are really going to have them? We target desktops too, not just laptops....

Again, Windows 8.1 and later require a TPM2 to pass validation. Any reasonable modern laptop that is supposed to run Windows hence has TPM. Just check if you have /dev/tpm. (If you don#t check if you might have turned it off in the firmware setup, it defaults to on there)

the way fscrypt works so far is that you have to upload the decryption key into the kernel keyring for your process, session or user. That makes it nasty to use for a system-wide data store such as /var/ since there's just no keyring for the "system" (you'd basically have to upload the key again and again for each service, each user logging in and so to ensure everyone has it). It's a deficiency of the fscrypt design (or the keyring design, if you prefer). There has been work on making the keyring hookup of fscrypt optional, but I don#t think this is available in released kernels yet (maybe not even unreleased ones). But that makes fscrypt a bad choice for /var right now. (in contrast to the leaking-a-ton-of-metadata thing it's an easier thing to fix though)

Yep. Seeing as we have time before such TPM/yubikey support appears, there's time to investigate generic solutions that don't depend on either; but also to assess the prevalence of TPM 2.0 and the receptiveness of users to adding on a yubikey instead if they want better security (or opt into a plymouth two passphrase regime).

yubikey support in /etc/crypttab is already available (hook-up via pkcs#11 already in fedora, alternative hook-up via fido2 is done but not merged yet, and thus not in fedora yet).

tpm2 support i just started working on, doesn't exist outside of my laptop yet

Is this really such a big issue? disks aren#t tiny anymore. At least I personally always ran stuff with /var split out and never felt the need to resize /var. I mean, aren't you sure you are making something big of something small?

It's a very big issue. We default to installing flatpaks under /, but I have placed them all under my ~/home to avoid running out of space in /. I'm still out of space. With our current default 75 GB /, it's just not big enough to run the mockbuild that I am trying to run, even if I delete /var/lib/mock before starting the build (that directory grows huge). If we keep split /home and / by default then we should double the size of / IMO, to seriously reduce the chances of this happening. Say you have a 200 GB disk, then that would be 150 GB / and 50 GB /home. That's just not good. Conclusion: shared space allocation is a requirement.

Maybe just not encrypt there, or only with user provided pw. i.e. when tpm is around default to encryption (maybe with opt-out), and if no tpm is around default to no encryption (with opt-in by user pw)

I want to agree, but...

But I wonder. I don't think my desktop has a TPM, and I built it just two years ago. How would I know? If it doesn't come as part of the motherboard, how many desktops are really going to have them? We target desktops too, not just laptops....

Again, Windows 8.1 and later require a TPM2 to pass validation. Any reasonable modern laptop that is supposed to run Windows hence has TPM. Just check if you have /dev/tpm. (If you don#t check if you might have turned it off in the firmware setup, it defaults to on there)

I do not have /dev/tpm. This is a desktop workstation. My motherboard is https://www.asrock.com/MB/AMD/X399%20Taichi/index.asp which I researched extensively before purchasing to ensure I was buying a high-quality product. So I'm not so sure we can assume TPM after all. Why would any company bother to manufacture motherboards that won't pass Windows 8 validation?

I can poke around in my UEFI firmware options, but they are very complex, and I rather doubt I'll find it there.

I do not have /dev/tpm. This is a desktop workstation. My motherboard is https://www.asrock.com/MB/AMD/X399%20Taichi/index.asp which I researched extensively before purchasing to ensure I was buying a high-quality product. So I'm not so sure we can assume TPM after all. Why would any company bother to manufacture motherboards that won't pass Windows 8 validation?

I have a similar one (asrock z390 Taichi) and I had to enable platform security in the firmware to get it. I think this ties into Intel ME though. They seem to sell a separate discrete version. Presumably any OEMS using Asrock boards enable this option so Windows can use it (or they include a discrete TPM).

BTW, I'm OK with having higher security level when a TPM is available. E.g. we might encrypt / only if TPM is present. But we don't need TPM to encrypt /home, right?

@lennart
Whether old or new, Apple laptops do not have a TPM. What's the fallback plan?

Bug 185631 - Hardware has TPM 2.0, but /dev/tpm does not exist Common hardware, HP, came with Windows 10, and Windows 10 sees and uses the TPM, but it fails in all linux kernels, even after the TPM is reset in firmware setup, and with the latest firmware updates from HP. No fix forthcoming. I can't estimate what percent of Fedora users would/could be impacted by the kernel just not supporting an entire make/model of TPM, but there are other bugs in this category on lkml and kernel.org. That's why I think TPM availability might be overstated. It's not enough to be in the laptop, it has to be supported by the kernel. I'm also not certain whether all TPMs can sanely support dual usage, i.e. can Windows and Fedora co-exist and share one TPM reliably, especially without data loss?

A fallback might be "get a yubikey". Is it fair to say: you need either a linux supported TPM 2.0, or a yubikey in order to have full disk encryption? Or is there yet another fallback, a user password entered at plymouth?

@lennart

Again, Windows 8.1 and later require a TPM2 to pass validation.

Windows 8.1 Hardware Compatibility Specification, required a TPM 1.2. Windows 10 Hardware Compatibility Specification requires TPM 2.0. Since the certification costs companies money, as it's part of a Microsoft marketing program, where the vendor get to use Microsoft Windows logos and stuff like that, there's actually quite a lot of hardware out there that is not certified, and therefore doesn't have to have a TPM. I have no idea how common TPMs really are. We should maybe do a survey?

Oh and my two and a half year old Intel NUC, has no TPM of either version. It did not come with Windows, did not claim to support Windows, so I figure they didn't pay for the "made for Windows" fee to Microsoft and therefore didn't have to comply with the Windows hardware certification.

So I'm 0 for 3. I'm not a scientific sample though.

Came across this https://dolosgroup.io/blog/2021/7/9/from-stolen-laptop-to-inside-the-company-network, and a good summary from Matthew Garrett @mjg59

https://twitter.com/mjg59/status/1420480679523930116

having the entire FDE key automatically sent over an unencrypted bus is a problem

If I understand correctly, mitigation options are: require a disk encryption password perhaps even just a numeric PIN, or Use TPM parameter encryption so the FDE key isn't sent in the clear.

The first option is an i18n and a11y problem: it's a difficult environment to present even a UI for numeric PIN entry. We'd need both audio and visual UIs to prompt the user, correct? And the system should power itself off after some timeout period. I'm not sure whether plymouth can do that today? @jwrdegoede

The second option, I can't assess what's involved in making that happen; maybe systemd's most recent TPM support already implements such a thing? @lennart

Summary from today's meeting: will probably want to do btrfs fscrypt if and only if a TPM is available. Owen and Neal to investigate further and prepare a document exploring various considerations.

@otaylor @ngompa can you provide a status update please?

The most recent patch series was submitted last month by @josef. I believe it'll land pretty soon, but he can provide more details.