Linux kernel: unlinked mountpoint dentry stays findable, letting lookups read covered files

In the Linux kernel, the following vulnerability has been resolved:

namespace: unhash a dentry before detaching the mounts on it

unlink(), rmdir() and rename() remove the entry from the filesystem, call detach_mounts() on the entry with the inode locked and then call d_delete() once the inode is unlocked.

The thing is that between detach_mounts() and d_delete() the dentry is still hashed and positive. But after detach_mounts() nothing covers the dentry anymore. A lookup that finds this dentry in the dcache can uncover what the mounts hid.

That's a problem when unlinking files or directories that are mountpoints in other mount namespaces. Everybody who had the underlying entry covered can race the detach_mount() call until d_delete() has run.

The race window isn't all that small. It encompasses namespace_unlock() with a full synchronize_rcu_expedited() grace period and the inode_unlock() of the entry.

In my experiments three walkers that kept trying read an overmounted file in 11938 of the 16140 unlinks that removed its mountpoint from a bind mount of the filesystem within a minute.

Fun fact, d_invalidate() has the same ordering problem but gets it right. It unhashes the dentry first and detaches the mounts afterwards. Let's do the same in __detach_mounts():

• A lookup that hasn't found the dentry yet misses it in the dcache and waits for the directory lock that the caller holds until the name is gone for good.

• A lockless lookup that found it already fails the mount_lock check while the dentry still counts as a mountpoint and the d_seq check once it doesn't, and retries.

This fixes the lockless path. We still need to fix the reference count lookup in a follow-up patch.

So d_drop() the dentry. The dentry stays positive and held but can't be found anymore. Then proceed with the detach and unlink.

Reproducer:

The reads were counted with the program below, three walkers for 60 s in a VM with 4 CPUs. The race window is stretched with the debug patch pasted here. fs.detach_race_walk_us sleeps in lookup_fast() once it found a mountpoint dentry and before its mounts are crossed. fs.detach_race_unlink_us sleeps in vfs_unlink() between detach_mounts() and d_delete().

// SPDX-License-Identifier: GPL-2.0 /*

• unlink_covered: unlink a file that is a mountpoint in a detached copy of
• its mount, against lookups that open it through that copy.
• A is a tmpfs with the file f1 (content SECRET_F) and the plain file
• MARK_A. A' is an open_tree(OPEN_TREE_CLONE) copy of A with MARK_A bound
• on A'/f1, held through an O_PATH fd on its root once the tree fd is
• closed. Walkers open f1 through that fd while the driver unlinks A/f1,
• where nothing is mounted on it. A walker may read MARK_A or get ENOENT.
• A read of SECRET_F is a hit: the name was found after its mount was
• gone. The lockless walks use openat2(RESOLVE_CACHED).
• usage: unlink_covered [-t seconds] [-w walkers] */ #ifndef _GNU_SOURCE #define _GNU_SOURCE #endif #include <errno.h> #include <fcntl.h> #include <pthread.h> #include <stdatomic.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <sys/mount.h> #include <sys/stat.h> #include <sys/syscall.h> #include <linux/openat2.h>

#ifndef __NR_open_tree #define __NR_open_tree 428 #endif #ifndef __NR_move_mount #define __NR_move_mount 429 #endif #ifndef __NR_openat2 #define __NR_openat2 437 #endif #ifndef OPEN_TREE_CLONE #define OPEN_TREE_CLONE 1 #endif #ifndef OPEN_TREE_CLOEXEC #define OPEN_TREE_CLOEXEC O_CLOEXEC #endif #ifndef MOVE_MOUNT_F_EMPTY_PATH #define MOVE_MOUNT_F_EMPTY_PATH 0x00000004 #endif

#define WORK "/tmp/uc" #define ADIR WORK "/A"

static int duration = 60, nwalkers = 3; static atomic_int stop, writer_waiting; static pthread_rwlock_t cur_lock = PTHREAD_RWLOCK_INITIALIZER; static int cur_fd = -1; /* the root of A', -1 wh ---truncated---