libcephfs: return the change_attr in the statx.stx_version field

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/client: bump the change_attr at the appropriate time for files

The semantics for a change_attr are that it should be incremented
whenever there is a change to the ctime in the inode. Add those
increments for the simple case of regular files. Directories however can
be fragmented so we'll need to do something more elaborate there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL

Suppose we have two clients. client1 holds FILE_EXCL cap and client2
holds AUTH_EXCL.  Both have the change_attr at the same value (call it
1).  client1 does 2 writes and its change_attr goes to 3. The client1
then queries for the change_attr and gets back 3 from the cache. The MDS
then recalls FILE_EXCL from client1 and now the MDS and client1 have the
same change_attr (3).

client2 then does a chmod on the file, and its change_attr goes to 2.
client1 then does a statx with STX_VERSION|STX_MODE. The MDS recalls the
AUTH_EXCL cap from client2, the change_attr in the MClientCaps is less
than the one in the MDS inode, so it gets discarded. client1 then sees
a new mode but the change_attr value has not changed, which violates the
rules.

Fix this with an extra increment of the MDS copy of the change_attr when
the caps being returned are dirty, and they don't contain exclusive write
caps.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/client: keep change_attr in Inode, CapSnap and MClientCaps

...and pass it around appropriately.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds: pass change_attr in InodeStat MClientReply, if feature is present

Signed-off-by: Jeff Layton <jlayton@redhat.com>

include/ceph_features: add FS_CHANGE_ATTR feature

...that is mapped to BTIME feature.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/MDCache: set change_attr to 0 on system_inodes

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/Server: set change_attr to 0 on creation

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/mdstypes: add change attribute to inode

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds/client: add btime to CapSnap and MClientCaps

Currently we don't have a mechanism to set the btime, but we will need
that eventually. If we want to allow the client to cache that change, we
need to be able to pass it back and forth between client and server.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

libcephfs: add a test for "lazy" statx

Create 2 clients. Create a file in client1, and do a lookup of it in
client2, and then ll_getattrx it from client2. chmod the file from
client1, ll_getattrx it client2 (this time with AT_NO_ATTR_SYNC) and
ensure that the ctime change is not seen.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

libcephfs: add a ceph_ll_getattrx and ceph_statx

New interfaces for fetching extended (and selective) stat information.
Additionally, applications can specify AT_NO_ATTR_SYNC in the flags to
indicate that they want to do a "lazy" statx that just hands out the
inode info from the cache, or AT_SYMLINK_NOFOLLOW to avoid following
symlinks when walking the path.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: move the device bitshift handling macros to Client.h

We'll need them in Client.cc now in addition to FUSE specific code.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

ceph: break up ll_getattr into two functions

...in preparation for adding a ll_getattrx.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: keep btime in Inode and InodeStat

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>

mds: pass btime in InodeStat MClientReply, if feature is present

Signed-off-by: Sage Weil <sage@redhat.com>

include/ceph_features: add FS_BTIME features

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDCache: set btime on system inodes

Signed-off-by: Sage Weil <sage@redhat.com>

mds/Server: set btime=ctime on creation

Signed-off-by: Sage Weil <sage@redhat.com>

mds/mdstypes: add btime to inode

Signed-off-by: Sage Weil <sage@redhat.com>

client: pass a mask parameter to path_walk

ll_walk expects to get a set of attributes out of a path_walk. Pass a
caps mask parameter into path_walk, and then apply it when we reach the
last component of the path.

This may prevent us from having to further iteract with the server after
the pathwalk, in some cases. If we know that we're going to need certain
caps to do the actual operation we can request them during the lookup
and may have all that we need by the time we go to do the real request.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #9301 from objoo/YD-mailmap-updates-MAY3

mailmap updates for upcoming 11.0.0

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #10878 from stiopaa1/mds_snaprealm_addconstToMem

mds/SnapRealm.h: add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10901 from stiopaa1/mds_mdcache_accessspecifierU

mds/MDCache.h: remove unneeded access specifier

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10894 from smithfarm/wip-suse-build-pycache

build/ops: force Python 3 packages to build in SUSE

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #10903 from SUSE/wip-release-notes-correct-correction

doc: fix some nits in release notes and releases table

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: add 11.0.0 to releases table

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: fix two broken links in releases.rst

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: release-notes.rst: move misplaced line

46264a3b6cb7f490680c79bf0957c36aeaea5f6c added the missing line to the wrong
section. This commit moves it to the right section (v0.94.8).

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10899 from SUSE/wip-retirement

doc: do not list all major versions in get-packages.rst

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10900 from SUSE/wip-hammer-release-notes-correction

doc: add missing PR to hammer 0.94.8 release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mds/MDCache.h: remove unneeded access specifier

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

doc: add missing PR to hammer 0.94.8 release notes

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: do not list all major versions in get-packages.rst

The list of major versions is difficult to maintain. This commit drops it and
replaces it with a link to releases.rst plus some general language about how we
recommend that everyone keep their clusters up-to-date.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10895 from liewegas/wip-changelog

doc: fix/add changelog for 10.2.2, 0.94.7, 0.94.8

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10844 from liewegas/wip-bluestore-fsck-leak

os/bluestore: fix when block device is not a multiple of the block size

Reviewed-by: Loic Dachary <ldachary@redhat.com>

unittest_bluefs: fix block device sizes

These should be a multiple of the block size (since we are feeding
the size directly into add_block_extent).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10779 from cbodley/wip-17073

rgw multisite: RGWDataSyncCR fails on errors from RGWListBucketIndexesCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10778 from cbodley/wip-17047

rgw multisite: RGWCoroutinesManager::run returns status of last cr

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10742 from cbodley/wip-rgw-test-multi-gateway

test/rgw: add --gateways-per-zone to test_multi.py

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10837 from ceph/wip-cmake

cmake: do not link against unused objects or libraries

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

doc/release-notes: fix v10.2.2 changelog link

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases: add v0.94.8 to timeline

Signed-off-by: Sage Weil <sage@redhat.com>

doc: add v0.94.8 changelog

Signed-off-by: Sage Weil <sage@redhat.com>

doc: add v0.94.7 changelog

Signed-off-by: Sage Weil <sage@redhat.com>

rpm: force creation of Python 3 __pycache__ on SUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>

Revert "ceph.spec.in: don't try to package __pycache__ for SUSE"

This reverts commit be39d6dc71af13eb3ee0d4482012a2091d121f66.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10547 from wjwithagen/wip-wjw-test-util

test/common/test_util.cc: FreeBSD does not have distro information

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9869 from yuyuyu101/wip-kqueue

msg/async/kqueue: avoid remove nonexist kqueue event

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10879 from dmick/wip-jerasure-aarch64

jerasure: include generic objects in neon jerasure lib (like sse3/4)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9943 from ceph/wip-ceph-disk-systemd

ceph-disk: do not activate device that is not ready

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10828 from michelmno/perf_local_flags

cmake: use PERF_LOCAL_FLAGS only if defined

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mds/SnapRealm.h: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9757 from SUSE/wip-16353

rpm: drop obsolete libs-compat and python-ceph-compat metapackages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #10730 from SUSE/wip-release-notes-draft

release-notes.rst: draft 0.94.8 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10805 from SUSE/wip-dont-package-__pycache__

ceph.spec.in: don't try to package __pycache__ for SUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Dirk Mueller <dmueller@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

jerasure: include generic objects in neon jerasure lib (like sse3/4)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #10577 from wjwithagen/wip-wjw-test-rados-striper

Rados-stripper: Flexible string matching for not found attributes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10770 from dillaman/wip-16773

librbd: interlock image refresh and exclusive lock operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10831 from cbodley/wip-17051

rgw: delete region map after upgrade to zonegroup map
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

os/bluestore: ensure block device size is a multiple of the block size

We might have a backing device that is an odd number of 512-byte sectors
but have the block_size configured to 4096.  Ensure the reported size
rounds down to avoid confusing other layers of the stack.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fsck on mkfs (on by default)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10824 from dachary/wip-17100-ceph-disk

tests: populate /dev/disk/by-partuuid for scsi_debug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10836 from weiqiaomiao/wqm-wip-period-update

rgw: fix period update --commit return error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10829 from liewegas/wip-readme-make-dist

README.md: replace package build instructions with tarball instructions
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #10576 from wjwithagen/wip-wjw-noise-lockdep

lockdep: verbose even if no logging is set

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rados-striper.sh: Use expect_failure and conditional stringmatching

 - Linux and FreeBSD do not have the same error strings

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10840 from david-z/wip-fix-manpage

doc: fix wrong osdkeepalive name in mount.ceph manpage

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix wrong osdkeepalive name in mount.ceph manpage

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

ceph.spec.in: don't try to package __pycache__ for SUSE

When building on openSUSE Tumbleweed, nothing seems to create
the various __pycache__ directories (so the build fails because
those files don't exist), and in any case they should be
created automatically at runtime, so shouldn't need to be
packaged.  However, the Fedora packaging guidelines suggest
including __pycache__, so I've used a %suse_version guard here.

Fixes: http://tracker.ceph.com/issues/17106
Signed-off-by: Tim Serong <tserong@suse.com>

tests: populate /dev/disk/by-partuuid for scsi_debug

The scsi_debug SCSI devices do not have a symlink in /dev/disk/by-partuuid
because they are filtered out by 60-persistent-storage.rules. That was
worked around by 60-ceph-partuuid-workaround-rules which has been
removed by 9f76b9ff31525eac01f04450d72559ec99927496.

Add create rules targetting this specific case, only for tests since the
problem does not show in real use cases.

Fixes: http://tracker.ceph.com/issues/17100
Signed-off-by: Loic Dachary <loic@dachary.org>

cmake: use PERF_LOCAL_FLAGS only if defined

in src/test/CMakeLists.txt
to avoid build error for ppc64/ppc64le archi

Fixes: http://tracker.ceph.com/issues/17104
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>

cmake: do not link against keyutils if not necessary

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not link secret.c if not necessary

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10811 from jimifm/master

osd/ReplicatedPG: remove unused param "op" from generate_subop()

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: fix period update --commit return error

period commit return error when the current period has a zonegroup which doesn't have a master zone

Fixes: http://tracker.ceph.com/issues/17110
Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

Merge pull request #10830 from SUSE/wip-fix-singleton-example

doc: fix singleton example in Developer Guide

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: delete region map after upgrade to zonegroup map

convert_regionmap() reads the region map and uses it to initialize the
zonegroup map.  but it doesn't remove the region_map afterwards, so
radosgw (and some radosgw-admin commands) will keep doing this on
startup, overwriting any changes made to the period/zonegroup map

Fixes: http://tracker.ceph.com/issues/17051
Signed-off-by: Casey Bodley <cbodley@redhat.com>

os/bluestore: fix whitespace in bitset apply helper

Signed-off-by: Sage Weil <sage@redhat.com>

doc: fix singleton example in Developer Guide

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10817 from stiopaa1/os_memstore_movePageSetObject

os/Memstore: move PageSetObject class to .cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10803 from stiopaa1/os_memstore_moveOmapIteratorImplClassToCC

os/MemStore: move OmapIteratorImpl to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10799 from stiopaa1/os_fusestore_removeUnneededHeader

os/FuseStore: remove unneeded header file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10801 from stiopaa1/os_memstore_addOverride

os/Memstore.h: add override to virtual functions

Reviewed-by: Sage Weil <sage@redhat.com>

README.md: replace package build instructions with tarball instructions

Signed-off-by: Sage Weil <sage@redhat.com>

test/common/test_util.cc: FreeBSD does not have distro information

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

librbd: remove unused refresh request logic

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: interlock image refresh and lock operations

Fixes: http://tracker.ceph.com/issues/16773
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: image state machine now has hooks for lock requests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: integrate asynchronous image rewatch state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper state machine for asynchronous watch recovery

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: exclusive lock now supports reacquiring a lost lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: store exclusive lock cookie instead of recalculating

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper state machine to update lock cookie

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_lock: support updating the lock cookie without releasing the lock

Fixes: http://tracker.ceph.com/issues/17015
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10743 from cbodley/wip-rgw-meta-sync-assert

rgw multisite: fix for assertion in RGWMetaSyncCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10032 from cbodley/wip-rgw-cr-deadlock

rgw: add deadlock detection to RGWCoroutinesManager::run()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10821 from atheism/arm-read-improve

arch/arm.c: remove unnecessary variable read for simplicity

Reviewed-by: Kefu Chai <kchai@redhat.com>

rpm: do not unnecessarily spawn shells for scriptlets

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: drop trouble-making divider comments

We have some %post scriptlets that fire up a shell just to execute a single
command like this:

%postun -n librados2
/sbin/ldconfig

Normally, it would be possible to optimize these as:

%postun -n librados2 -p /sbin/ldconfig

Except that, due to an RPM bug, there must not be *any* content (not even comments)
between the %postun and the next section.

See https://bugzilla.redhat.com/show_bug.cgi?id=1003962#c0 for details.

Signed-off-by: Nathan Cutler <ncutler@suse.com>