client: properly trim unlinked inode

Client should trim inode from its cache when receiving a cap message
with nlink == 0. But the corresponding code has a bug, it does nothing
when inode's nlink is already 0.

Fixes: #13903
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7246 from ukernel/jewel-14377

ceph_test_libcephfs: tolerate duplicated entries in readdir

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7209 from ukernel/jewel-11517

client: check if Fh is readable when processing a read

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7199 from ukernel/jewel-14254

mds: fix client cap/message replay order on restart

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7274 from dillaman/wip-helgrind-race-jewel

helgrind: additional race conditionslibrbd: journal replay should honor inter-event dependencies

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: missing lock on state transition

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: annotate false-positive race conditions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7272 from dillaman/wip-14385-jewel

qa: disable rbd/qemu-iotests test case 055 on RHEL/CentOSlibrbd: journal replay should honor inter-event dependencies

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: disable rbd/qemu-iotests test case 055 on RHEL/CentOS

Fixes: #14385
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph_test_libcephfs: tolerate duplicated entries in readdir

If a dirfrag get fragmented in the middle of readdir, the readdir
result may contain duplicated entries. (but there shouldn't be
any missing entries)

Fixes: #14377
Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: add config option to suspend logging

The option creates long-standing unsafe MDS requests. It help in
testing unsafe request related corner cases.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix completed cap flush handling

Client re-send cap flush when MDS restarts. The cap flush message
may release some caps even if the corresponding flush is already
completed.

Fixes: #13546
Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: delay handling client caps until corresponding inode is created

When handling client caps in clientreplay stage, it's possible that
corresponding inode does not exist because the client request which
creates inode hasn't been replayed. To handle this corner case, we
delay handling caps message until corresponding inode is created.

Fixes: #14254
Signed-off-by: Yan, Zheng <zyan@redhat.com>

ceph_test_libcephfs: check file open mode

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: check if Fh is readable before read

Fixes: #11517
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7220 from dillaman/wip-13810-jewel

tests: notification slave needs to wait for master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7208 from dillaman/wip-14163-jewel

helgrind: fix real (and imaginary) race conditions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: notification slave needs to wait for master

If the slave instance starts before the master, race
conditions are possible.

Fixes: #13810
Backport: infernalis, hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: log write/discard errors from clip operation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

10.0.2

osdc: hold rwlock when updating crush location config

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

msg/simple: destroy spinlock in destructor

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osdc: hold writer lock when updating session ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: hold lock while testing state variable

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

config: avoid re-entrant locking on observer callback

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: delay registering PointerWQ with thread pool

Derived classes should be responsible for registration to avoid
accessing a partially constructed object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: suppress race conditions in external libraries

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: annotate false-positive race conditions

Fixes: #14163
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

deb,rpm: add valgrind to build dependencies

Provides necessary support for annotating helgrind
false-positives.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mds: fix flushing caps check

Client may flush and drop caps at the same time. If client need to
send cap reconnect before the caps get flushed. The issued caps in
the cap reconnect does not include the flushing caps. When choosing
lock states we should consider the flushing caps.

The check for caps haven't been flushed is wrong, fix it.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: avoid re-sending cap flushes twice during MDS recovers

During MDS recovers, client may send same cap flushes twice. The
first time is when MDS enters reconnect stage (only for flushing
caps which are also revoked), the second time is when MDS goes
active. If we send cap flushes when enters reconnect stage, we
should avoiding sending again.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7204 from dillaman/wip-14308-jewel

librbd: better handling of exclusive lock transition period

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7205 from ceph/wip-python-dev-dep

packaging: add build dependency on python devel package

Reviewed-by: Alfredo Deza <adeza@redhat.com>

packaging: add build dependency on python devel package

This is required for building python modules with cython, but for some
reason is not a direct dependency (on debian only a recommends) for
the cython package itself.

Refs: #14059
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

librbd: add additional granularity to lock states

Allow all pre-release actions to see that the exclusive lock is
still owned and all post-acquire actions to see that the exclusive
lock is now owned.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initialize object map before replaying journal

The journal might replay write events which will need to update the
object map.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not accept RPC requests while transitioning lock state

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid error message when peer owns exclusive lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7147 from athanatos/wip-14247

Wip 14247

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7148 from athanatos/wip-14248

OpRequest: release the message throttle when unregistered

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6801 from athanatos/wip-13965

Wip 13965

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7143 from dillaman/wip-librbd-rename-request

librbd: correct include guard in RenameRequest.h

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: correct include guard in RenameRequest.h

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7129 from dillaman/wip-14167-jewel

librbd: disable copy-on-read when not exclusive lock owner

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'wip-14241-jewel' of git://github.com/dillaman/ceph into jewel

librbd: don't use write_full optimization with clones

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'wip-client-readdir-fix' of git://github.com/ukernel/ceph into jewel

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

librbd: disable write_full optimization if image has parent

Fixes: #14241
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new test case for write_full optimization

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: directly execute copy-on-read copyup operations

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

OSD::ScrubJob: correctly handle small osd_scrub_interval_randomize_ratio

utime_t &operator+=(utime_t, double) exists, let's just use that and
not muck about with converting the double to an int.

Fixes: 14247
Signed-off-by: Samuel Just <sjust@redhat.com>

Revert "osd: fix arithmetic exception when scrub_min_interval is small"

This reverts commit a828a4df54f040a39f54c4cdecde234376301be5.

OpRequest: release the message throttle when unregistered

We don't want messages in the OpTracker history hanging on to
message throttle.

Fixes: #14248
Backport: hammer, firefly
Signed-off-by: Samuel Just <sjust@redhat.com>

librbd: disable copy-on-read when not exclusive lock owner

Fixes: #14167
Backport: infernalis, hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: fsx should disable journaling feature for krbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7114 from dillaman/wip-14234-jewel

librbd: reduce verbosity of common error condition logging

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'wip-14237-jewel' of git://github.com/dillaman/ceph into jewel

qa/workunits/rbd: import_export should use clean temp space

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7113 from dillaman/wip-14168-jewel

journal: disconnect watch after watch error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/rbd: import_export should use clean temp space

Fixes: #14237
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: reduce verbosity of common error condition logging

Fixes: #14234
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: disconnect watch after watch error

Fixes: #14168
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Revert "rgw: use smart pointer for C_Reinitwatch"

This commit introduced regressions, issues #14173, #14180.

The problem is that the context deletes itself after execution,
so using a shared_ptr to hold it is problematic, as we'll eventually
have a double free. Also, it doesn't solve the underlying leak issue.

This reverts commit e9daed221e7c6fd84aef78c3d5fc18470b916fcb.

Merge pull request #7101 from dillaman/wip-14221-jewel

qa/workunits/rbd: rbd-nbd test should use sudo for map/unmap ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/rbd: rbd-nbd test should use sudo for map/unmap ops

Fixes: #14221
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6933 from jcsp/wip-14048

mds: tear down connections from `tell` commands

Merge pull request #7043 from dillaman/wip-14170-jewel

librbd: do not ignore self-managed snapshot release result

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: do not ignore self-managed snapshot release result

Fixes: #14170
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7042 from dillaman/wip-14164-jewel

librbd: properly handle replay of snap remove RPC message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7040 from dillaman/wip-14092-jewel

librbd: ensure librados callbacks are flushed prior to destroying

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7035 from dillaman/wip-14122-jewel

librbd: clear error when older OSD doesn't support image flags

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: properly handle replay of snap remove RPC message

Fixes: #14164
Backport: infernalis
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ensure librados callbacks are flushed prior to destroying image

Fixes: #14092
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clear error when older OSD doesn't support image flags

Fixes: #14122
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7002 from dillaman/wip-14092-jewel

tests: flush op work queue prior to destroying MockImageCtx

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: flush op work queue prior to destroying MockImageCtx

Fixes: #14092
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6815 from badone/wip-13993

common: log: Assign LOG_DEBUG priority to syslog calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6979 from dillaman/wip-14123-jewel

librbd: correct issues discovered when cache is disabled

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6974 from dillaman/wip-14108-jewel

librbd: correct issues discovered during teuthology testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: skip journal event if write/discard was no-op

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object map ENOENT optimizations should complete asynchronously

This ensures a consistent lock ordering between the optimized path and
a librados callback. Remove the cache writebach handler's custom finisher
and re-use the op_work_queue.

Fixes: #14123
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: update exclusive lock state when shutting down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: extra request required if journal enabled w/ caching

Fixes: #14108
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: stop the copyup thread during shutdown

Fixes: #14107, #14108
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6972 from dillaman/wip-14106-jewel

rbd: merge_diff test should use new --object-size parameter instead of --order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/rbd: use --object-size instead of --order

Fixes: #14106
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: process crushed, rbd creating with striping parameters

Signed-off-by: Na Xie <xie.na@h3c.com>

log: Log.cc: Assign LOG_DEBUG priority to syslog calls

Fixes: #13993
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #6932 from aiicore/test_fix

tests: allow object corpus readable test to skip specific incompat instances

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6941 from jcsp/wip-14029

mds: fix setvxattr (broken in a536d114)

Merge pull request #6948 from ceph/revert-6939-bp-remove-bz2dev

Revert "makefiles: remove bz2-dev from dependencies"

Reviewed-by: Dan Mick <dmick@redhat.com>

Revert "makefiles: remove bz2-dev from dependencies"

Merge pull request #6692 from SUSE/wip-13860

ceph.spec.in: add BuildRequires: systemd

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6939 from branch-predictor/bp-remove-bz2dev

makefiles: remove bz2-dev from dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6943 from smithfarm/wip-doc-dev-typo

doc: fix typo in developer guide

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix typo in developer guide

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mds: fix setvxattr (broken in a536d114)

There was an accidental move of this line
when adding the MAY_SET_POOL check, which
was causing setxattr to proceed before
it had the right locks, and thereby apply
its checks on bad data (symptom was failing
to detect that the file had data written to it).

Fixes: #14029
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6821 from H3C/wip-cephfs-f11322

ceph-fuse: add process to ceph-fuse --help

Reviewed-by: John Spray <john.spray@redhat.com>

makefiles: remove bz2-dev from dependencies

The only thing that uses bzip2-devel is RocksDB, and it's optional, not
requirement. Drop the bzip2-devel/libbz2-dev dependency entirely, and
let RocksDB use it only if it is already present.

Fixes: #13981
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

ceph-fuse: add process to ceph-fuse --help

Signed-off-by: Wei Feng <feng.wei@h3c.com>

Merge pull request #6934 from smithfarm/wip-14070

doc: Update list of admin/build-doc dependencies

Reviewed-by: Loic Dachary <ldachary@redhat.com>