qa/test: added supported ditro

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21190 from smithfarm/wip-21033-jewel

jewel: build/ops: rpm: bump epoch ahead of ceph-common in RHEL base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #21213 from smithfarm/wip-23026-jewel

jewel: rgw: data sync of versioned objects, note updating bi marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21185 from smithfarm/wip-23158-jewel

jewel: mds: underwater dentry check in CDir::_omap_fetched is racy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18010 from jdurgin/wip-rocksdb-jewel

jewel: core: enable rocksdb for filestore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21199 from smithfarm/wip-22403-jewel

jewel: osd: replica read can trigger cache promotion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21200 from ovh/bp-jewel-backport-22448

jewel: osd/PrimaryLogPG: dump snap_trimq size

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21197 from smithfarm/wip-23521-jewel

jewel: ceph_authtool: add mode option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20381 from pdvian/wip-22941-jewel

jewel: core: librados: Double free in rados_getxattrs_next

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21203 from smithfarm/wip-22394-jewel

jewel: librbd: cannot copy all image-metas if we have more than 64 key/value pairs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21098 from smithfarm/wip-23348-jewel

jewel: rgw: inefficient buffer usage for PUTs

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #21215 from smithfarm/wip-23546-jewel

jewel: journal: Message too long error when appending journal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21189 from smithfarm/wip-23356-jewel

jewel: cephfs: client: prevent fallback to remount when dentry_invalidate_cb is true but root->dir is NULL

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #21232 from pdvian/wip-23543-jewel

jewel: rbd-nbd: fix ebusy when do map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21228 from dillaman/wip-22396-jewel

jewel: librbd: cannot clone all image-metas if we have more than 64 key/value pairs

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21227 from dillaman/wip-21442-jewel

jewel: rbd: clean up warnings when mirror commands used on non-setup pool

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21225 from dillaman/wip-20637-jewel

jewel: rbd-mirror: ignore permission errors on rbd_mirroring object

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21224 from dillaman/wip-21689-jewel

jewel: librbd: list_children should not attempt to refresh image

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21223 from dillaman/wip-21915-jewel

jewel: rbd-mirror: strip environment/CLI overrides for remote cluster

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21220 from dillaman/wip-21867-jewel

jewel: librbd: object map batch update might cause OSD suicide timeout

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21219 from dillaman/wip-23305-jewel

jewel: librbd: create+truncate for whole-object layered discards

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21207 from smithfarm/wip-23525-jewel

jewel: rbd: is_qemu_running in qemu_rebuild_object_map.sh and qemu_dynamic_features.sh may return false positive

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21206 from smithfarm/wip-23012-jewel

jewel: rbd: [journal] allocating a new tag after acquiring the lock should use on-disk committed position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21205 from smithfarm/wip-22578-jewel

jewel: rbd: rbd-mirror split brain test case can have a false-positive failure until teuthology

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21125 from smithfarm/wip-23508-jewel

jewel: tests: test_admin_socket.sh may fail on wait_for_clean

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20627 from pdvian/wip-23153-jewel

jewel: rbd: TestLibRBD.RenameViaLockOwner may still fail with -ENOENT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

rgw: fix memory fragmentation problem reading data from client.

mg_read returns 0 on EOF.  In some versions of civetweb, mg_read can
return "short" reads.  The logic in the rest of ceph depends on a read
always filling its buffer when possible.  So loop here and fill the
buffer.  Looping to fill the buffer means we must also track when
mg_read returns EOF, because after that further calls to mg_read will return -1.

Fixes: https://tracker.ceph.com/issues/23207
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit d9a150b1f1f9ec3952555186722c9b13d7448e2d)

Conflicts:
    src/rgw/rgw_civetweb.cc:
        - master RGWCivetWeb::RGWCivetWeb() becomes RGWMongoose::RGWMongoose() in jewel,
          with slightly different initializer list; adapt mg_read() call for jewel
        - master RGWCivetWeb::read_data() becomes RGWMongoose::read_data() in jewel,
          and inside this function we return -EIO in jewel, instead of throwing an exception

Merge pull request #18304 from smithfarm/wip-21546-jewel

jewel: rgw: file write error

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #21100 from cbodley/wip-23311

jewel: rgw: s3website error handler uses original object name

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21073 from linuxbox2/wip-rgw-tcmalloc

jewel: rgw: tcmalloc

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20800 from theanalyst/jewel-fe-fixes

jewel: rgw:  make init env methods return an error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20639 from mkogan1/wip-rgw-t23171

jewel: rgw: core dump, recursive lock of RGWKeystoneTokenCache

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20421 from oritwas/wip-rgw-jewel-22124

jewel: rgw: bucket resharding should not update bucket ACL or user stats

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20076 from smithfarm/wip-22774-jewel

jewel: rgw: file deadlock on lru evicting

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #19887 from pdvian/wip-22636-jewel

jewel: rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18207 from smithfarm/wip-21205-jewel

jewel: rgw: bi list entry count incremented on error, distorting error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21264 from smithfarm/wip-rgw-drop-btrfs-jewel

jewel: tests: rip out vestiges of btrfs from rgw suite

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #21184 from smithfarm/wip-21786-jewel

jewel: osd: OSDMap cache assert on shutdown

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

qa: rgw/multifs: test on XFS, drop BTRFS

objectstore/filestore-xfs.yaml was overriding fs/btrfs.yaml anyway,
which was confusing.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa: rgw/verify: test on XFS, drop BTRFS

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21156 from smithfarm/wip-22031-jewel

jewel: mds: FAILED assert(get_version() < pv) in CDir::mark_dirty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21172 from smithfarm/wip-22384-jewel

jewel: tests: src/test/libcephfs/test.cc:376: Expected: (len) > (0), actual: -34 vs 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21163 from smithfarm/wip-22380-jewel

jewel: cephfs: client reconnect gather race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20111 from pdvian/wip-22764-jewel

jewel: cephfs-journal-tool: add "set pool_id" option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21152 from smithfarm/wip-21489-jewel

jewel: tests: failures from pjd fstest

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21175 from smithfarm/wip-22970-jewel

jewel: mds: session reference leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #21208 from ukernel/jewel-23529

jewel: cephfs: fix tmap_upgrade crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21084 from pdvian/wip-23413-jewel

jewel: log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20622 from dzafman/wip-info-soid

jewel: follow-on: osd: be_select_auth_object() sanity check oi soid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21158 from smithfarm/wip-21784-jewel

jewel: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21135 from smithfarm/wip-23523-jewel

jewel: tests: unittest_pglog timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20999 from pdvian/wip-23411-jewel

jewel: legal: remove doc license ambiguity

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20882 from dzafman/wip-23307

jewel: ceph-objectstore-tool command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20763 from tchaikov/wip-23185-jewel

jewel: ceph.in: bypass codec when writing raw binary data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

PendingReleaseNotes: note about changing omap backend to rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

os/filestore: disable rocksdb compression

Experience working with customer escalations suggests that disabling
compression improves performance, and the storage overhead is generally
not a concern for the metadata and omap data we are storing.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b878ead071b328e5fe7309a2368383e67679e9f7)

Conflicts:
src/common/options.cc

Config options moved files since jewel.

common/options: enable multiple rocksdb compaction threads for filestore

One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.

For high-write and delete omap workloads I've seen up to 8 compaction
threads be used.  There's little overhead to having a higher max than
are needed, so set the default to 8.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 023fa810aa6b3af305e9027e3f717e54d1bb2712)

Conflicts:
src/common/options.cc (moved to config_opts.h)

tools/ceph-kvstore-tool: clean up destination store object

If left alive, background threads in the db may crash

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 718fc15831a2cb4eb91ed2aa2f694b47329f9189)

tools/ceph-kvstore-tool: add destination store type to store-copy

This allows e.g. copying from leveldb into rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a65a7f0430a32a7748b54f4babecba0f58a5e42f)

Makefile-rocksdb.am: update for new rocksdb version

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

osd: switch filestore to default to rocksdb

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ba96197e6d4614377c80cf16a1b29b6e6ac70f90)

Conflicts:
src/common/config_opts.h (trivial)

rocksdb: remove experimental status

We've been testing rocksdb on mons in QA for months now
with no problems.  We've also observed no problems with
its use in bluestore.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 68138893648e052a6549fecf23b0b58af921dfc7)

include/assert: define ceph_abort_msg(cct, msg)

Abort with an error string.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 23d6438f1253b593dd9a0e909e2ccbd045e168b8)

kv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result

If the underlying disk is missing, the current logic won't check the actual
reason why Get/Set failed, it will result to client get a empty key/value
pair which is not expected. The correct logic should be abort
right now. Otherwise, it will leads to undefined behavior.

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 8c60b55d7d49bc6fa1d8d076c025fc83283c0776)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

os/kv: beautify kvdb statistics

Improves the presentation of rocksdb dump statistics

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
(cherry picked from commit 8e1d4ac92e2f9cd5f7438a99459bc063adf2542b)

src/kv/RocksDBStore.cc:fix typos

Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
(cherry picked from commit b26555a0422423a9fef0034aa9313b3a4c1308ef)

kv/RocksDBStore: dump extended stats

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 9fda9ce0b0fbf795469b50f6e9ebcb9bd0bc32a4)

Conflicts:
src/common/config_opts.h
src/kv/RocksDBStore.cc
src/kv/RocksDBStore.h
(trivial resolution)

kv/RocksDBStore: no reason to make write_batch using new/delete

discards new/delete for bat

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 79a6611c0aa2c5b16435fad53268dd29a8541188)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

Logging rocksdb transaction in ceph log

The rocksdb transaction is now been logged into ceph log based
on debug level. If transaction is failed, the error code,
error string and the entire transaction is dumped in the log.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
(cherry picked from commit 46f1f9e5ca2f4e34393dfa1952fa4283a859d869)

RocksDBStore: use Get for multiget

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit a44f4f7238585996f83f7e35e01587671fe675d4)

kv/RocksDBStore: convert get() to use rocksdb Get() instead of iterator

For RocksDB, iterator will bypass row cache.
We don't want this to happen for get(), after row cache is enabled.

Signed-off-by: Jianjian Huo <jianjian.huo@ssi.samsung.com>
(cherry picked from commit f6ac5579190549291f6385775070c305dc8fba8b)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

tests: unittest_rocksdb_option: remove deprecated option

* disable_data_sync was removed from rocksdb, see
  https://github.com/facebook/rocksdb/blob/02799ad77a16332ee5bfb570f62ab6162a788a5a/HISTORY.md#530-03082017.
  so remove it.
* and check options.compression, which was commented out.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 94f4b48cd4be1b09ae5ee74b61c069e08424c3a5)

rocksdb update

This is the version in 12.2.0, that we have been running for ~6 month or so.

Signed-off-by: Sage Weil <sage@redhat.com>

kv/Makefile: fix rocksdb build options (USE_RTTI=1)

Signed-off-by: Sage Weil <sage@redhat.com>

kv/RocksDBStore: Add table options for filter and index tuning.

This is a modified cherry-pick of commit 9c15338 to work with the old
config_opts settings.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9c15338ad3049f5171cdecafb9370d6bf5dbe913)

Merge pull request #21234 from tchaikov/wip-jewel-rocksdb-without-jemalloc

jewel: rocksdb: disable jemalloc explicitly

Reviewed-by: Nathan Cutler <ncutler@suse.com>

rocksdb: disable jemalloc explicitly

rocksdb's makefile tries to detect jemalloc automatically and compiles
with it enabled if it's found, but if ceph disable jemalloc, it won't
link against jemalloc. this causes link failure if we compile ceph
without jemalloc enabled in a host with jemalloc library installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
src/kv/Makefile.am: this change is not cherry-picked from
master. as master does not use autotools anymore. and master is
using rocksdb's cmake script for building, the latter does not
jemalloc by default.

rbd-nbd: fix ebusy when do map

When doing rbd-nbd map, if the Ceph service is not available,
the codes will wait on rados.connect(), unless killing the process.
In that case, the close_nbd logic is skipped with NBD_CLEAR_SOCK ioctl
not called. On the CentOS 7 kernel, it leaves nbd->file not cleared, which
causes the subsequent map requests return EBUSY, this patch fixes it
by connecting Ceph first prior to calling NBD_SET_SOCK ioctl

Fixes: http://tracker.ceph.com/issues/23528
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit ab77dcc0170c0d63795fe0d50427cda630bfd593)

Conflicts:
src/tools/rbd_nbd/rbd-nbd.cc: - Resolved for cfg changes & devpath.empty()
      - Removed cfg->exclusive condition as exclusive
bool not available

rbd: mirror "get" actions now have cleaner error messages

Fixes: http://tracker.ceph.com/issues/21319
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 053be9847ff0052348aa259520d641923e57537d)

Conflicts:
src/librbd/api/Mirror.cc: methods moved from internal.cc
src/tools/rbd/action/MirrorPool.cc: refactored post-Jewel

cls/rbd: avoid recursively listing the watchers on rbd_mirroring object

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 79be496ac098859304efcabc7b5eb8afca6eca91)

Conflicts:
src/cls/rbd/cls_rbd.cc: trivial resolution

test_librbd:add a test case of 70 key/val pairs in TestClone2

Signed-off-by: PCzhangPC <pengcheng.zhang@easystack.cn>
(cherry picked from commit 85713bdcc3870728dce02f87ee3cfd36257ae533)

test_librbd:add a test case of 70 key/val pairs in TestClone

Signed-off-by: PCzhangPC <pengcheng.zhang@easystack.cn>
(cherry picked from commit 3e08577a296bc4ce2c1592849108926b45e9b541)

librbd: cannot clone all image-metas if we have more than 64 key/value pairs

Signed-off-by: PCzhangPC <pengcheng.zhang@easystack.cn>
(cherry picked from commit ccc56384032be7f6bd48e28b6825b3ce589c7cf7)

Conflicts:
src/librbd/image/CloneRequest.h/cc: logic exists in internal.cc

rbd-mirror: ignore permission errors on rbd_mirroring object

Fixes: http://tracker.ceph.com/issues/20571
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6c2d896ea2cee515f0fecec41adbec1e7d6dee23)

Conflicts:
src/tools/rbd_mirror/ClusterWatcher.cc: trivial resolution

librbd: list_children should not attempt to refresh image

The snap_lock is being held when this method is invoked, which can
result in a deadlock.

Fixes: http://tracker.ceph.com/issues/21670
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4c585d826f38ff97d3a484a30eca0588c79396b4)

Conflicts:
src/librbd/internal.cc: functions re-arranged in later release

rbd-mirror: strip environment/CLI overrides for remote cluster

Fixes: http://tracker.ceph.com/issues/21894
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry-picked from commit 70dc22e03a115b130b9c214030d5996276a1634b)

Conflicts:
    src/tools/rbd_mirror/Replayer.cc: utilize old-style config accessors
    src/tools/rbd_mirror/Replayer.h: trivial resolution

librbd: fix snap create/rm may taking long time

  fix snap create/rm may taking long time
  http://tracker.ceph.com/issues/22716

Signed-off-by: Song Shun <song.shun3@zte.com.cn>
(cherry picked from commit d04ed348a1b20e5ea5bedada2462cb41f0f1d85a)

librbd: batch large object map updates into multiple chunks

Fixes: http://tracker.ceph.com/issues/21797
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 04c5d12acc045731fbf1d0ddce276b5743d9fd89)

Conflicts:
src/librbd/ObjectMap.h: trivial resolution
src/librbd/object_map/UpdateRequest.cc: trivial resolution
src/librbd/object_map/UpdateRequest.h: trivial resolution

librbd: combine trim state machine object map batch update states

The PRE/POST states were previously divided into two halves for handling
the copy-up batch and the direct removal batch. This can be simplified by
just using a single PRE/POST for the entire deletion region.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 72ce4576fa2b562799a5bc78bd423cfabe097d67)

Conflicts:
src/librbd/operation/TrimRequest.cc: trivial resolution
src/librbd/operation/TrimRequest.h: trivial resolution

cls/rbd: object map update now utilizes constant-time bit vector operations

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fc99603ea2b5b268181d38507b8b6fb55ae0a6d0)

common/bit_vector: provide constant time iteration of underlying bufferlist

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit daa29f7d2b50bc4b44a108a0f91d45ce90fc16e5)

Conflicts:
src/common/bit_vector.hpp: trivial resolution

common/buffer: expose hidden const deference operator

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f35947e3c6661dfe9f951896bc54bc8da158b490)

librbd: create+truncate for whole-object layered discards

A whole-object layered discard is implemented as a truncate rather
than a delete: a dummy object is needed to prevent the CoW machinery
from kicking in.  However, a truncate on a non-existent object is
a no-op.  If the object doesn't exist in HEAD, a discard request is
effectively ignored -- parent blocks are still seen afterwards.

A non-exclusive create on an existing object is also a no-op, so the
fix is to do a compound create+truncate instead.

(derived from commit 0514480b16b0a1e76a9ba31cc3ba58a32b7bee18)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: limit number of appends sent in one librados op

Fixes: http://tracker.ceph.com/issues/23526
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d49cfcdeb75aa6456aec9909ef6a7dbc2bc5f5e1)

rgw: log unlink_instance mtime as object's mtime

This is needed so that when squashing entries during data
sync we get a consistent mtime for that object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit bef22609de95843008b5d02eb075252bb97fbe90)

mds: fix check of underwater dentries

Underwater dentry is dentry that is dirty in our cache from journal
replay, but had already been flushed to disk before the mds failed.
To decide if an dentry is underwater, original code compares dirty
dentry's version to on-disk dirfrag's version. This method is racy
because CDir::log_mark_dirty() can increase dirfrag's version without
adding log event. After mds failover, version of dirfrag from journal
replay can be less than on-disk dirfrag's version. So newly dirtied
dentry can be equal to or less than the on-disk dirfrag's version.

The race can cause incorrect fragstat/rstat

Fixes: http://tracker.ceph.com/issues/23032
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9d271696b7735ab5b7384cdd386d6ac3eaafe437)

Conflicts:
src/mds/CDir.cc - conditional being replaced by "if (!dn)" is different
  in jewel; refrain from changing "touching wanted dn"
                          conditional
src/mds/CDir.h - undef_inodes pointer, which is being deleted, has
                         different type in jewel

tools/cephfs: fix tmap_upgrade crash

MDSUtility::shutdown() is called by deconstructor of MDSUtility.
shouldn't call it manually.

introduced by commit 1c7b6b00 "jewel: cephfs-journal-tool: move
shutdown to the deconstructor of MDSUtility" make

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23529

qa/suites/rbd: set qemu task time_wait param

so workloads qemu_dynamic_features.sh and qemu_rebuild_object_map.sh,
which check if qemu is finished with periodicity 60 sec, have enough
time to detect this before the rbd image is removed.

Fixes: https://tracker.ceph.com/issues/23502
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 6fe8d6470506a9947f685b05683e2e046457a9f5)

Conflicts:
qa/suites/rbd/maintenance/qemu/xfstests.yaml - trivial resolution

qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished

In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit af15b2cef46c8d3ab1f9770601ae1abfe9c6b300)

Conflicts:
qa/tasks/qemu.py - trivial resolution

journal: flush commit positions should wait for refresh

Fixes: http://tracker.ceph.com/issues/22945
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 24df022e0b34788953734c2e622c336077958e78)

Conflicts:
src/journal/JournalMetadata.cc - trivial resolution