Merge pull request #17649 from dillaman/wip-21346

jewel: qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17166 from smithfarm/wip-20719-jewel

jewel: rgw: Truncated objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17165 from smithfarm/wip-20715-jewel

jewel: rgw: radosgw-admin data sync run crash

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17164 from smithfarm/wip-20712-jewel

jewel: rgw: not initialized pointer cause rgw crash with ec data pool

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17159 from smithfarm/wip-20709-jewel

jewel: rgw: radosgw-admin: bucket rm with --bypass-gc and without --purge-data doesn't throw error message

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17156 from smithfarm/wip-20673-jewel

jewel: rgw: multisite: RGWRadosRemoveOmapKeysCR::request_complete return val is wrong

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17148 from smithfarm/wip-20292-jewel

jewel: rgw: multisite: log_meta on secondary zone causes continuous loop of metadata sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17147 from smithfarm/wip-20290-jewel

jewel: rgw: rgw_file: prevent conflict of mkdir between restarts

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16856 from prallabh/wip-20895

jewel: rgw: bucket index check in radosgw-admin removes valid index

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16767 from linuxbox2/jewel-rgw-mpart-race

jewel: rgw : fix race in RGWCompleteMultipart

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17574 from smithfarm/wip-21297-jewel

jewel: libradosstriper processes arbitrary printf placeholders in user input

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: fix radosgw-admin data sync run crash

If sync thread have run before, then run data sync init. sync_status is still remain in rados pool. so no matter sync_status exists or not, if state is StateInit, sync_status.sync_info.num_shards should be updated.

Fixes: http://tracker.ceph.com/issues/20423
Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
(cherry picked from commit c307910d7131fc290f00bb8e33876e667afb72ec)

Conflicts:
    src/rgw/rgw_data_sync.cc (no data_sync_module or instance_id in jewel)

Merge pull request #16299 from smithfarm/wip-20262-jewel

jewel: rgw: datalog trim can't work as expected

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17167 from smithfarm/wip-20720-jewel

jewel: multisite: RGWPeriodPuller does not call RGWPeriod::reflect() on new period

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15556 from cbodley/wip-19847

jewel: rgw: multisite: fixes for meta sync across periods

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17155 from smithfarm/wip-20641-jewel

jewel: rgw: multisite: lock is not released when RGWMetaSyncShardCR::full_sync() fails to write marker

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17278 from smithfarm/wip-20728-jewel

jewel: rgw: multipart parts on versioned bucket create versioned bucket index entries

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: we no longer use log_meta

Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit ac8b0077c1f53ff0037c4163489b838eebf8c247)

rgw: is_single_zonegroup doesn't use store or cct

Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 0b0090cee249c5380efa4bb416d724a7e2dbfc08)

rgw: log_meta only for more than one zone

Fixes: http://tracker.ceph.com/issues/20357
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 088dba03ccc65609ef1c51306389ebcf1126ec8c)

rgw: only log metadata on metadata master zone

Fixes: http://tracker.ceph.com/issues/20244
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b8272f3607074a2f7cbfd08f7bbc82f22cf120ba)

Merge pull request #16296 from smithfarm/wip-20267-jewel

jewel: rbd: api: is_exclusive_lock_owner shouldn't return -EBUSY

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17385 from dillaman/wip-19957

jewel: librbd: reacquire lock should update lock owner client id

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17402 from dillaman/wip-20515

jewel: librbd: fail IO request when exclusive lock cannot be obtained

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17412 from dillaman/wip-18704

jewel: librbd: prevent self-blacklisting during break lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: relax greps to support upgrade formatting change

Fixes: http://tracker.ceph.com/issues/21181
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 273c84578b06905229a7c6eae7b1a12bbe60de95)

Conflicts:
qa/workunits/rbd/import_export.sh: trivial resolution

Merge pull request #15189 from mslovy/wip-19996-jewel

jewel: osd: fix infinite loops in fiemap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16711 from cbodley/wip-20346

jewel: rgw: meta sync thread crash at RGWMetaSyncShardCR

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17552 from dillaman/wip-21279

jewel: test/cls_rbd: mask newer feature bits to support upgrade tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17514 from smithfarm/wip-21232-jewel

jewel: core: interval_set: optimize intersect_of insert operations

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16293 from smithfarm/wip-20460-jewel

jewel: core: test_envlibrados_for_rocksdb.yaml fails on crypto restart

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rbd: fixed cache override

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 98061bb3d7ce6309ddb04ea4d7e9d44a7ecd09c6)

libradosstriper: remove format injection vulnerability

Fixes: http://tracker.ceph.com/issues/20240
Signed-off-by: Stan K <redrampage@selectel.ru>
(cherry picked from commit e94d3b9661d287e500cdf4e6c102d2e3bb04475e)

Merge pull request #17084 from badone/wip-ceph-disk-dmcrypt-key-jewel

jewel: ceph-disk: Use stdin for 'config-key put' command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17008 from asheplyakov/19182-bp-jewel

jewel: mon: fix force_pg_create pg stuck in creating bug

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17133 from smithfarm/wip-21035-jewel

jewel: ceph-disk: systemd unit timesout too quickly

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16316 from smithfarm/wip-20362-jewel

jewel: rgw: VersionIdMarker and NextVersionIdMarker are not returned when listing object versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph_test_objectstore: disable filestore_fiemap

This very reliably triggers a test failure for
ObjectStore/StoreTest.Synthetic/1.

FIEMAP is bad!  Do not use it!

Signed-off-by: Sage Weil <sage@redhat.com>

test/cls_rbd: mask newer feature bits to support upgrade tests

Fixes: http://tracker.ceph.com/issues/21217
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit aeb8f29d21625f3570e2f3e6dd26594d1a6df810)

Merge pull request #16703 from ddiss/wip-20837-jewel

jewel: ceph-disk: don't activate suppressed journal devices

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17396 from smithfarm/wip-21113-jewel

jewel: cephfs: get_quota_root sends lookupname op for every buffered write

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #17009 from dreamhost/wip-16463-jewel

jewel: rgw: aws4: add rgw_s3_auth_aws4_force_boto2_compat conf option

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16963 from smithfarm/wip-20972-jewel

jewel: cephfs: ceph-fuse segfaults at mount time, assert in ceph::log::Log::stop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16952 from dreamhost/wip-20966-jewel

jewel: rgw: radosgw-admin: fix bucket limit check argparse, div(0)

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #16951 from linuxbox2/jewel-aws4-plus

jewel: rgw: replace '+' with "%20" in canonical query string for s3 v4 auth

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16880 from linuxbox2/jewel-rgw-dir-suggest

jewel: rgw: Fix up to 1000 entries at a time in check_bad_index_multipart

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16720 from prallabh/wip-jewel-20821

jewel: rgw: Do not decrement stats cache when the cache values are zero

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

interval_set: optimize intersect_of for identical spans

Optimize comparisons for identical spans of intervals.
When this patch is combined with the previous map insert
optimization, a benchmark using 400000 identical
intervals shows a 7 times performance improvement in
comparison to without the patches.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit b6a035666c2765f8895ee9991348dbc025613ed7)

interval_set: optimize intersect_of insert operations

Use the std::map insert method with hint iterator to optimize
inserts. This increases performance more than 3.5 times for
large numbers of intervals. This will help performance
especially in the PGPool::update method, where profiling data
has shown that intersection operations are a hot spot. The
following benchmark data is for 400000 intervals:

    4 +-+--+----+----+----+----+----+----+----+----+--+-+
P     +    +    +    +    +    +    +    +  *************
E     |                             ********            |
R 3.5 +-+                       ****                  +-+
F     |                   ******                        |
O     |                 **                              |
R   3 +-+           ****                              +-+
M     |          ***                                    |
A     |        **                                       |
N 2.5 +-+     *                                       +-+
C     |     **                                          |
E     |     *                                           |
    2 +-+ **                                          +-+
R     |  **                                             |
A     | **                                              |
T 1.5 +**                                             +-+
I     |**                                               |
O     +*   +    +    +    +    +    +    +    +    +    +
    1 +*+--+----+----+----+----+----+----+----+----+--+-+
      0   0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9

                        SET SIZE RATIO

The above chart was generated using benchmark results
from the following program:

#include <iostream>
#include <sys/timeb.h>
#include "include/interval_set.h"

int main(int argc, char *argv[])
{
  const int interval_count = std::stoi(argv[1]);
  const int interval_distance = 4;
  const int interval_size = 2;
  const int sample_count = 8;
  const int max_offset = interval_count * interval_distance;
  interval_set<int> a, b, intersection;

  for (int i = 0; i < max_offset; i+=interval_distance) {
    a.insert(i, interval_size);
  }

  for (int m = 1; m < 100; m++) {
    float ratio = 1 / float(m);

    for (int i = 0; i < max_offset; i+=interval_distance*m) {
      b.insert(i, interval_size);
    }

    struct timeb start, end;
    int ms = 0;
    for (int i = 0; i < sample_count; i++) {
      ftime(&start);
      intersection.intersection_of(a, b);
      ftime(&end);
      ms += (int) (1000.0 * (end.time - start.time)
        + (end.millitm - start.millitm));
      intersection.clear();
    }
    b.clear();

    std::cout << ratio << "\t" << ms << std::endl << std::flush;
  }
}

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 32bc0430f70b057d1bba623252e92ab9f279028d)

Merge pull request #15966 from prallabh/jewel

jewel: rgw: Custom data header support

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16144 from smithfarm/wip-20364-jewel

jewel: mon: osd crush set crushmap need sanity check

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osdc/Objecter: release message if it is not handled

Fixes: http://tracker.ceph.com/issues/19741
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5c2774234c20d809d56d005b4db34070f3029684)

Conflicts:
   src/osdc/Objecter.h (in master, ms_fast_dispatch function declaration has an
                       "override" specifier; in jewel, this specifier is missing)

crypto: allow PK11 module to load even if it's already initialized

there is chance that other pieces of application loads PK11 module
already and does not finalize it before calling common_init_finish().

also, upon fork, PK11 module resets its entire status including `nsc_init`,
by which PK11 module tell if it is initialized or not. so the behavior
of NSS_InitContext() could be different before and after fork. that's
another reason to ignore CKR_CRYPTOKI_ALREADY_INITIALIZED error (see
NSS_GetError()).

Fixes: http://tracker.ceph.com/issues/19741
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fcc3effd8b447ef0c54b4c806b8f6e996d7467dd)

Merge pull request #17210 from dzafman/wip-20730-jewel

jewel: core: disable skewed utilization warning by default

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16405 from dzafman/wip-20041-jewel

jewel: osd: scrub_to specifies clone ver, but transaction include head write ver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16169 from Vicente-Cheng/wip-20511-jewel

jewel: osd: ReplicatedPG: solve cache tier osd high memory consumption

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16167 from Vicente-Cheng/wip-20492-jewel

jewel: osd: omap threadpool heartbeat is only reset every 100 values

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16151 from Vicente-Cheng/wip-20349-jewel

jewel: cephfs: df reports negative disk "used" value when quota exceed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16150 from Vicente-Cheng/wip-20403-jewel

jewel: mds: save  projected path into inode_t::stray_prior_path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16141 from smithfarm/wip-20270-jewel

jewel: tests: LibRadosMiscConnectFailure.ConnectFailure hang

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16285 from smithfarm/wip-18137-jewel

jewel: rbd-mirror: image sync should send NOCACHE advise flag

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #16295 from smithfarm/wip-20265-jewel

jewel: rbd: cli: ensure positional arguments exist before casting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16276 from smithfarm/wip-19563-jewel

jewel: rgw: folders starting with _ underscore are not in bucket index

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16268 from cbodley/wip-16072-jewel

jewel: rgw: rgw_common.cc: modify the end check in RGWHTTPArgs::sys_get

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16266 from cbodley/wip-20408

jewel: rgw: multipart copy-part remove '/' for s3 java sdk request header

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15988 from cbodley/wip-jewel-15983

jewel: rgw: fix infinite loop in rest api for log list

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15449 from smithfarm/wip-19720-jewel

jewel: rgw: rgw_file: fix size and (c|m)time unix attrs in write_finish

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #17351 from liewegas/wip-prune-past-intervals-jewel

osd: add osd_hack_prune_past_intervals

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: prevent self-blacklisting during break lock

(derived from commit 5c590acaec4dd66a9a8c3aa0ec8ab904dd350216)

Fixes: http://tracker.ceph.com/issues/18704
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: osd crush set crushmap need sanity check

The sanity check verifying the new crushmap does not remove crush rules
that are in use is not exclusive to ceph setcrushmap.

Fixes: http://tracker.ceph.com/issues/19302
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit ed760457bf154c10adf75c6df046eecab7eb8e4b)

 crush: when take place the crush map should consider the rule is in used

Signed-off-by: song baisen song.baisen@zte.com.cn
(cherry picked from commit 09bf6f2858d09c8c314a8c242a9c70d2834718dc)

librbd: fix missing write block validation in IO work queue

(derived from commit 8e76ebd93fd7525f6909a7da178e9b92c98e1ebe)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: test dynamic features with cache disabled

The journaling logic follows different paths depending on the
cache configuration.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 57a507d845339e9052122755bd460eb6d61d6060)

qa/tasks/qemu: rbd cache is enabled by default

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fdc4c9221f2c54d5fb55e74491ebc5165f73e776)

test: unit tests for librbd IO work queue failure path

(derived from commit 6e23ef358f29ca65a2a6b08b133416415e117b06)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: cleanup interface between IO work queue and IO requests

This will help mock the testing of the IO work queue

(derived from commit e0834d12e91bf085a5c9503f1c69a2bee2d21edf)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common: improve the ability to mock PointerWQ classes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2d90bd8fdb30705d05569d242fc0cc4e36a2c7db)

Conflicts:
src/common/WorkQueue.h: trivial resolution
src/librbd/AioImageRequestWQ.cc: file renamed

librbd: exclusive lock failures should bubble up to IO

(derived from commit 048d475127b600b6a40bd5e0c3a0daf8133294f4)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: directly inform IO work queue when locks are required

Due to lock dependency issues between librbd locks and the the thread
pool lock, it's impossible to directly determine if the lock is
required within the _void_dequeue method. Therefore, this state needs
to be internally tracked by the work queue.

(derived from commit 4a525671b3541a0a208dd039ac96f42bc8fca2cc)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clean up variable naming in IO work queue

(derived from commit 9d8ae624858beb6fdc8ee2674b8e8d9b9cf14cbe)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert ImageRequestWQ to template

(derived from commit b9569585fab5654b6c29552c5eb009dcddfaaa3b)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

os:kstore fix unittest for FiemapHole

kstore always return [0, object_size] regardless of offset and length

Signed-off-by: Ning Yao <yaoning@unitedstack.com>
(cherry picked from commit dddae89fa7956fdfca1c09910b4e5b42ecbd22cf)

client: skip lookupname if writing to unlinked file

When a client writes to an unlinked file, get_quota_root tries to
resolve the filename via an MDS lookupname op. The op always results in
-13 permission denied for path-restricted caps or -2 no such file or
directory otherwise. More importantly, the repeated lookupname ops slow
down buffered writes significantly.

Don't do the lookupname for unlinked files; use the root_ancentor's
quota instead.

Fixes: http://tracker.ceph.com/issues/20945
Backport: jewel, luminous
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 1d617eee2e8c32f7300cb2cb8dd37b3714a4699f)

Merge pull request #16059 from vumrao/wip-vumrao-20353

jewel: osd: Add heartbeat message for Jumbo Frames (MTU 9000)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15726 from asheplyakov/20314-bp-jewel

jewel: mon: fail to form large quorum; msg/async busy loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

librbd: reacquire lock should update lock owner client id

(derived from commit 21ce5e16703776cbae20ac981dff4ec7cc2bc9c8)

Fixes: http://tracker.ceph.com/issues/19957
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: add osd_hack_prune_past_intervals

Last ditch (but dangerous) method of reducing memory usage for
past_intervals, which can help very very unhappy clusters recovery.
A properly implemented version of this is in luminous.  This hacky
version was used successfully to recover multiple jewel-based clusters,
but is still only recommended for use when the OSD is otherwise unable
to recover.

This change is not cherry-picked from master because luminous implements
a more sophisticated version of this that changes the past intervals
representation entirely, but it is too invasive to backport.  This
workaround to prune just on startup should be sufficient for emergencies.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15719 from Vicente-Cheng/wip-20146-jewel

jewel: rgw: 'gc list --include-all' command infinite loop the first 1000 items

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15477 from smithfarm/wip-19767-jewel

jewel: rgw: Delete non-empty bucket in slave zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16015 from asheplyakov/19208-bp-jewel

jewel: osd: osd_internal_types: wake snaptrimmer on put_read lock, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15762 from smithfarm/wip-17385-jewel

jewel: ceph cli: Rados object in state configuring race

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15474 from smithfarm/wip-19559-jewel

jewel: osd: objecter full_try behavior not consistent with osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15947 from asheplyakov/20428-bp-jewel

jewel: osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid missing wakeup signal

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15475 from tchaikov/wip-16239-jewel

qa: add a sleep after restarting osd before "tell"ing it

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15473 from smithfarm/wip-20080-jewel

jewel: build/ops: rpm: set subman cron attributes in spec file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Thomas Serlin <tserlin@redhat.com>

Merge pull request #15460 from smithfarm/wip-17843-jewel

jewel: rbd: object-map: batch updates during trim operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15488 from dillaman/wip-20023-jewel

jewel: rbd: rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15602 from tchaikov/wip-20175-jewel

jewel: tests: decouple ceph_test_librbd_api from libceph-common

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16124 from smithfarm/wip-19174-jewel

jewel: rbd: rbd_clone_copy_on_read ineffective with exclusive-lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: add a field to store generic user data in the bucket index,
that can be populated/fetched via a configurable custom http header

Signed-off-by: Pavan Rallabhandi <prallabhandi@walmartlabs.com>
(cherry picked from commit abca7a86c3cfbb58fafb5d057d9d6f5017a53704)

Conflicts:
src/rgw/rgw_op.cc
Number of arguments to RGWPutObjProcessor_Multipart::do_complete has been changed in master with an extra field accounted_size added, which is not required in Jewel.
         Number of arguments for the processor->complete() has been changed in master, which is not required in Jewel.

src/rgw/rgw_op.h
A new member field `crypt_http_responses` is added in the class `RGWPutObj` in master, which is not required in Jewel.

src/rgw/rgw_rados.cc
         In the function RGWPutObjProcessor::complete() there is an extra argument to do_complete() in master, which is not required in Jewel.
         In the function RGWPutObjProcessor_Atomic::do_complete, a call to obj_op.write_meta has accounted_size, which is not required in Jewel.
         There is an extra argument accounted_size in RGWRados::Bucket::UpdateIndex::complete in master, which is not required in Jewel.
         In RGWRados::Bucket::UpdateIndex::complete, RGWObjEnt has been removed in master, which has to be retained in Jewel.
         In RGWRados::cls_obj_complete_op, user_data is added to the rgw_bucket_dir_entry_meta structure.
         In RGWRados::cls_bucket_list, the user_data field of RGWObjEnt is populated.

src/rgw/rgw_rados.h
         In UpdateIndex::complete(), remove_objs is of type rgw_obj_key in Jewel instead of rgw_obj_index_key
         Corrected various function declarations as appropriate.
         RGWPutObjProcessor_Multipart is not part of this file in Jewel.

src/rgw/rgw_rest_swift.cc
In RGWListBucket_ObjStore_SWIFT::send_response(), accounted_size is not required in Jewel.

src/rgw/rgw_common.h
Add user_data field in RGWObjEnt structure.

src/rgw/rgw_json_enc.cc
Add user_data field while dumping RGWObjEnt.