common,test: g_ceph_context->put() upon return

prior to this change, global_init() could create a new CephContext
and assign it to g_ceph_context. it's our responsibilty to release
the CephContext explicitly using cct->put() before the application
quits. but sometimes, we fail to do so.

in this change, global_init() will return an intrusive_ptr<CephContext>,
which calls `g_ceph_context->put()` in its dtor. this ensures that
the CephContext is always destroyed before main() returns. so the
log is flushed before _log_exp_length is destroyed.

there are two cases where global_pre_init() is called directly.
- ceph_conf.cc: g_ceph_context->put() will be called by an intrusive_ptr<>
  deleter.
- rgw_main.cc: global_init() is called later on on the success code
  path, so it will be taken care of.

Fixes: http://tracker.ceph.com/issues/17762
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb1cda96713b2ec0f6418c4cbe3d964c2020729c)

Conflicts:
src/ceph_fuse.cc
src/ceph_mgr.cc
src/global/global_init.cc
src/rgw/rgw_main.cc
src/test/compressor/test_compression.cc
src/test/compressor/test_compression_plugin.cc
src/test/compressor/test_compression_plugin_snappy.cc
src/test/compressor/test_compression_plugin_zlib.cc
src/test/compressor/test_compression_snappy.cc
src/test/compressor/test_compression_zlib.cc
src/test/erasure-code/TestErasureCode.cc
src/test/erasure-code/TestErasureCodeExample.cc
src/test/erasure-code/TestErasureCodeIsa.cc
src/test/erasure-code/TestErasureCodeJerasure.cc
src/test/erasure-code/TestErasureCodeLrc.cc
src/test/erasure-code/TestErasureCodePlugin.cc
src/test/erasure-code/TestErasureCodePluginIsa.cc
src/test/erasure-code/TestErasureCodePluginJerasure.cc
src/test/erasure-code/TestErasureCodePluginLrc.cc
src/test/erasure-code/TestErasureCodePluginShec.cc
src/test/erasure-code/TestErasureCodeShec.cc
src/test/erasure-code/TestErasureCodeShec_thread.cc
src/test/fio/fio_ceph_objectstore.cc
src/test/librados/misc.cc
src/test/mon/PGMap.cc
src/test/msgr/test_async_networkstack.cc
src/test/msgr/test_userspace_event.cc
src/test/objectstore/Allocator_test.cc
src/test/objectstore/BitAllocator_test.cc
src/test/objectstore/test_bluefs.cc
src/test/objectstore/test_bluestore_types.cc
src/test/objectstore/test_memstore_clone.cc
src/test/osd/TestPGLog.cc
src/test/rgw/test_http_manager.cc
src/test/rgw/test_rgw_compression.cc
src/test/test_mempool.cc
src/tools/rados/rados.cc

crushtool: s/exit(EXIT_FAILURE)/return EXIT_FAILURE/

so the destructor(s) can be called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e01b89ed6be6b99fec5c725f4bc5769b42468cac)

Conflicts:
src/tools/crushtool.cc - jewel does not have 17feefbcb3105553b763cb7ce123b20b77b95857

global/signal_handler: reset injected segv after test

~CephContext() => ~TypedSingletonWrapper() => ~MempoolObs =>
unregister_command() => ldout() << "unregister_command" =>
Log::submit_entry() => *(volatile int *)(0) = 0xdead;

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d932c8f2f23263924103a900714db82ee87f6eef)

Conflicts:
        src/log/Log.cc - jewel has "namespace log" instead of "namespace
                         logging" (trivial resolution)

test_cors.cc: fix the mem leak

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c8a3777203482cabf6739a8ba69b127df8697628)

Merge pull request #14195 from cbodley/wip-19353

jewel: rgw: use separate http_manager for read_sync_status

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14066 from asheplyakov/19321-bp-jewel

jewel: rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14064 from asheplyakov/19211-bp-jewel

jewel: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13842 from smithfarm/wip-19145-jewel

jewel: rgw: a few cases where rgw_obj is incorrectly initialized

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13837 from smithfarm/wip-19048-jewel

jewel: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13724 from asheplyakov/18626-bp-jewel

jewel: rgw: Use decoded URI when verifying TempURL

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14643 from smithfarm/wip-revert-14427

Wip revert 14427

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14602 from asheplyakov/19646-jewel

jewel: ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14449 from smithfarm/wip-test-doc-oversight

tests: fix oversight in yaml comment

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13606 from smithfarm/wip-19062-jewel

jewel: build/ops: enable build of ceph-resource-agents package on rpm-based os

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #14654 from smithfarm/wip-19461-jewel

jewel: admin ops: fix the quota section

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14680 from smithfarm/wip-19711-jewel

jewel: [test] test_notify.py: rbd.InvalidArgument: error updating features for image test_notify_clone2

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: rbd master/slave notify test should test active features

Fixes: http://tracker.ceph.com/issues/19692
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0dcba41cba96566d0b8da54cf0316d523b88ded2)

Merge pull request #14402 from shinobu-x/wip-17331-jewel

 jewel: ceph-disk: ceph-disk list reports mount error for OSD having mount options with SELinux context

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13865 from smithfarm/wip-19158-jewel

jewel: rgw: health check errors out incorrectly

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

doc: rgw: correct the quota section

Add the missing option and fix typo.

Fixes: http://tracker.ceph.com/issues/19397
Signed-off-by: Chu, Hua-Rong <hrchu@cht.com.tw>
(cherry picked from commit 51a88267f0d7f51aeb62092949b66b9f6c062e15)

Merge pull request #14383 from smithfarm/wip-19547-jewel

jewel: build/ops: rbdmap.service not included in debian packaging (jewel-only)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #14143 from smithfarm/wip-19355-jewel

jewel: rgw: when converting region_map we need to use rgw_zone_root_pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14136 from smithfarm/wip-19330-jewel

jewel: rgw: upgrade to multisite v2 fails if there is a zone without zone info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13872 from smithfarm/wip-19163-jewel

jewel: doc: radosgw-admin: add the 'object stat' command to usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13863 from smithfarm/wip-19155-jewel

jewel: rgw: typo in rgw_admin.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13779 from smithfarm/wip-18866-jewel

jewel: rgw: 'radosgw-admin sync status' on master zone of non-master zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Revert "osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag."

This reverts commit 2d68822c784eb4d62d3b0198ed4ec04404dbffb3.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "osdc/Objecter: resend RWORDERED ops on full"

This reverts commit f2474042ecd6560323673170c13f2cb964406e70.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14596 from gregsfortytwo/wip-17916-divergent

Wip 17916 divergent

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-disk: enable directory backed OSD at boot time

https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6
introduced a regression preventing directory backed OSD from starting at
boot time.

For device backed OSD the boot sequence starts with ceph-disk@.service
and proceeds to

    systemctl enable --runtime ceph-osd@.service

where the --runtime ensure ceph-osd@12 is removed when the machine
reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at
boot time.

However directory backed OSD solely rely on the ceph-osd@.service unit
to start at boot time and will therefore fail to boot.

The --runtime flag is selectively set for device backed OSD only.

Fixes: http://tracker.ceph.com/issues/19628
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit f425a127b7487d2093c8c943f0bcdec3d673d601)

Conflicts:
src/ceph-disk/ceph_disk/main.py: trivial: Jewel does not support
   OpenRC and other inits, hence no corresponding 'elif'

pglog: require users set a config option before ignoring divergent_priors

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: pglog: clean up divergent_priors off disk when running; don't assert on startup

Fixes: http://tracker.ceph.com/issues/17916
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14492 from gregsfortytwo/wip-jewel-snaptrim

Backport snap trimming improvements to Jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'jewel' into wip-jewel-snaptrim

PendingReleaseNotes: discuss snap trim improvements

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

PrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine

Rather than blocking the main op queue, just pause for that amount of
time between state machine cycles.

Also, add osd_snap_trim_sleep to a few of the thrasher yamls.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2ed7759cfeb03e71f0fbd98fe7c2db2bb741861c)

Conflicts:
src/osd/PrimaryLogPG.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 67336454a4cee66522bc0ca01b2c58b8960f75ec)

Conflicts:
qa/suites/rados/thrash/thrashers/default.yaml
qa/suites/rados/thrash/thrashers/pggrow.yaml
src/osd/OSD.h
src/osd/ReplicatedPG.cc
src/osd/ReplicatedPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

rados: check that pool is done trimming before removing it

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4aebf59d906fa3e03d21bdac182f89fe3cd4c802)
(cherry picked from commit 34398c29b3c57f00d932cf96570f882dce64a82b)

osd/ReplicatedPG: limit the number of concurrently trimming pgs

This patch introduces an AsyncReserver for snap trimming to limit the
number of pgs on any single OSD which can be trimming, as with backfill.
Unlike backfill, we don't take remote reservations on the assumption
that the set of pgs with trimming work to do is already well
distributed, so it doesn't seem worth the implementation overhead to get
reservations from the peers as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 21cc515adfb225ba70f1d80b1b76f0345c214c22)

Conflicts:
src/osd/PrimaryLogPG.cc
src/osd/PrimaryLogPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 68ea24396ca6450d4d8042a7c5f51306b7d199fa)
(cherry picked from commit c7176b869898c870b56b1762958652d801af4c4c)

Conflicts: Many. As evidenced by involving two distinct patches
in this one commit, it wasn't a clean backport.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13834 from smithfarm/wip-18969-jewel

jewel: rgw: Change loglevel to 20 for 'System already converted' message

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13833 from smithfarm/wip-18908-jewel

jewel: rgw: the swift container acl does not support field .ref

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13214 from ovh/bp-osd-updateable-throttles-jewel

jewel: osd: allow client throttler to be adjusted on-fly, without restart

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14326 from shinobu-x/wip-15025-jewel

jewel: osd: new added OSD always down when full flag is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13874 from smithfarm/wip-19171-jewel

jewel: doc: rgw S3 create bucket should not do response in json

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #13492 from shinobu-x/wip-18516-jewel

jewel: build/ops: "osd marked itself down" will not recognised if host runs mon + osd on shutdown/reboot

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13254 from shinobu-x/wip-14609-jewel

jewel: common: radosstriper: protect aio_write API from calls with 0 bytes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13489 from shinobu-x/wip-18955-jewel

jewel: ceph-disk: bluestore --setgroup incorrectly set with user

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14070 from smithfarm/wip-19339-jewel

jewel: tests: dummy suite fails in OpenStack

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14329 from smithfarm/wip-19493-jewel

jewel: ceph-disk: Racing between partition creation & device node creation

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #14427 from smithfarm/wip-19140-jewel

jewel: osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14324 from shinobu-x/wip-19371-jewel

jewel: common: monitor creation with IPv6 public network segfaults

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14112 from shinobu-x/wip-19192-jewel

jewel: tools: brag fails to count "in" mds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14150 from smithfarm/wip-18823-jewel

jewel: tests: run-rbd-unit-tests.sh assert in lockdep_will_lock, TestLibRBD.ObjectMapConsistentSnap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14152 from smithfarm/wip-18893-jewel

jewel: librbd: Incomplete declaration for ContextWQ in librbd/Journal.h

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14154 from smithfarm/wip-18948-jewel

jewel: tests: additional rbd-mirror test stability improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14148 from smithfarm/wip-18778-jewel

jewel: rbd: rbd --pool=x rename y z does not work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14083 from smithfarm/wip-19357-jewel

jewel: rbd: systemctl stop rbdmap unmaps all rbds and not just the ones in /etc/ceph/rbdmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13154 from smithfarm/wip-18496-jewel

jewel: librbd: Possible deadlock performing a synchronous API action while refresh in-progress

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13244 from smithfarm/wip-18775-jewel

jewel: rbd: qemu crash triggered by network issues

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13809 from asheplyakov/18321-bp-jewel

jewel: librbd: remove image header lock assertions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13107 from smithfarm/wip-18669-jewel

jewel: tests: [  FAILED  ] TestLibRBD.ImagePollIO in upgrade:client-upgrade-kraken-distro-basic-smithi

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13585 from asheplyakov/jewel-bp-16585

jewel: msg: set close on exec flag

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14371 from tchaikov/wip-19429-jewel

jewel: tests: clone workunit using the branch specified by task

Nathan Cutler <ncutler@suse.com>

Merge pull request #14325 from shinobu-x/wip-18619-jewel

jewel: osd: degraded and misplaced status output inaccurate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14236 from smithfarm/wip-19392-jewel

jewel: mon: remove bad rocksdb option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14181 from mslovy/wip-19394-jewel

jewel: osd: bypass readonly ops when osd full

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14113 from shinobu-x/wip-19319-jewel

 jewel: cli: RadosImport::import should return an error if Rados::connect fails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #14047 from asheplyakov/reindex-on-pg-split

jewel: osd: reindex properly on pg log split

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14044 from mslovy/wip-19311-jewel

jewel: core: os/filestore: fix clang static check warn use-after-free

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13932 from asheplyakov/18911-bp-jewel

jewel: rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13831 from jan--f/wip-19206-jewel

jewel: fs: Invalid error code returned by MDS is causing a kernel client WARNING

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13827 from tchaikov/wip-19185-jewel

jewel: osd: ReplicatedPG: try with pool's use-gmt setting if hitset archive not found

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13788 from shinobu-x/wip-18235-jewel

jewel: core: os/filestore/HashIndex: be loud about splits

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13786 from shinobu-x/wip-19129-jewel

jewel: build/ops: ceph-base package missing dependency for psmisc

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13732 from liewegas/wip-19119-jewel

jewel: doc: PendingReleaseNotes: note about 19119

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13541 from shinobu-x/wip-18929-jewel

 jewel: osd: restrict want_acting to up+acting on recovery completion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13477 from asheplyakov/jewel-bp-18951

jewel: osd: --flush-journal: sporadic segfaults on exit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13261 from shinobu-x/wip-18587-jewel

jewel: mon: OSDMonitor: make 'osd crush move ...' work on osds

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

tests: fix oversight in yaml comment

When the file was copied from the hammer version, the word "hammer"
was not changed to "infernalis".

Signed-off-by: Nathan Cutler <ncutler@suse.com>
This cannot be cherry-picked from master because the test has been dropped.

10.2.7

osdc/Objecter: resend RWORDERED ops on full

Our condition for respecting the FULL flag is complex, and involves
the WRITE | RWORDERED flags vs the FULL_FORCE | FULL_TRY flags.  Previously,
we could block a read bc of RWORDRED but not resend it later.

Fix by capturing the complex condition in a respects_full() bool and using
it both for the blocking-on-send and resending-on-possibly-notfull-later
checks.

Fixes: http://tracker.ceph.com/issues/19133
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c4b73f19a7be13ff412eef804efcd8c18ed4dae6)

osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 07b2a22210e26eac1b2825c30629788da05e5e12)

ceph-disk: Populate mount options when running "list"

Also tidy up by moving duplicated code into a function

Fixes: http://tracker.ceph.com/issues/17331
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 7943ab2e01e24f2dfc5b6f1d3ffdc8a49e01af45)

debian: replace SysV rbdmap with systemd service

Stop shipping /etc/init.d/rbdmap in the Debian packages. Ship the
rbdmap.service systemd unit file instead.

The corresponding change has already been made for RPMs, in
9224ac2ad25f7d017916f58b642c0ea25305c3e5.

For Upstart-based systems (eg Ubuntu Trusty), the Debian packages
already contain rbdmap.conf.

(This gets us a tiny bit closer to being able to remove the rbdmap SysV
script from our tree entirely.)

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 839807118dda2fb4d57ed9d50ec46e3ee0e2820a)

Conflicts:
    debian/ceph-common.install (retain /etc/init.d/rbdmap so jewel users can choose sysvinit or systemd)
    debian/rules (retain /etc/init.d/rbdmap so jewel users can choose sysvinit or systemd)

qa/tasks/workunit.py: use "overrides" as the default settings of workunit

otherwise the settings in "workunit" tasks are always overridden by the
settings in template config. so we'd better follow the way of how
"install" task updates itself with the "overrides" settings: it uses the
"overrides" as the *defaults*.

Fixes: http://tracker.ceph.com/issues/19429
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 47080150a17d238f38d9da824d227393ad767aad)

tasks/workunit.py: specify the branch name when cloning a branch

c1309fb failed to specify a branch when cloning using --depth=1, which
by default clones the HEAD. and we can not "git checkout" a specific
sha1 if it is not HEAD, after cloning using '--depth=1', so in this
change, we dispatch "tag", "branch", "HEAD" using three Refspec classes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 9ca7ccf5f1739f731da8bf31260594aea3a2932d)

Conflicts:
qa/tasks/workunit.py (trivial resolution)

tasks/workunit.py: when cloning, use --depth=1

Help avoid killing git.ceph.com.  A depth 1 clone takes about
7 seconds, whereas a full one takes about 3:40 (much of it
waiting for the server to create a huge compressed pack)

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit c1309fbef300a062138ac40eb5d3e5081b833072)

build/ops: rpm: move $CEPH_EXTRA_CONFIGURE_ARGS to right place

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: explicitly provide --with-ocf to configure

Fixes: http://tracker.ceph.com/issues/19546
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(Note: This cannot be cherry-picked because master uses cmake, but
the fix does bring the jewel spec file into better alignment its master
counterpart, at least as far as this one little bit is concerned.)

rgw: use separate http_manager for read_sync_status

concurrent users of read_sync_status() use different cr managers, when get_resource must
use http_manager related to the cr manager.

Fixes: http://tracker.ceph.com/issues/19236
Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
(cherry picked from commit c412024889f8995d98096ac863bafee71624bd70)

rgw: pass cr registry to managers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ef4d7eab11fb5d2a41c9c28b9eb8b075aaff0d05)

 Conflicts:
src/rgw/rgw_rados.cc: removed ref to RGWSyncLogTrimThread (not backported)

rgw: use separate cr manager for read_sync_status

RGWCoroutinesManager::run() is not reentrant, so concurrent users of
read_sync_status() must use different managers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6b1e40d7a21c549b55e6576ec56973c8d3c092d1)

rgw: change read_sync_status interface

RGWDataSyncStatusManager::read_sync_status() now operates on the given
parameter, rather than its internal member variable. this allows
multiple concurrent readers, which is needed for the rest interface

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b7cd4e0e8b879b5e528de75bea3307585b96cbf2)

rgw: don't ignore ENOENT in RGWRemoteDataLog::read_sync_status()

rest handlers for sync status need to return ENOENT errors. the only
other callers are in radosgw-admin, so the ENOENT errors are ignored at
those call sites instead

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2cc533b30c0f23c0750ea8d02c51b3b3d3b4821a)

PendingReleaseNotes: warning about 'osd rm ...' and #19119

See http://tracker.ceph.com/issues/19119

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit be96003c464481d8e84825178d600234a0d64d22)

ceph-disk: Adding retry loop in get_partition_dev()

There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/<device>.

It appear that waiting a very short is usually enough to get the partition beein populated.

Analysis:
update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and
the actual creation on the /sys/block/<device>/* entrypoint.
On our CI that race occurs pretty often but trying to reproduce it locally never been possible.

This patch is almost a workaround rather than a fix to the real problem.
It offer retrying after a very short to be make a chance the device to appear.
This approach have been succesful on the CI.

Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs.

A typical output from the build running on a CI with that code.
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda
get_partition_dev: Found partition 2 for /dev/sda after 1 tries
        get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid

fixes: #19428

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 93e7b95ed8b4c78daebf7866bb1f0826d7199075)

ceph-disk: Reporting /sys directory in get_partition_dev()

When get_partition_dev() fails, it reports the following message :
    ceph_disk.main.Error: Error: partition 2 for /dev/sdb does not appear to exist
The code search for a directory inside the /sys/block/get_dev_name(os.path.realpath(dev)).

The issue here is the error message doesn't report that path when failing while it might be involved in.

This patch is about reporting where the code was looking at when trying to estimate if the partition was available.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 413c9fcfbe8e6ab33d73b8428090ccacc33c5d15)

osd: don't share osdmap with objecter when preboot

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit a5a3644eecc49b4eea890c6999fe87536495dcbe)

osd: Calculate degraded and misplaced more accurately

Calculate num_object_copies based on the larger of pool size,
up set size and acting set size.

Calculate num_objects_degraded as the difference between num_object_copies
and all copies found on acting set and backfilling up set OSDs.

Calculate num_objects_misplaced as all copies on acting set OSDs not in up set
less copies that have been backfilled to up set OSDs.

Fixes: http://tracker.ceph.com/issues/18619
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8423bc40759cca137f61e7b755411719a84369d4)

common: fix segfault in public IPv6 addr picking

sockaddr is only 16 bytes big, so declaring net as sockaddr
and then casting to sockaddr_in6 in case of IPv6 cannot
work.

using sockaddr_storage works for both IPv4 and IPv6, and is
used in other code parts as well.

note that the tests did not find this issue as they declared
the bigger structs and casted the references to (sockaddr *)

Fixes: http://tracker.ceph.com/issues/19371
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
(cherry picked from commit ae2ee3d3835fe25b35eeb1a841ee5234cd69eb65)

Merge pull request #14230 from linuxbox2/wip-jewel-expand-argv

jewel: rgw_file: expand argv

Reviewed-by: Casey Bodley <cbodley@redhat.com>