ReplicatedPG::hit_set_setup: fix hit_set_remove_all call

We only want to do it if the pool config changed AND we are
primary && active.

fd38902dd4693b9b72f7532833e78f5df2c9efa5 partially fixed a related
bug.  This should be backported along with it.

Fixes: 13192
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 36e4a80c39f7daab2a35efc04650ae8b7b11c4e0)

osd/: assert in HitSet constructor if type is TYPE_NONE as well

Otherwise impl is unpopulated and we risk segfaults in several
methods.  It also seems like it would always indicate a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8e5a801b43babf751b619ed9606d7be7e5b12837)

osd: only remove all the hit set objects on primary osd

Only the primary PG is allowed to remove all the hit set objects. And
the PG should be in the active or peered states.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit fd38902dd4693b9b72f7532833e78f5df2c9efa5)

osd: implement hit_set_remove_all

When hit set is not configured on startup or on a change, remove all
previous hitsets.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit be28319bf3dc54b4b6f400d83405a29facfe3bd4)
Conflicts:
        src/osd/ReplicatedPG.cc
          pass the new argument of `p->using_gmt` to get_hit_set_archive_object()

mon: disable gmt_hitset if not supported

the gmt_hitset is enabled by default in the ctor of pg_pool_t, this
is intentional. because we want to remove this setting and make
gmt_hitset=true as a default in future. but this forces us to
disable it explicitly when preparing a new pool if any OSD does
not support gmt hitset.

Fixes: #12968
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e48cec3dc93b3988dcd8924933deb1b3a43e1d0f)

Conflicts:
src/mon/OSDMonitor.cc
remove unrelated hunk

osd: do not let OSD_HITSET_GMT reuse the feature bit

* to ease the backport to hammer

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5a4f6a866bb675195ad83c38fd23238d078c78ed)

Conflicts:
src/include/ceph_features.h
minor changes to resolve conflicts

osd: Decode use_gmt_hitset with a unique version

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2bc5a48f4c5d3667213be3a7b5a0e0f5ef9daf4f)

Conflicts:
src/osd/osd_types.cc
minor changes to resolve conflicts

mon: print use_gmt_hitset in "ceph osd pool get"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc2bcf760f2d2f20fc4a2fa78ba52475a23e55bf)

Conflicts:
src/mon/OSDMonitor.cc
use the old way to dump pool info

mon: add "ceph osd pool set $pool use_gmt_hitset true" cmd

allow "ceph osd pool set $pool use_gmt_hitset <true|1>" as long as
the cluster supports gmt hitset.

Fixes: #9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 03a1a3cf023a9aeb2fa26820e49e5efe3f3b3789)

osd: use GMT time for the object name of hitsets

* bump the encoding version of pg_hit_set_info_t to 2, so we can
  tell if the corresponding hit_set is named using localtime or
  GMT
* bump the encoding version of pg_pool_t to 20, so we can know
  if a pool is using GMT to name the hit_set archive or not. and
  we can tell if current cluster allows OSDs not support GMT
  mode or not.
* add an option named `osd_pool_use_gmt_hitset`. if enabled,
  the cluster will try to use GMT mode when creating a new pool
  if all the the up OSDs support GMT mode. if any of the
  pools in the cluster is using GMT mode, then only OSDs
  supporting GMT mode are allowed to join the cluster.

Fixes: #9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42f8c5daad16aa849a0b99871d50161673c0c370)

 Conflicts:
src/include/ceph_features.h
src/osd/ReplicatedPG.cc
src/osd/osd_types.cc
src/osd/osd_types.h
fill pg_pool_t with default settings in master branch.

Merge pull request #5885 from Abhishekvrshny/wip-13034-hammer

osd: copy-from doesn't preserve truncate_{seq,size}

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6192 from dachary/wip-13401-hammer

mon: fix crush testing for new pools

Reviewed-by: Sage Weil <sage@redhat.com>

crush/CrushTester: test fewer inputs when running crushtool

If there are a lot of crush rules (say, 100) then the test can
take a long time.  100 values per rule should be enough to catch
most issues.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0f82f461b33d93d868e185912a2c7e4074d06900)

Conflicts:
src/crush/CrushTester.cc
  in hammer the crushtool validation is via a shell
       and not via an internal subprocess utility

tests: update to match crushmap validation message

http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 21a1e75d8a7bad89a48cd9d36902c5d609be5015)

Conflicts:
qa/workunits/cephtool/test.sh
  the test structure in hammer is different, adapt
  the relevant test.

mon/OSDMonitor: fix crush injection error message

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1b3090d50e5bd5ca3e6e396b23d2d9826896c718)

Conflicts:
src/mon/OSDMonitor.cc
  hammer has a separate error message for EINTR
  which is in the context of the fix for error message

mon/OSDMonitor: only test crush ruleset for the newly created pool

Otherwise, we test *all* crush rules.. which might be a lot, and
which is a big waste of time and effort.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 524b0bdcc45c2f4b95f2239c988e93250f337f3d)

Conflicts:
src/mon/OSDMonitor.cc : trivial resolution

crush/CrushTester: allow testing by ruleset

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aa238e5ed50f44a94caf84567267e4f6be8732a2)

Conflicts:
src/crush/CrushTester.cc
          in hammer the crushtool validation is via a shell
  and not via an internal subprocess utility
src/tools/crushtool.cc
  ceph_argparse_withint is preferred to ceph_argparse_witharg

Merge pull request #5887 from Abhishekvrshny/wip-13044-hammer

LibCephFS.GetPoolId failure

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6172 from dachary/wip-13354-hammer

qa/workunits/cephtool/test.sh: don't assume crash_replay_interval=45

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/workunits/cephtool/test.sh: don't assume crash_replay_interval=45

e.g., it's 5 in teuthology's ceph.conf.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 80b7237e5e74d12b9d8b1f96ea535c96bdff9c6f)

Merge pull request #6156 from dachary/wip-13170-hammer

doc: update docs to point to download.ceph.com

Reviewed-by: Abhishek Varshney <abhishek.varshney@flipkart.com>

doc: remove mention of ceph-extra as a requirement

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a6f07e9fa3bd5e50a2d8cfa387b44a39d383fa65)

doc: remove ceph-extras

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c0ef84fd22aac58b6ec8e58eb4f8dffed306d39e)

doc: correct links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 387d7800359154950431d0984c756f43f21dd9b4)

doc: Added "Hammer" in the list of major releases.

Fixes: #12771
Signed-off-by: ritz303 <ritz_303@yahoo.com>
(cherry picked from commit 582f0f64455079290ad3b4ae7338b716e170911f)

osd/ReplicatedPG: tolerate promotion completion with stopped agent

We may start a promotion, then get a pool update that disables the
agent, and then complete it.  We should not segfault in this case.

Fixes: #13190
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd9ce66e8b76787f69c5e337332d57aa72c8b8d5)

Merge pull request #5715 from dachary/wip-12592-hammer

RGW returns requested bucket name raw in Bucket response header

Merge pull request #5719 from dachary/wip-12855-hammer

segmentation fault when rgw_gc_max_objs > HASH_PRIME

Merge pull request #5720 from dachary/wip-12854-hammer

the arguments 'domain' should not be assigned when return false

Merge pull request #5930 from liewegas/wip-11798-hammer

upstart: limit respawn to 3 in 30 mins (instead of 5 in 30s)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5908 from liewegas/wip-simple-connect-hammer

Pipe: Drop connect_seq increase line

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

upstart: limit respawn to 3 in 30 mins (instead of 5 in 30s)

It may take tens of seconds to restart each time, so 5 in 30s does not stop
the crash on startup respawn loop in many cases.  In particular, we'd like
to catch the case where the internal heartbeats fail.

This should be enough for all but the most sluggish of OSDs and capture
many cases of failure shortly after startup.

Fixes: #11798
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eaff6cb24ef052c54dfa2131811758e335f19939)

Pipe: Drop connect_seq increase line

Revert commit 0fc47e267b6f8dcd4511d887d5ad37d460374c25.

When accepting and "connect.connect_seq == existing->connect_seq",
existing->state maybe STATE_OPEN, STATE_STANDBY or STANDY_CONNECTING.
This commit only fix partial problem and want to assert
"(existing->state == STATE_CONNECTING)".

So later we added codes to catch
"(existing->state == STATE_OPEN || existing->state == STATE_STANDBY)"
before asserting.

Backport: dumpling, firefly, giant

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
(cherry picked from commit 67225cb3ee1f6d274a02293724942bdb25cec8ca)

Merge pull request #5767 from dachary/wip-12597-hammer

Crash during shutdown after writeback blocked by IO errors

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5687 from liewegas/wip-hammer-feature-hammer

include/ceph_features: define HAMMER_0_94_4 feature

Merge pull request #5892 from ceph/wip-13060-hammer

osd: allow peek_map_epoch to return an error

Reviewed-by: David Zafman <dzafman@redhat.com>

osd/PG: peek_map_epoch: skip legacy PGs if infos object is missing

- pg is removed
- osd is stopped before pg is fully removed
- on restart, we ignore/skip the pg because its epoch is too old
- we upgrade to hammer and convert other pgs, skipping this one, and then
  remove the legacy infos object
- hammer starts, tries to parse the legacy pg, and fails because the infos
  object is gone, crashing.

The fix is to continue ignoring the zombie pg.

Fixes: #16030
Signed-off-by: Sage Weil <sage@redhat.com>

osd: allow peek_map_epoch to return an error

Allow PG::peek_map_epoch to return an error indicating the PG
should be skipped.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f15d9585edc5a12ac2d076951076247253b897c2)

[fixed *pepoch default of 0]
[fixed other return paths in peek_map_epoch]

crypto: fix unbalanced ceph::crypto::init/ceph::crypto:shutdown

we may create a CephContext without calling common_init_finish(), then
delete the CephContext. In this case, ceph::crypto:init() is not called,
so CephContext::~CephContext() should not call ceph::crypto::shutdown().

Fixes: #12598
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 98a85ec6464d8ec3fc7f0c647ac97c8cf002ebe2)

ReplicatedPG,Objecter: copy_get should include truncate_seq and size

Otherwise, we break CephFS over cache tiers.

Fixes: #12551
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 6f9ee7961eee9ee3a61a07cbe0d8d289ee98fa9a)

Conflicts:
src/osd/ReplicatedPG.cc

Merge pull request #5769 from dachary/wip-12850-hammer

Crash during TestInternal.MultipleResize

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5768 from dachary/wip-12849-hammer

[  FAILED  ] TestLibRBD.BlockingAIO

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5697 from tchaikov/wip-12638-hammer

mon: add a cache layer over MonitorDBStore

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5381 from dachary/wip-12499-hammer

ceph-fuse 0.94.2-1trusty segfaults / aborts

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5757 from dachary/wip-12836-hammer

WBThrottle::clear_object: signal on cond when we reduce throttle values

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5759 from dachary/wip-12841-hammer

recursive lock of md_config_t (0)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5761 from dachary/wip-12843-hammer

long standing slow requests: connection->session->waiting_for_map->connection ref cycle

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5762 from dachary/wip-12844-hammer

osd suicide timeout during peering - search for missing objects

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5763 from dachary/wip-12846-hammer

osd/PGLog.cc: 732: FAILED assert(log.log.size() == log_keys_debug.size())

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5764 from dachary/wip-12847-hammer

common: do not insert emtpy ptr when rebuild emtpy bufferlist

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5373 from dachary/wip-12489-hammer

pg_interval_t::check_new_interval - for ec pool, should not rely on min_size to determine if the PG was active at the interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5383 from dachary/wip-12504-hammer

rest-bench common/WorkQueue.cc: 54: FAILED assert(_threads.empty())

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5765 from dachary/wip-12883-hammer

cache agent is idle although one object is left in the cache

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5754 from dachary/wip-12588-hammer

Change radosgw pools default crush ruleset

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5377 from dachary/wip-12396-hammer

register_new_pgs() should check ruleno instead of its index

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5758 from dachary/wip-12839-hammer

Mutex Assert from PipeConnection::try_get_pipe

tests: tiering agent and proxy read

Verify that an object promoted to a cache tier because of a proxy read
is evicted as expected.

http://tracker.ceph.com/issues/12673 Refs: #12673

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 7924231930732bd297d3bd034c8295e96cb81088)

osd: trigger the cache agent after a promotion

When a proxy read happens, the object promotion is done in parallel. The
agent_choose_mode function must be called to reconsider the situation
to protect against the following scenario:

  * proxy read
  * agent_choose_mode finds no object exists and the agent
    goes idle
  * object promotion happens
  * the agent does not reconsider and eviction does not happen
    although it should

http://tracker.ceph.com/issues/12673 Fixes: #12673

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit e1f58feb9b1d20b72f2eb2eefdea5982e0cddccd)

Merge pull request #5276 from dachary/wip-11824-hammer

implicit erasure code crush ruleset is not validated

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5382 from dachary/wip-12500-hammer

segfault launching ceph-fuse with bad --name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5367 from dachary/wip-12311-hammer

read on chunk-aligned xattr not handled

Reviewed-by: Sage Weil <sage@redhat.com>

lockdep: allow lockdep to be dynamically enabled/disabled

librbd test cases attempt to enable lockdep coverage via the librados
API.  Use a configuration observer to register/unregister lockdep
support.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af0cade0293be22e74a3293765676c7b227b06c4)

tests: librbd API test cannot use private md_config_t struct

Remove all depencencies on md_config_t and instead use librados API
methods to get/set configuration values.

Fixes: #12479
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2c51aadb447d3447b5dace19c5cb6268e94247c8)

tests: ensure old-format RBD tests still work

Override the RBD default image format back to version 1
to ensure tests properly cover the old format.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3b95edb5bd373bd05f83516eccfd30a6b5cf4693)

librados_test_stub: implement conf get/set API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4d03c664f2a9362009c7e3077ed65e2989f0f21b)

crypto: use NSS_InitContext/NSS_ShutdownContex to avoid memory leak

Switched to context-aware NSS init/shutdown functions to avoid conflicts
with parent application.  Use a reference counter to properly shutdown the
NSS crypto library when the last CephContext is destroyed.  This avoids
memory leaks with the NSS library from users of librados.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9fa0112dd9286178da1d76020158503b2062d252)

auth: use crypto_init_mutex to protect NSS_Shutdown()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit cf058174fe6be26129eb52f4fb24d0f7146ef7af)

auth: reinitialize NSS modules after fork()

Fixes: #11128
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4c24d0cc074462ae258b5cf901cd884bb4f50a53)

librbd: prevent race condition between resize requests

It was possible that the same resize request could be sent twice
if a completed resize op started a newly created resize op while
it was also being concurrently started by another thread.

Fixes: #12664
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6ec431bac1cf9446f3af90209d193dfcf003f2c5)

librbd: Add a paramter:purge_on_error in ImageCtx::invalidate_cache().

If bh_write met error, it will try again. For closing image, if met this
issue, it will trigger a assert:
>>2015-02-03 15:22:49.198292 7ff62d537800 -1 osdc/ObjectCacher.cc: In
function 'ObjectCacher::~ObjectCacher()' thread 7ff62d537800 time
>>2015-02-03 15:22:49.195927osdc/ObjectCacher.cc: 551: FAILED
>>assert(i->empty())

Now add purge_on_error, when shutdown_cache it set true.
In ImageCtx::invalidate_cache, if met error and purge_on_error is true,
purge the dirty bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 35def5c81f7fc83d55d18320e4860c6a63d4c7f5)

Conflicts:
src/librbd/ImageCtx.cc : trivial resolution

librbd: Remvoe unused func ImageCtx::read_from_cache.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 101440a41253680770f94bc380af7072c7adaebf)

osdc: clean up code in ObjectCacher::Object::map_write

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 9f80c2909ace09cd51c24b49c98a093e0e864dca)

osdc: Don't pass mutex into ObjectCacher::_wait_for_write.

Because the mutex is the same as ObjectCacher::lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit d7cf7aeea5cba1ffa8e51ff1ad2424b1ec161a12)

osdc: After write try merge bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 1a48a8a2b222e41236341cb1241f0885a1b0b9d8)

osdc: Make last missing bh to wake up the reader.

Avoid wakeup early and wait again.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit d582bda090b3339d03e25b0e6d0971ad0267f476)

osdc: For trust_enoent is true, there is only one extent.

Now the judgement only in conditon which will return -ENOENT.
But o->exists don't depend on the extent size. It only depend on
trust_enoent. So move this judgement at the first of _readx().
Make this bug ASAP occur.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 2449fddc13b5ce8bbf1bb4ecaa5d6937f54e54d1)

osdc: In _readx() only no error can tidy read result.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 540346d4a901d8041c9fd74641c98cdfd2e1ab32)

Merge pull request #5223 from SUSE/wip-12305-hammer

ceph.spec.in: running fdupes unnecessarily

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5716 from dachary/wip-12851-hammer

Ensure that swift keys don't include backslashes

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #5717 from dachary/wip-12591-hammer

rgw: create a tool for orphaned objects cleanup

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #5755 from dachary/wip-12589-hammer

ceph-disk zap should ensure block device

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5732 from ceph/wip-11455-hammer

rgw: init some manifest fields when handling explicit objs

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5721 from dachary/wip-12853-hammer

RGW Swift API: X-Trans-Id header is wrongly formatted

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5498 from ceph/wip-12432-hammer

rgw: HTTP return code is not being logged by CivetWeb

Reviewed-by: Loic Dachary <ldachary@redhat.com>

OSD: break connection->session->waiting message->connection cycle

Fixes: #12338
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7f15e22be291df139c85909b96b2366f6dbadf37)

osd/PGLog: dirty_to is inclusive

There are only two callers of mark_dirty_to who do not pass max,
and they are both in the merge_log extending tail path.  In that
case, we want to include the last version specified in the log
writeout.  Fix the tail extending code to always specify the
last entry added, inclusive.

Fixes: #12652
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f0ca14df0641daa04eee39d98d8bd0faf46e4e6d)

common: fix code format

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit 15e5ebe14787bbbc6b906d72ccd2d418d762cfad)

test: add test case for insert empty ptr when buffer rebuild

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit 2d2f0eb338906742d516a2f8e6d5d05505be8f47)

common: fix insert empty ptr when bufferlist rebuild

Fixes: #12775
Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit fb1b6ddd7bfe40d4a38d5ec59b1c3f59f2a83f23)

Conflicts:
src/common/buffer.cc : because invalidate_crc did not exist
        in hammer

Merge pull request #5527 from SUSE/wip-12585-hammer

OSD crash creating/deleting pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: copy the RecoveryCtx::handle when creating a new RecoveryCtx instance from another one

Fixes: 12523
Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
(cherry picked from commit 6ab9efe799b947f2ef6113df8cc542c73760107d)

config: skip lockdep for intentionally recursive md_config_t lock

lockdep can't handle recursive locks, resulting in false positive
reports for certain set_val_or_die() calls, like via
md_config_t::parse_argv() passed "-m".

Fixes: #12614
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 415226995ebe841cc9e8639a3233150a75aecaed)

osd: Keep a reference count on Connection while calling send_message()

Fixes: #12437
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a140085f467889f2743294a3c150f13b62fcdf51)

WBThrottle::clear_object: signal if we cleared an object

Fixes: #12223
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8447b08d60d5cf7dce20f8b839cc2cb5a93782de)

ceph-disk: always check zap is applied on a full device

zap only makes sense on a full device and this is verified for

    ceph-disk prepare --zap-disk

but not for

    ceph-disk zap

Unify the two and add a test to check that it fails as expected when
trying to zap a directory.

http://tracker.ceph.com/issues/11272 Fixes: #11272

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 8e1bcf21f090fc98ac9b495d76614e3012f5872a)

Conflicts:
src/test/ceph-disk.sh : trivial resolution

librados: Make librados pool_create respect default_crush_ruleset

Make pool_create in librados use the osd_pool_default_crush_replicated_ruleset

Fixes: #11640
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
(cherry picked from commit da96a89033590277460aef1c80f385bd93d625e1)

Merge pull request #5551 from ceph/wip-corpus-hammer

ceph-object-corpus: add 0.94.2-207-g88e7ee7 hammer objects

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-object-corpus: add 0.94.2-207-g88e7ee7 hammer objects

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: init some manifest fields when handling explicit objs

Fixes: #11455
When dealing with old manifest that has explicit objs, we also
need to set the head size and head object correctly so that
code that relies on this info doesn't break.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit d7bf8cb594e5276d1c80544f5ec954d52b159750)

mon: test the crush ruleset when creating a pool

We want to fix the following scenario:

 * an erasure code plugin (or another part of the code) creates a
   ruleset
 * the ruleset crashes during mapping (for whatever reason)
 * ceph osd pool create uses the bugous ruleset
 * the monitors try to do mapping a crash

Having a bugous ruleset in the crush map is very difficult prevent. The
catastrophic event of using it with a newly created pool can however be
prevented by calling the CrushTester just before creating the pool and
after all implicit or explicit crush ruleset creation happened.

http://tracker.ceph.com/issues/11814 Fixes: #11814

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit f1e86be589803596e86acc964ac5c5c03b4038d8)

Conflicts:
src/test/mon/osd-crush.sh
          removed the run_mon because in hammer it is shared
          between all tests
        src/mon/OSDMonitor.cc
          prepare_new_pool changed stringstream to *ostream