osd/PrimaryLogPG: request osdmap update in the right block

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit afc6624f768ea3c6e2d155122797db5cce8836f7)

Merge pull request #17796 from jdurgin/wip-21428-luminous

osd: request new map from PG when needed

Reviewed-by: Sage Weil <sage@redhat.com>

osd: request new map from PG when needed

The fast dispatch refactor in 3cc48278bf0ee5c9535d04b60a661f988c50063b
eliminated the osdmap subscription in the ms_fast_dispatch path, which
meant ops could reach a PG without having the latest map. In a cluster
with few osdmap updates, where the monitor fails to send a new map to
an osd (it tries one random osd), this can result in indefinitely
blocked requests.

Fix this by adding an OSDService mechanism for scheduling a new osdmap
subscription request.

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit dd33360e5a4ed06228af552edadee88e66c5aba9)

Merge pull request #17724 from theanalyst/wip-21374-luminous

luminous: incorrect erasure-code space in command ceph df

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17745 from tchaikov/wip-luminous-20529

luminous: rocksdb,cmake:  build portable binaries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17635 from smithfarm/wip-21342-luminous

luminous: ceph mgr versions shows active mgr as Unknown

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17634 from smithfarm/wip-21341-luminous

luminous: mon/OSDMonitor: deleting pool while pgs are being created leads to assert(p != pools.end) in update_creating_pgs()

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17624 from tchaikov/wip-luminous-pr-17599

luminous: encoding: reset optional<> if it is uninitialized

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17487 from smithfarm/wip-21231-luminous

luminous: core: interval_set: optimize intersect_of insert operations

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17712 from batrick/bp21221

luminous: MDCache::try_subtree_merge() may print N^2 lines of debug message

Reviewed-By: Yan Zhen <zyan@redhat.com>

Merge pull request #17623 from liewegas/wip-pr-17572-luminous

tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17584 from jdurgin/wip-recovery-delete-encoding-luminous

osd/PGBackend, MOSDPGRecoveryDelete[Reply]: handle incompatible encoding

Reviewed-By: Sage Weil <sage@redhat.com>

Merge pull request #17686 from ukernel/luminous-21070

luminous: mds: fix 'dirfrag end' check in Server::handle_client_readdir

Reviewed-By: Patrick Donnelly <pdonelly@redhat.com>

Merge pull request #17711 from batrick/bp20594

luminous: mds: support limiting cache by memory

Reviewed-By: Yan Zheng <zyan@redhat.com>

Merge pull request #17714 from batrick/bp21222

luminous: MDS: standby-replay mds should avoid initiating subtree export

Reviewed-By: Yan, Zheng <zyan@redhat.com>

Merge pull request #17570 from theanalyst/wip-luminous-21233

luminous: rgw: memory leak in MetadataHandlers

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17642 from theanalyst/wip-21350-luminous

luminous: rgw: data encryption sometimes fails to follow AWS settings

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17674 from theanalyst/luminous-rgw-reshard-enable

luminous: rgw: reshard enable

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17479 from smithfarm/wip-21139-luminous

luminous: rgw: put lifecycle configuration fails if Prefix is not set

Reviewed-By: Daniel Gryniewicz <dang@redhat.com>

cmake: do not pass $SIMD_COMPILE_FLAGS to rocksdb cmake

which enables SSE42 globally in rocksdb. and we will end up with a
binary not portable on non-SSE42 enabled machines.

Fixes: http://tracker.ceph.com/issues/20529
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
this change is not cherry-picked from master. because the
PR targeting master (https://github.com/ceph/ceph/pull/17388) is
still pending on review. and the cmake changes is different if
we want to use a recent commit of rocksdb, as it's doing differently
in cmake to address the portability issues.

rocksdb: pick up portable fixes

to pick up the the fix to disable SSE42 globally, and only enable it on
crc32c. this change is pushed to ceph/rocksdb:luminous.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
this change is not cherry-picked from master, as the master PR
(https://github.com/ceph/ceph/pull/17388) is still pending on review.
and the latest rocksdb's cmake was revised to address the portability
issues. so the fix on ceph side is different if we want to use a
recent rocksdb's commit.

rgw: fix resource leak in rgw_bucket.cc and rgw_user.cc

Fixes: http://tracker.ceph.com/issues/21214
Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
(cherry picked from commit 1f23976adab74e25ce06941692a881521885c6ee)

rgw: add missing override in list_keys_init()

Added missing override in list_keys_init().

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit bfc999f1db96293e6a2188922074031fa0b40ecc)

Merge pull request #17612 from smithfarm/wip-qemu-cleanup

build/ops: rpm: do not create librbd.so.1 symlink in /usr/lib64/qemu

Reviewed-By: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #17478 from smithfarm/wip-21138-luminous

luminous: rgw: object copied from remote src acl permission become full-control issue

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17445 from smithfarm/wip-21110-luminous

luminous: rgw: send data-log list infinitely

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

mon: incorrect MAX AVAIL in "ceph df"

Fixes: http://tracker.ceph.com/issues/21243
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit 365558571c59dd42cf0934e6c31c7b4bf2c65026)

Merge pull request #17488 from smithfarm/wip-21241-luminous

luminous: rgw: usage of --inconsistent-index should require user confirmation and print a warning

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17484 from smithfarm/wip-21184-luminous

luminous: tests: NameError: global name 'name' is not defined

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17474 from smithfarm/wip-21114-luminous

luminous: cephfs: tests: FS_DEGRADED spurious health warnings in some sub-suites

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

Merge pull request #17471 from smithfarm/wip-21107-luminous

luminous: cephfs: client/mds has wrong check to clear S_ISGID on chown

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

ceph: fix "stopping mds can't export subtree"

caused by commit
790c34138f "Make standby-replay mds avoid initiating subtree export"

Fixes: http://tracker.ceph.com/issues/21378
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c294051987de1ff79540060a0563fa02c8f03f91)

Make standby-replay mds not migrate empty subtree when trimming cache

Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit 178ca58fec2d820f61724d223addac1c4fa67086)

Make standby-replay mds avoid initiating subtree export

Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit 790c34138fbdf995c72e558057b9e64c882b522b)

mds: print subtrees only after merge

During up:resolve, the MDS tries to merge each subtree with its parent. During
testing, QE found that many thousands of subtrees in a directory (made possible
using pins) would cause the MDS to spend minutes printing out subtree maps to
the debug log. This causes the heartbeat code to consider the MDS as stalled so
beacons are no longer sent to the mons resulting in the MDS being removed from
the rank.

A more complete solution to this problem is to selectively print subtrees
relating to the operation (e.g. the subtree and its parents).

Fixes: http://tracker.ceph.com/issues/21221
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1485783
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d0747a37fd06053b2206bb9a952f7ab77f0db2f0)

mds: optimize MDCache::rejoin_scour_survivor_replicas()

avoid iterating dentries if dirfrag is non-auth

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d32a2376047bc1363b9f7a2ed09060ec84c78b97)

mds: fix MDSCacheObject::clear_replica_map

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit fd447405bef347da80ed5611331b211021b1628b)

mds: support limiting cache by memory

This introduces two config parameters:

    mds_cache_memory_limit: Sets the soft maximum of the cache to the given
    byte count. (Like mds_cache_size, this doesn't actually limit the maximum
    size of the cache. It just dictates the steady-state size.)

    mds_cache_reservation: This replaces mds_health_cache_threshold everywhere
    except the Beacon heartbeat sent to the mons. The idea here is to specify a
    reservation of memory (5% by default) for operations and the MDS tries to
    always maintain that reservation. So, the MDS will recall caps from clients
    when it begins dipping into its reservation of memory.

mds_cache_size still limits the cache by Inode count but is now by-default 0
(i.e. unlimited). The new preferred way of specifying cache limits is by memory
size. The default is 1GB.

Fixes: http://tracker.ceph.com/issues/20594
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1464976
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 06c94de584e6cd7d347bcdfb79d9fef4fed0d277)

 Conflicts:
PendingReleaseNotes
src/mds/MDCache.cc

common: refactor of lru

Avoids an unnecessary "max" size of the LRU which was used to calculate the
midpoint. Instead, just dynamically move the LRUObjects between top and bottom
on-the-fly.

This change is necessary for a cache which which does not limit by the number
of objects but by some other metric. (In this case, memory.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 12d615b3c546211b48ee75921f0e06371dd62dae)

mds: resolve unsigned coercion compiler warning

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0c2032c2876c3e2cd0d23ab883fc72375731cadf)

common: use safer uint64_t for list size

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0ddd260a32e781ff6bfa49c1a01d6b2d80e5dcd3)

common: add bytes2str pretty print function

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7fff24e10e9cacc94ec320b87e60c7735ff028b6)

mds: check if waiting is allocated before use

This prevents accidental allocation of the map.

Also, privatize the variable to protect from this in child classes.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 055020ce80b1f08d258b4c023bf5465ed7a46034)

mds: go back to compact_map for replicas

Zheng observed that an alloc_ptr doesn't really work in this case since any
call to get_replicas() will cause the map to be allocated, nullifying the
benefit. Use a compact_map until a better solution can be written. (This means
that the map will be allocated outside the mempool.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5d67b5cc57ac803e60c2921f554cd0507ed4848e)

mds: use mempool for cache objects

The purpose of this is to allow us to track memory usage by cached objects so
we can limit cache size based on memory available/allocated to the MDS.

This commit is a first step: it adds CInode, CDir, and CDentry to the mempool
but not all of the containers in these classes (e.g. std::map). However,
MDSCacheObject has been changed to allocate its containers through the mempool
by converting compact_* containers to the std versions offered through mempool
via the new alloc_ptr.

(A compact_* class simply wraps a pointer to the std:: version to reduce memory
usage of an object when the container is only occasionally used. The alloc_ptr
allows us to achieve the same thing explicitly with only a little handholding:
when all entries in the wrapped container are deleted, the caller must call
alloc_ptr.release().)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e035b64fcb0482c3318656e1680d683814f494fe)

mds: cleanup replica_map access

The gymnastics protecting the map failed as the code evolved. Just expose it
normally with a getter.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d1b6cadd6c1514aeacfbacec253305e1a715c387)

common: add alloc_ptr smart pointer

This ptr is like a unique_ptr except it allocates the underlying object on
access. The idea being that we can save memory if the object is only needed
sometimes.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5fa557d2713558038af7579de8a3ca56d58b8d90)

common: add warning on base class use of mempool

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0d0fa804ee0fa775ae18f9b7bde2317bb6af52d)

common: use atomic uin64_t for counter

Making this interface thread-safe...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 59b5931a2f47f90fced779fb89e06868c739c688)

Merge pull request #17646 from theanalyst/wip-luminous-multifixes

luminous: rgw fixes for multisite tests in luminous

Reviewed-By: Casey Bodley <cbodley@redhat.com>

mds: fix 'dirfrag end' check in Server::handle_client_readdir

The 'dirfrag end' check can be wrong when not reading from beginning
of dirfrag.

Fixes: http://tracker.ceph.com/issues/21070
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 88332c0f2f0a7a99639d1de237d5bb68edddbe70)

Merge pull request #17647 from dillaman/wip-21345

luminous: qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: LCFilter_S3::to_xml the second "prefix" missed "/"

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
(cherry picked from commit dcef8555e1bf2656aeb5316c4b78f1facf2ae172)

test/rgw: fix test_encrypted_object_sync for 3+ zones

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8762c4b29051b2b49bfd8c1e0ceaa1aee6417a74)

luminous: rgw: re enable dynamic resharding

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #17653 from ceph/backport-wip-bz1478598

luminous: ceph-volume allow filtering by `uuid`, do not require osd id

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #17485 from smithfarm/wip-21185-luminous

luminous: rgw: rgw_file:  incorrect lane lock behavior in evict_block()

Reviewed-By: Orit Wasserman <oritwas@redhat.com>

Merge pull request #17476 from smithfarm/wip-21135-luminous

luminous: rgw: bucket index sporadically reshards to 65521 shards

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17470 from smithfarm/wip-21104-luminous

luminous: build/ops: ceph-fuse RPM should require fusermount

Reviewed-By: Ken Dreyer <ktdreyer@redhat.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #17477 from smithfarm/wip-21137-luminous

luminous: mgr: 500 error when attempting to view filesystem data

Reviewed-By: John Spray <jcsp@redhat.com>

Merge pull request #17495 from smithfarm/wip-21235-luminous

luminous: tests: thrashosds read error injection doesn't take live_osds into account

Merge pull request #17483 from smithfarm/wip-21183-luminous

luminous: mgr: Crash in MonCommandCompletion

Merge pull request #17486 from smithfarm/wip-21188-luminous

luminous: pybind: dashboard usage graph getting bigger and bigger

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17350 from liewegas/wip-pr-17334-luminous

os/bluestore: reshard-fix wrong shard length

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17473 from smithfarm/wip-21112-luminous

luminous: cephfs: get_quota_root sends lookupname op for every buffered write

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17465 from smithfarm/wip-21045-luminous

luminous: rbd: TestMirroringWatcher.ModeUpdated: periodic failure due to injected message failures

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17565 from theanalyst/wip-21278-luminous

luminous: the standbys are not updated via ceph tell mds.* command

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17566 from theanalyst/wip-21267-luminous

luminous: Incorrect grammar in FS message 1 filesystem is have a failed mds daemon

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17557 from theanalyst/wip-21269-luminous

luminous: some generic options can not be passed by rbd-nbd

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17448 from smithfarm/wip-21118-luminous

luminous: rgw: need to stream metadata full sync init

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17464 from smithfarm/wip-20964-luminous

luminous: rbd: switch to new config option getter methods

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

ceph-volume lvm.trigger just split the osd_id once

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ecd5af88da2815603283742c661d27e2a48f9f7e)

ceph-volume tests catch improper parsing of osd uuid

When the osd ID existed in the UUID in the same format (e.g. '1-') the
parsing would break returning a bogus UUID that is impossible to find

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5392697121ef150f0edac05df14d116cf0148593)

tests: remove 'ceph-' prefix from test node hostnames

This updates our Vagrantfile to match the changes in ceph-ansible
introduced by https://github.com/ceph/ceph-ansible/commit/298a63c43

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f41d359ebaec8762ed1a7b8cfd85c23f351a5070)

ceph-volume lvm.api PVolumes should filter by all matching tags, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b8613c1d7bf56c49d1b7f1c41a37073b8739bbc3)

ceph-volume tests pv API should also match all pv_tags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 120fd6a196045c8511c3eaafec37de43003bf450)

ceph-volume tests prove all volume tags have to match in vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe2a7106decb586e83f5b60d6c18c454afe4883e)

ceph-volume lvm.api all vg tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d679855aa077e2de3f20316625d478a3697c78f)

ceph-volume tests prove all tags have to match in lvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 80e401ca970066541ae7255be1caf42883048df4)

ceph-volume lvm.api all lv tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dbaf30c3e4d921df2a92812383c8805a77eb8f5b)

ceph-volume lvm.activate allow not using osd_id

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 99a8a719695591558da29b3d16a347d5bff8bf3d)

ceph-volume tests prove missing args.osd_id is fine

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d1a1a885fd65073308c205949bc70b41805f92f)

ceph-volume tests add is_root and lv fixtures to conftest

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d18d5e139be82de36e1a60340525adbf79fdedd9)

Merge pull request #17446 from smithfarm/wip-21115-luminous

luminous: rgw multisite: objects encrypted with SSE-KMS are stored unencrypted in target zone

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17468 from smithfarm/wip-21052-luminous

luminous: build/ops: RHEL 7.3 Selinux denials at OSD start

Reviewed-By: Ken Dreyer <ktdreyer@redhat.com>

Merge pull request #17444 from smithfarm/wip-20968-luminous

luminous: rgw: bytes_send and bytes_recv in the msg of usage show returning is 0 in master branch

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17555 from theanalyst/wip-21270-luminous

luminous: rgw: shadow objects are sometimes not removed

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17558 from theanalyst/wip-21277-luminous

luminous: [cls] metadata_list API function does not honor `max_return` parameter.

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17556 from theanalyst/wip-21265-luminous

luminous: [cli] rename of non-existent image results in seg fault

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17560 from theanalyst/wip-21288-luminous

luminous: [test] various teuthology errors

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17561 from theanalyst/wip-21289-luminous

luminous: [rbd] image-meta list does not return all entries

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

qa/workunits/rbd: relax greps to support upgrade formatting change

Fixes: http://tracker.ceph.com/issues/21181
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 273c84578b06905229a7c6eae7b1a12bbe60de95)

Conflicts:
qa/workunits/rbd/import_export.sh: trivial resolution

Merge pull request #17336 from tchaikov/wip-luminous-pr-16774

luminous: mgr/dashboard: fix rbd's pool sub menu

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

rgw: Data encryption is not follow the AWS agreement

Encryption request headers should not be sent for GET requests and HEAD
requests if your object uses SSE-KMS/SSE-S3 or you’ll get an HTTP 400
BadRequest error.

Signed-off-by: hechuang <hechuang@xsky.com>
(cherry picked from commit dfe1a7c1e6c470d623c70d83c32cc5b7a921bbc5)

mon: fix dropping mgr metadata for active mgr

drop_standby() was killing it and it was only getting added
back in certain locations.  Instead, make the metadata
drop conditional and only do it in the places we're
really dropping the daemon, not when we're promoting
it to active.

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 29c6f9adf178f6611a625740f395e397cad9147b)

qa/tasks/mgr: regression test for 21260

(aka http://tracker.ceph.com/issues/21260)

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 28cb4bb2e68298a8a307eb2de6121b60db43cb86)

mon/OSDMonitor: don't create pgs if pool was deleted

A pool may be deleted while in the process of creating pgs. If that
happens, we should not queue additional pgs from that pool to be
created.

Fixes: http://tracker.ceph.com/issues/21309
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 7a8a88032ec0bf7f1b485419a11646d7441faf2c)

mon/OSDMonitor: fix typo

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit bc5fe2e9099dbb560c2153d3ac85f38b46593a77)

encoding: reset optional<> if it is uninitialized

* should reset it, in case we reuse it after initializing it.
* initialize the value of `p` using the C++11 style initializer, so it
  is zero-initialized.
* revert 2a83ef3c which disables a warning of:
 ./include/encoding.h:317:7: warning: 't' may be used uninitialized in
    this function [-Wmaybe-uninitialized]
 where the `t` is the temporary variable for initializing the value of
 `p`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0363236dc754180a0215a818d58587943f84fb53)

tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG

Recently we plan to bring a Jewel cluster into Luminous.
After that is done, which turns out to be a big success,
we then try to transform all FileStore osds into BlueStore ones
offline but with no luck. The ceph_objectstore_tool keeps complaining:
--------------------------------------------------------------------
dup from filestore: /var/lib/ceph/osd/ceph-20.old
      to bluestore: /var/lib/ceph/osd/ceph-20
fsid d444b253-337d-4d15-9d63-86ae134ec9ac
65 collections
1/65 meta
cannot get bit count for collection meta: (61) No data available
--------------------------------------------------------------------

The root cause is that for FileStore Luminous will always try to rewrite
pg "bits" as a file attribute on "Load" if that is not available.
But since meta pg is never loaded (we skip it during OSD::load_pgs()),
we actually never get the chance to do so; hence making the
dup method from ceph_objectstore_tool very unhappy since it always
expects to see such a attribute from underlying store.

Fix the above problem by manually skipping loading the "bits" attribute
if underlying OS is FileStore for dup.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit af2c4847e42945a878e2864319185d795b0664a5)