Merge pull request #11718 from xiexingguo/xxg-wip-bluestore-1101

os/bluestore: add counter to trace blob splitting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10248 from ktdreyer/wip-doc-no-rpm-md

doc: rm "type=rpm-md" from yum repositories

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #11690 from yonghengdexin735/wip-zzz-fixword

src/common/perf_counters.h: fix wrong word

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: add sanity check when compress_extent_map()

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: formatting changes

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: add counter to trace blob splitting

As we want to know if it really happens and how often does it happen.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11568 from dillaman/wip-17588

librbd: cannot access non-primary image when mirroring force disabled

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11280 from ceph/wip-17440

rgw: fixes for virtual hosting of buckets
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11559 from dillaman/wip-17618

librbd: exclusive lock incorrectly initialized when switching to head revision

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11623 from dillaman/wip-cls-rbd-group

librbd: ensure consistency groups will gracefully fail on older OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11703 from dillaman/wip-17750

test: skip TestLibRBD.DiscardAfterWrite if skip partial discard enabled

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11714 from dachary/wip-17757-eio

tests: disable failing tests

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

tests: disable failing unittest_erasure_code_plugin_jerasure

While figuring out what's wrong with it.

Refs: http://tracker.ceph.com/issues/17561

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: disable failing test-erasure-eio.sh

While figuring out what's wrong with it.

Refs: http://tracker.ceph.com/issues/17757

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11382 from yuyuyu101/wip-fix-shadow

osd/PGBackend: fix collection_list shadow return value

Reviewed-by: Kefu Chai <kchai@redhat.com>
tchaikov

Merge pull request #11547 from javeme/check-pgid

osd/command tell: check pgid at the right time

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11203 from stiopaa1/osd_pg_replaceCountPart1

osd/PG.cc: prevent repeated searching of map/set

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11242 from stiopaa1/osd_replicatedpg_removeCount1

osd/ReplicatedPG: remove unneeded use of count

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11365 from badone/wip-lsb_release-remove

common: Remove the runtime dependency on lsb_release

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer (Red Hat) <kdreyer@redhat.com>

test: skip TestLibRBD.DiscardAfterWrite if skip partial discard enabled

Fixes: http://tracker.ceph.com/issues/17750
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: rm "type=rpm-md" from yum repositories

As far as I can tell, this is only relevant to SUSE's Zypper, and yum
does not need it.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

librbd: unit test cases and minor fixes for mirror::DisableRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/journal: use librados::IoCtx reference for helper state machines

This facilitates being able to properly mock test cases around journal
cls method calls.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: restore journal access when force disabling mirroring

If mirroring is force disabled on a demoted image, the journal was
being left in an inconsistent ownership state.

Fixes: http://tracker.ceph.com/issues/17588
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new async journal promote state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new async journal open state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move journal tag decode helpers to shared location

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: re-add support for disabling a corrupt journal

Fixes: http://tracker.ceph.com/issues/16740
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11249 from wido/osd-flush-journal-shutdown

osd: Flush Journal on shutdown

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11686 from ceph/wip-ceph-rest-api-new-addr

ceph-rest-api: understand the new style entity_addr_t representation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11530 from markhpc/wip-rocksdb_tuning

common/config_opts.h: Optimized RocksDB WAL settings.

Merge pull request #11688 from stiopaa1/mds_snapserver_addConstToMemFun

mds/SnapServer: add const to member function

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

mds/SnapServer: add const to member function

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

ceph-rest-api: understand the new style entity_addr_t representation

the new entity_addr_t representation was introduced by 3561eb3, we
should update ceph_rest_api.py accordingly.

Fixes: http://tracker.ceph.com/issues/17742
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10993 from Vicente-Cheng/fix-alloc-choose_tries

crush: make counting of choose_tries consistent

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11246 from badone/wip-set_xattr_limits_via_conf-fix

os/filestore: Fix erroneous WARNING: max attr too small

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11614 from runsisi/wip-fix-PK11_FreeSymKey-segfault

auth: fix NULL pointer access when trying to delete CryptoAESKeyHandler instance

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11654 from hjwsm1989/fix

mon: print the num_pools and num_objects in 'ceph -s -f json/json-p…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11639 from atheism/mgr-PyModules-remove-duplicated-else-if

mgr: PyModules.cc: remove duplicated if condition for fs_map

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11659 from tchaikov/wip-delete-cct-later

librados: postpone cct deletion

Reviewed-by: Sage Weil <sage@redhat.com>

src/common/perf_counters.h:fix wrong word

Signed-off-by: zhang.zezhu <zhang.zezhu@zte.com.cn>

Merge pull request #11314 from jcsp/wip-mgr-flake8

pybind: clean up mgr stuff for flake8

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11650 from wjwithagen/wip-wjw-log-loging

Log: Replace namespace log with logging

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11680 from tchaikov/wip-17728

test/ceph_test_msgr: do not use Message::middle for holding transient…

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11397 from dzafman/wip-scrub-misc

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/ceph_test_msgr: do not use Message::middle for holding transient data

Message::middle is used for holding encoded data, so we we can not stuff
it with payload and leave the "payload" field empty. this change
refactors the ceph_test_msgr by introducing a Payload class which
encodes all test data in it.

Fixes: http://tracker.ceph.com/issues/17728
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11677 from ceph/wip-rocksdb-no-march-native

rocksdb: do not build with --march=native

Reviewed-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #11678 from tchaikov/wip-silence-rbd-warning

librbd: silence the unused variable warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: silence the unused variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

rocksdb: do not build with --march=native

we can not assume that building host has the same arch (or the same cpu
capability) with the target machine.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11663 from stiopaa1/mds_mdslog_addConstToMemFunc

mds/MDLog: add const to member functions

Merge pull request #11668 from Enproduktion/master

doc: Fixed mapping error in legacy mds command

Merge pull request #11446 from ukernel/wip-cephfs-scan-links

tools/cephfs: add scan_links command which fixes linkages errors

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11609 from stiopaa1/mds_snaprealm_unneededuseofcount

mds/SnapRealm: remove unneeded use of count

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11625 from stiopaa1/mds_damageTable_removeUnneededCount

mds/DamageTable.cc: remove unneeded use of count

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9304 from dzafman/wip-13937

osd: handle EC recovery read errors

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11645 from dillaman/wip-librbd-discard-after-write

test: new librbd discard after write test case

Merge pull request #11670 from stiopaa1/mds_mutation_addConstToMemFun

mds/Mutation.h: add const to member functions

Merge pull request #11608 from atheism/mds-duplicated-check

client: Client.cc: remove duplicated op type checking against CEPH_MD…

client: Client.cc: correct the op type checking for CEPH_MDS_OP_CREATE

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>

global: Always log loading erasure code for tests

Fix for broken test-erasure-code.sh and test-erasure-eio.sh

Signed-off-by: David Zafman <dzafman@redhat.com>

test.sh: Make check for flags more robust

Low space broke test, saw "flags nearfull,pauserd,pausewr...."

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Remove extra objectstore_tool call which causes a recovery

Caused by: 70e000a9a42c50eda48f5d3b9e432ffc3a70f75b

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Handle object removals in a non-racey way

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix hang on unfound object after mark_unfound_lost is done

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Handle recovery read errors

Fixes: http://tracker.ceph.com/issues/13937
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix log messages

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: CLEANUP: Remove unused pending_read member

Remove unused struct

Signed-off-by: David Zafman <dzafman@redhat.com>

Revert "test: Disable tests due to recovery race"

This reverts commit 5bc55338f5a1645bc651811fae2f89ad855ff86e.

Conflicts:
src/test/Makefile.am (no longer exists)
src/test/erasure-code/Makefile.am (no longer exists)

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #11582 from batrick/ceph-dumpable

core: set dumpable flag after setuid

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

mds/Mutation.h: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

doc: Fixed mapping error in legacy mds command

Signed-off-by: Malte Fiala <maltefiala@produktion.io>

Merge pull request #11666 from tchaikov/wip-cmake

cmake: remove include/Makefile.am

cmake: remove include/Makefile.am

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/filestore: Fix erroneous WARNING: max attr too small

When we read the conf values from the conf file we call handle_conf_change which
calls set_xattr_limits_via_conf but this is before m_fs_type is initialised so
we fall through to the default comaprison with
filestore_max_xattr_value_size_other (1024) and erroneously issue the warning.
This happens at every startup but subsequent calls once the filestore is mounted
proceed as expected. Don't call set_xattr_limits_via_conf unless the filestore
is mounted.

Fixes: http://tracker.ceph.com/issues/17420
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #11638 from tchaikov/wip-clang

include/enc: make clang happy

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11658 from hjwsm1989/doc

doc: change the osd_max_backfills default to 1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mds/MDLog: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #10887 from michaelsevilla/mantle

fs: Mantle: A Programmable Metadata Load Balancer

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11641 from vshankar/rbd-data-pool-assert

librbd: possible assert failure creating image when using data pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

common/cmdparse,msg/async: remove unnecessary std::move()

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: postpone cct deletion

RefCountedObject::put() prints log using cct, but RadosClient
relinquishes the reference to cct too early, so when its
`RefCountedObject` member variables print log when they are
destructed, the cct is already freed. hence we have a segfault.

to address this problem, we need postpone destructure of it after all
other RefCountedObject member variables are destroyed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11570 from tchaikov/wip-revise-SubmittingPatches

doc: reformat SubmittingPatches with more rst syntax

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #11630 from cbodley/wip-rgw-curl-upgrade

rgw: remove suggestion to upgrade libcurl
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

osd/PGBackend: fix collection_list shadow return value

Fixes: http://tracker.ceph.com/issues/17713
Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11385 from bassamtabbara/wip-rocksdb-external-project

cmake: use external project for rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: remove ending period in section titles

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: change the osd_max_backfills default to 1
Fixes: http://tracker.ceph.com/issues/17701
Signed-off-by: huangjun <hjwsm1989@gmail.com>

Merge pull request #11653 from ktdreyer/doc-epel-instructions

doc: standardize EPEL instructions

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: use external project for rocksdb

This commit makes rocksdb an external cmake project. This ensures
that rocksdb is no longer built from src/rocksdb, and that make clean
works correctly.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

pickup new rocksdb with support for linux cmake

This picksup a new version of rocksdb with support for
linux cmake.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

Merge pull request #11619 from dachary/wip-17662-ceph-disk

ceph-disk: allow using a regular file as a journal

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11652 from ktdreyer/doc-mgr-syntax

doc: fix mgr literal block rST syntax

Reviewed-by: Kefu Chai <kchai@redhat.com>

  mon: print the num_pools and num_objects in 'ceph -s -f json/json-pretty'.
Fixes: http://tracker.ceph.com/issues/17703
Signed-off-by: huangjun <hjwsm1989@gmail.com>

librbd: assert failure when using data pool

This fixes a silly assert that's hit during image creation
(cli/api) when the data pool specified is same as the pool
specified by -p/--pool option (or the default).

Signed-off-by: Venky Shankar <vshankar@redhat.com>

doc: standardize EPEL instructions

Prior to this change, the documentation pages contained different ways
to enable EPEL. Pick a simple, secure (https) way and standardize on
that.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: fix mgr literal block rST syntax

Prior to this change, there were double-colon ("::") characters in the
HTML docs to indicate literal blocks.

The proceeding paragraphs were not pre-formatted because there was no
newline between the double-colon and the to-be-pre-formatted paragraph.

Move the "::" characters inline to simplify the syntax and make Sphinx
properly interpret these texts as literal blocks.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #11594 from cbodley/wip-rgw-init-bucket-sync

rgw: fix for passing temporary in InitBucketSyncStatus

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11505 from cbodley/wip-17571

rgw: clean up RGWShardedOmapCRManager on early return

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11504 from cbodley/wip-17568

rgw: RGWSimpleRadosReadCR tolerates empty reads

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>