organizationmap: Ashish Singh affiliation

Signed-off-by: Ashish Singh <assingh@redhat.com>

Merge pull request #17086 from optimistyzy/818_write

bluestore/NVMEDevice: fix the bug in write function

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17104 from iliul/fix-duplicate-words

doc: delete duplicate words

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: delete duplicate words

Signed-off-by: iliul <liul124@chinaunicom.cn>

Merge pull request #17060 from cyx1231st/fix_reqid_assignment_in_oprequest

osd: fix reqid assignment for reply messages in OpRequest

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16687 from Songweibin/wip-xml-format

common: adding line break at end of some cli results

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15118 from squidboylan/fix_15872

debian: lock ceph user during purge

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17059 from tchaikov/wip-optional-lz4

compressor: conditionalize on HAVE_LZ4

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17015 from lixiaoy1/lat_fix

os/bluestore: Add lat record of deferred_queued and deferred_aio_wait

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16955 from xiexingguo/wip-app-ls

mon/OSDMonitor: "osd pool application get" support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17064 from xiexingguo/wip-20970

mon/PGMap: reweight::by_utilization - skip DNE osds

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17067 from renhwztetecs/renhw-wip-mon-cleanup

mon: some cleanup

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #17052 from dzafman/wip-pidfile

tests: Increase sleep in test_pidfile.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15691 from vshankar/mirror-ha-policy

rbd-mirror: introduce basic image mapping policy

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17076 from ceph/toc-sidebar-fix

doc/templates update toctree call to include hidden entries

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #17093 from dzafman/wip-runs-message

tests: run-standalone.sh improve error message

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc update template call for toctree

Up until version 1.1.3 of Sphinx, this call would show hidden toctree
entries. Most of Ceph's toctree's are hidden, so when the version of
Sphinx was bumped to >1.6, the sidebar stopped showing entries

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17026 from dzafman/wip-add-trap

tests: qa/standalone: Add trap for signals to restore the kernel core pattern

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: Add lat record of deferred_queued and deferred_aio_wait

Originally bluestore defines the two lat deferred_queued_lat and
deferred_aio_wait_lat, but doesn't use it.
This patch is to add the records that each TXC stays on deferred queue
deferred ios.

Signed-off-by: Xiaoyan Li xiaoyan.li@intel.com

rbd-mirror: load/update image map state machine

Signed-off-by: Venky Shankar <vshankar@redhat.com>

rbd-mirror: track on-disk image to instance map

Signed-off-by: Venky Shankar <vshankar@redhat.com>

tests: run-standalone.sh improve error message

Signed-off-by: David Zafman <dzafman@redhat.com>

NVMEDevice: fix the write bug

This patch can solve the write issue when we use spdk nvme driver
to handle I/Os from db bdev in the future.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #17070 from ktdreyer/rpm-macros-in-comments

rpm: rm macros in comments

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

mon/PGMap: reweight::by_utilization - skip DNE osds

EC could set one or more members of acting set to CRUSH_ITEM_NONE,
which as a result can cause pgs_by_osd.resize() attempt to apply
for a large amount of memory which we can not afford.

Fix the above problem by always excluding a current DNE osd from
acting set.

Fixes: http://tracker.ceph.com/issues/20970
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17043 from renhwztetecs/renhw-wip-pgmap-pool

mon/pgmap: update pool nearfull display

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17002 from mychoxin/yuanxin_adjust_blockdevice

os/bluestore: move aio_callback{,_priv} to base class BlockDevice

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16940 from jcsp/wip-20955

mgr: add missing call to pick_addresses

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16407 from dzafman/wip-20243

osd: Improve size scrub error handling and ignore system attrs in xattr checking

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: add new line break when get command descriptions

Signed-off-by: songweibin <song.weibin@zte.com.cn>

rbd: adding newline operation when only '--format' used

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #17062 from trociny/wip-qa-rbd-mirror-data-pool

qa/workunits/rbd: test data pool is mirrored correctly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16517 from gohighsec/master

rbd: check if an image is already mapped before rbd map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16591 from PCzhangPC/fix7

rbd: introduce commands of "image-meta ls/rm"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17073 from dillaman/wip-20567

rbd-mirror: Set the data pool correctly when creating images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: Update mirroring docs to mention data-pool configuration

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
(cherry picked from commit 1e001c19e98ac1cd2428dd66f047ea4d6b2b01b1)

rbd-mirror: Set the data pool correctly when creating images

Previously the data-pool option was ignored by rbd-mirror, so when using
erasure coding data for the image on the secondary site would end up in
the metadata pool.

Configure the data-pool using the data-pool's name from the primary site
when creating images on the secondary site.

Fixes: http://tracker.ceph.com/issues/20567
Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
(cherry picked from commit dffaac83d19de544e279a530393b3998f1e1fbb4)

Merge pull request #17069 from yuriw/wip_add_luminous

Decreased amount of jobs on master, kraken, luminous runs

Merge pull request #17040 from mdw-at-linuxbox/rgw-master-fix-plus

Rgw master fix plus

rpm: rm macros in comments

rpm expands all macros in a .spec file, even those in comments. Drop the
percent signs so rpm will not expand these.

This change silences rpmlint's warning about macros in comments.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #17066 from amitkumar50/cov-os-1

os: Initializing uninitialized members aio_info

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

os: Initializing uninitialized members aio_info

Fixes the coverity issue:

** 717332 Uninitialized pointer field
2. uninit_member: Non-static class member field iocb.data is not
initialized in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member field iocb.key is not
initialized in this constructor nor in any functions that it calls.
6. uninit_member: Non-static class member field iocb.__pad2 is not
initialized in this constructor nor in any functions that it calls.
8. uninit_member: Non-static class member field iocb.aio_lio_opcode
is not initialized in this constructor nor in any functions that it calls.
10. uninit_member: Non-static class member field iocb.aio_reqprio
is not initialized in this constructor nor in any functions that it calls.

CID 717332 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
12. uninit_member: Non-static class member field iocb.aio_fildes is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

osd/OpRequest: fix reqid assignment for reply messages

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Decreased amount of jobs on master, kraken, luminous runs

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #16896 from amitkumar50/cov-librado-2

rados: Initializing members of librados/TestCase.h

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16939 from trociny/wip-nonlinux-fsx

test/librbd: make fsx build on non-linux platform

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17027 from joscollin/wip-cleanup-Wsign-compare

test/osd: silence warnings from -Wsign-compare

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/workunits/rbd: test data pool is mirrored correctly

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

cmake: require lz4 v1.7 for building lz4 compressor

Signed-off-by: Kefu Chai <kchai@redhat.com>

compressor: conditionalize on HAVE_LZ4

do not enable lz4 compressor plugin unless HAVE_LZ4 is ON. so, for
example, when the compressor type is "random", the compressor factory
won't complain with:

load failed dlopen(): "/usr/lib64/ceph/compressor/libceph_lz4.so: cannot
open shared object file: No such file or directory"

anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #16891 into master

* refs/remotes/upstream/pull/16891/head:
mds/MDSDaemon: add 'is_valid=false' when failed to parse caps

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17022 from ceph/wip-doc-sans-gitbuilder

Update docs to remove gitbuilder and add shaman references

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tests: Increase sleep in test_pidfile.sh

Signed-off-by: David Zafman <dzafman@redhat.com>

doc/rados/configuration comment out '...' on ini blocks

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/api comment out '...' on ini blocks

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install describe shaman urls and how to retrieve list and repo files

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install update international for non-US

Use of "international" is ambiguous, for non-US users, international
could mean US.

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install remove repetitive dev release information for rpm distros

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install remove mention of autobuild key for dev packages

Dev packages are no longer signed, and the key is not used at all

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install remove Apache and FastCGI instructions for RGW

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: update repo-access to point to ceph-ci.git

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17025 from ceph/wip-doc-warnings

fix sphinx build warnings and errors

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc: fix short underline on release-notes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #15597 from ceph/wip-19200

selinux: Allow nvme devices

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc exclude man_index when not building man pages

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: create an index/toctree only for man pages

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: exclude mgr from man pages, define a man index.rst

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/release-notes use new ref label for erasure code profiles

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations use new ref labels in health-checks

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add a ref label to erasure code profiles

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add a ref label to pool application

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations use new ref label in health-checks

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add ref label for placement group number

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/api/python document required_state manually

Signed-off-by: Alfredo Deza <adeza@redhat.com>

admin: bump sphinx to 1.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations use the new ref label for crush map tunables

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add label for crush map tunables for ref linking

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/cephfs add label to health messages for use in refs linking

Signed-off-by: Alfredo Deza <adeza@redhat.com>

admin: do not allow errors or warnings from doc build

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations include pg-repair document in the toctree

Signed-off-by: Alfredo Deza <adeza@redhat.com>

pybind fix rbd trash_move docstring, needed a newline

Signed-off-by: Alfredo Deza <adeza@redhat.com>

pybind: fix invalid doctring in Rados.require_state

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add missing link for crush pdf

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/operations add newline after definition

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/radosgw add newline after field list

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/radosgw fix missing newline on definition

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados/api remove '*' in automethod call

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/logging fix unexpected indent in paragraph

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: bump the max depth of the toctree, so that it can find nested indexes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc escape and remove invalid chars in release notes, causing sphinx errors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados use func reference for  rados_set_alloc_hint linking

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #16967 from liewegas/wip-upgrade-health

mon: fix legacy health checks in 'ceph status' during upgrade; fix jewel-x upgrade combo

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/pgmap: add objects prefix for unfound type

the objects prefix display indicates the scope of the "unfound",
like objects "misplaced" err.

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mon/pgmap: update pool nearfull display

if "pool full" and "pool nearfull" show the same,
we will not be able to distinguish between these two scenes in health check.

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

radosgw: fix small sized chunked uploads with s3

The logic to compute the number of bytes to copy with a chunked file
upload was failing to take into account the amount of data that had been
consumed, but was still pending in the parse buffer.  This would cause
strange behavior ranging from "just works" to "fails".  There was also a
strange "works, one byte at a time" mode.  Using the correct stream_pos
offset eliminates the bad behavior.

This fix also adds debug statements to make the correct behavior easier to see.

Fixes: http://tracker.ceph.com/issues/21003
Signed-off-by: Marcus Watts <mwatts@redhat.com>

mon/OSDMonitor: "osd pool application ls" support

It would be a pain if we have to call 'ceph osd dump --format=json-pretty'
to find out these each time...

Demo output:
(1) ceph osd pool application get
{
    "cephfs_data_b": {
        "cephfs": {}
    },
    "cephfs_metadata_a": {
        "cephfs": {}
    },
    "test_pool": {
        "rbd": {
            "test": "me"
        }
    }
}

(2) ceph osd pool application get test_pool
{
    "rbd": {
        "test": "me"
    }
}

(3) ceph osd pool application get test_pool rbd
{
    "test": "me"
}

(4) ceph osd pool application get test_pool rbd test
me

Fixes: http://tracker.ceph.com/issues/20976
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17012 from joscollin/wip-rgw-todo-warning

rgw: Drop #warning TODO

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17037 from theanalyst/release-notes-12-1-4

doc: release notes for v12.1.4 Luminous

Reviewed-By: Gregory Farnum <gfarnum@redhat.com>
Reviewed-By: Ken Dreyer <ktdreyer@redhat.com>

doc: release notes for v12.1.4 Luminous

Adding the bits of changelog that went in post 12.1.3, one of the PRs
was directly cherry-picked onto Luminous and hence does not have a PR
reference.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #16808 from liupan1111/wip-add-report

test/fio: generate db histogram to help debug rocksdb performance