Merge pull request #18155 from smithfarm/wip-doc-releases-tweak

doc: improve links in doc/releases.rst

doc: improve links in doc/releases.rst

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18153 from liewegas/wip-lts

doc/releases: drop LTS/stable line from second chart

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

doc/releases: tweaks

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases: drop LTS/stable line from second chart

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18152 from ceph/wip-fix-doc-exceptions

doc raise exceptions with a base class

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18117 from liewegas/wip-lts

doc/releases: update release cycle docs

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

doc/releases: update release cycle docs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18150 from xiexingguo/wip-21618

qa/standalone/scrub/osd-scrub-repair.sh: drop omap_digest flag

Reviewed-by: Sage Weil <sage@redhat.com>

doc parse two lines for getting the context

So that signatures can get parsed when they are split like:

PG::RecoveryState::RepWaitBackfillReserved::react(
  const RemoteReservationCanceled &evt)

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18151 from theanalyst/doc-changelog-10.2.10

doc: Adding changelog for 10.2.10

doc raise exceptions with a base class

Although valid in very old Python, it is no longer possible to raise
"bare" exceptions without a class.

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: Adding changelog for 10.2.10

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

qa/standalone/scrub/osd-scrub-repair.sh: drop omap_digest flag

This is no longer set if we are backed by bluestore, which we are by
default.  See be078c8b7b131764caa28bc44452b8c5c2339623

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18064 from tchaikov/wip-vectors-equal

osd: replace vectors_equal() with operator==(vector<>, vector<>)

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge PR #18032 into master

* refs/pull/18032/head:
Merge commit 'refs/pull/4/head' of github.com:batrick/ceph into ptl-tool-2
script/ptl-tool.py: Label Validation
Merge commit 'refs/pull/2/head' of github.com:batrick/ceph into ptl-tool-2
Merge commit 'refs/pull/1/head' of github.com:batrick/ceph into ptl-tool-2
script/ptl-tool.py: Fix the Comment example
script/ptl-tool.py: Fixed error message
ptl-tool: make more configurable and usable

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

githubmap: update contributors

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge commit 'refs/pull/4/head' of github.com:batrick/ceph into ptl-tool-2

* refs/pull/4/head:
  script/ptl-tool.py: Label Validation

Merge pull request #18134 from liewegas/wip-21660

src/messages/MOSDMap: reencode OSDMap for older clients

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17974 from jcsp/wip-doc-mgr-iptables

doc: update firewall doc to mention ceph-mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

src/messages/MOSDMap: reencode OSDMap for older clients

We explicitly select which missing bits trigger a reencode.  We
already had jewel and earlier covered, but kraken includes all of
the previously mentioned bits but not SERVER_LUMINOUS.  This
prevents kraken clients from decoding luminous maps.

Fixes: http://tracker.ceph.com/issues/21660
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18070 from liewegas/wip-21613

osd: fix replica/backfill target handling of REJECT

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17939 from mkogan1/wip-rgw-21196

rgw: update the usage read iterator in truncated scenario

Merge pull request #18028 from liewegas/wip-pick-ifname

common/pick_address: add {public,cluster}_network_interface option

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #18091 from vasilyMellanox/gid_idx_fix

msg/async/rdma: uninitialized variable fix

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18092 from tchaikov/wip-cmake-cleanup

cmake: do not link against unused or duplicated libraries

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18094 from xiexingguo/wip-tracker-21618

qa/standalone/scrub/osd-scrub-repair.sh: add extents flag into object_info_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17907 from amitkumar50/cov-rgw-10

rgw: Ignoring the returned error

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18106 from kmroz/wip-keystone-revocator-thread-assert

rgw: stop/join TokenCache revoke thread only if started.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/releases: drop LTS vs stable from chart; update luminous eol data

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18120 from liewegas/wip-bluefs-allocation

os/bluestore: make bluefs behave better near enospc

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18114 from dillaman/wip-21670

librbd: list_children should not attempt to refresh image

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18118 from liewegas/wip-21470

os/bluestore: fix another aio stall/deadlock

Reviewed-by: Varada Kari <varada.kari@gmail.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18105 from gregsfortytwo/wip-sigpipe-porting

msg: reimplement sigpipe blocking

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17710 from Songweibin/wip-add-pybind

pybind/rados: add support open_ioctx2 API

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18079 from liewegas/wip-21618

qa/standalone/scrub/osd-scrub-repair: no -y to diff

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #18088 from dillaman/wip-21629

osd: additional protection for out-of-bounds EC reads

Reviewed-by: Sage Weil <sage@redhat.com>

script/ptl-tool.py: Label Validation

Validated the label and verified the existence of the label at the beginning of build_branch().
This will avoid:
* Assigning invalid testing labels like 'abcd' and other existing labels like 'bug', 'cleanup' etc; as testing labels
* Stops spamming the Label list in the Ceph repo by creating new (invalid format) labels everytime. The labels can be created manually in github and ptl-tool only uses them.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge commit 'refs/pull/2/head' of github.com:batrick/ceph into ptl-tool-2

* refs/pull/2/head:
  script/ptl-tool.py: Fixed error message

Merge commit 'refs/pull/1/head' of github.com:batrick/ceph into ptl-tool-2

* refs/pull/1/head:
  script/ptl-tool.py: Fix the Comment example

os/bluestore: add bluestore_bluefs_min_free

We need at least ~1GB free so we can write out new SSTs (which are 256MB
each).

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: crash on enospc

We were passing this error back to rocksdb, which would then crash (much)
later with a corrupted SST file!  No good.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use normal Context for async deferred_try_submit

I'm not quite sure why the FunctionContext did not ever execute on the
finisher thread (perhaps the [&] captured some state on the stack that it
shouldn't have?).  In any case, using a traditional Context here appears
to resolve the problem (of the async deferred_try_submit() never executing,
leading to a bluestore stall/deadlock).

Fixes: http://tracker.ceph.com/issues/21470
Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: wake kv thread when blocking on deferred_bytes

We need to wake the kv thread whenever setting deferred_aggressive to
ensure that txns with deferred io that have committed but haven't submitted
their deferred writes get submitted.  This aligns us with the other
users of deferred_aggressive (e.g., _osr_drain_all).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18099 from dillaman/wip-21628

librbd: journal should ignore -EILSEQ errors from compare-and-write

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

osd/PG: add explicit CANCEL MBackfillReserve type

This lets us distinguish between REJECT and CANCEL for clarity.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: separate event for RemoteReservationCanceled

Right now we transparently map a RemoteReservationRejected into a
*Canceled event because this what peers send over the wire.  Even
once new peers start sending and explicit CANCEL, old peers will
still do so, so we'll maintain this mapping for a while.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17959 from gaosibei/branch2

RGW: Multipart upload may double the quota

osd/PG: separate verb 'Reject' from passive 'rejected'

This reduces pg->reject_reservation() callsites from 2 to 1 and
makes the state transitions a bit more explicit.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18087 from liu-chunmei/fix_dpdk_build

build: fix dpdk build error

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #18115 from mogeb/rados-doc

doc: add osd_max_object_size in osd configuration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18072 from dragonylffly/wip-optimize-assert

assert(false)->ceph_abort()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #18073 from dragonylffly/wip-check-monitor-address-configuration

mon: check monitor address configuration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18082 from Liuchang0812/wip-21624

os/bluestore: make BlueStore opened by start_kv_only umountable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18083 from Liuchang0812/wip-21625

tool: ceph-kvstore-tool doesn't umount BlueStore properly

Reviewed-by: Sage Weil <sage@redhat.com>

fix dpdk build error.
    when ./do_cmake.sh add  -DWITH_DPDK=ON -DUSE_CRYPTOPP=ON -DWITH_NSS=OFF -DWITH_TESTS=OFF
    if build debian package, need remove some file froms install since we disabled TESTS.

Signed-off-by: chunmei <chunmei.liu@intel.com>

doc: add osd_max_object_size in osd configuration

Having osd_max_write_size in the doc without osd_max_object_size
is confusing.

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

Merge pull request #18110 from ceph/wip-volume-api-change

ceph-volume lvm api refactor/move

Reviewed By: Andrew Schoen <aschoen@redhat.com>

Merge pull request #17380 from ceph/wip-21189

mgr/status: format byte quantities in base 2 multiples

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: list_children should not attempt to refresh image

The snap_lock is being held when this method is invoked, which can
result in a deadlock.

Fixes: http://tracker.ceph.com/issues/21670
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17746 from elliswuuuuuuuu/wip-prometheus

ceph-mgr: can not change prometheus port for mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Revert "REVERT msgr: force use of the SIGPIPE stopper for testing"

This reverts commit a76181df4a71ad15aee8dd2e4adcebce8330fae4.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

REVERT msgr: force use of the SIGPIPE stopper for testing

Note that since this sets MSG_NOSIGNAL to 0 it won't take effect
on the ::send[msg] invocations

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

msgr: add a mechanism for Solaris to avoid dying on SIGPIPE

This is fairly clean: we define an RAII object in the Messenger.h on
Solaris, and "declare" it with a macro in the implementations. There's
no code duplication and on Linux it's just entirely compiled out.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Revert "SIGPIPE suppression for platforms without SO_NOSIGPIPE or MSG_NOSIGNAL"

This reverts commit 131deb39769c1187c334ee84f552d3be01f1751b. It added
code that shouldn't have entered the project repo.

Conflicts:
src/msg/async/AsyncConnection.cc
src/msg/async/AsyncConnection.h
src/msg/simple/Pipe.cc
src/msg/simple/Pipe.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #18089 from liewegas/wip-localpool-min-size

mgr/localpool: default to 3x; allow min_size adjustment

Reviewed-by: John Spray <john.spray@redhat.com>

mgr/localpool: fix rule selection

The 'osd pool create' arg parsing is broken; the rule name for
'ceph osd pool create $name $numpgs replicated $rulename' is passed
via the erasure_code_profile param.  Too many req=false options
without a way to disambiguate them.

Work around it by passing both 'rule' and 'erasure_code_profile'
keys, so that if/when the hack in OSDMonitor.cc is removed it will
still work.  Blech.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume tests move lvm api tests into its new test module

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests refactor lvm api import for test_listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests refactor lvm api import for api tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests refactor lvm api import for activate tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests refactor lvm api import in conftest

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume devices.lvm refactor importing api to come from api/lvm.py

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume move lvm/api.py to api/lvm.py so disk can consume it

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume create an api module to hold common api modules

Signed-off-by: Alfredo Deza <adeza@redhat.com>

script/ptl-tool.py: Fix the Comment example

The example shown in the comment needs export to work as expected.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18048 from badone/wip-boost-mirrors

make-dist,cmake: Try multiple URLs to download boost before failing

Reviewed-by: Kefu Chai <kchai@redhat.com>

make-dist,cmake: Try multiple URLs to download boost before failing

Remove SPOF during boost download for make-dist and later cmake > 3.7

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

rgw: stop/join TokenCache revoke thread only if started.

Thread::join triggers an assert otherwise.

Fixes http://tracker.ceph.com/issues/21666

Signed-off-by: Karol Mroz <kmroz@suse.de>

Merge pull request #18097 from dillaman/wip-21663

qa/workunits/rbd: fixed variable name for resync image id

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

librbd: journal should ignore -EILSEQ errors from compare-and-write

Fixes: http://tracker.ceph.com/issues/21628
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/standalone/scrub/osd-scrub-repair: no -y to diff

With -y you can't see the entire line when it is long, which is
needed to identify the diff failure in
http://tracker.ceph.com/issues/21618

Instead, let the interactive user specify the option if they want it.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18052 from xiexingguo/wip-clear-pin-stats-invalid

osd/PrimaryLogPG: clear pin_stats_invalid bit properly on scrub-repair completion

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18061 from xiexingguo/wip-crc

osd/PrimaryLogPG: do not set data/omap digest blindly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18066 from shun-s/wip-objectstore-tool

tool: change default objectstore from filestore to bluestore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18078 from liewegas/wip-21614

qa/suites/rados/singleton/all/recover-preemption: handle slow starting osd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18095 from ceph/wip-bz1491250

ceph-volume lvm list

Reviewed by: Andrew Schoen <aschoen@redhat.com>

doc/ceph-volume add lvm/list to the toctree

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume create the reference for the list sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume add create and list to the index

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests create a unit test module for listing features

Signed-off-by: Alfredo Deza <adeza@redhat.com>

qa/workunits/rbd: fixed variable name for resync image id

Fixes: http://tracker.ceph.com/issues/21663
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18006 from adamwg/awg/rbd-mirror-default-data-pool

rbd-mirror: Improve data pool selection when creating images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17721 from amitkumar50/cov-librbd-4

librbd: address coverity false positives

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17696 from amitkumar50/cov-librbd-3

librbd: address coverity false positives

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17660 from gmayyyha/striping-feature-21360

librbd: fix don't send get_stripe_unit_count if striping is not enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #17461 from PCzhangPC/bench_rw_test

 rbd:introduce rbd bench rw(for read and write mix) test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17576 from ceph/wip-rm-1-minsize

qa/tests/rados: Remove unsupported 2-size-1-min-size config

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: always use CMAKE_RUNTIME_OUTPUT_DIRECTORY as the unittests' path

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/unittest_dns_resolve: #include <arpanameser_compat.h>

so it compiles on osx

Signed-off-by: Kefu Chai <kchai@redhat.com>