Merge pull request #8834 from liewegas/wip-pg-role

osd: set pg role more carefully

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6421 from xuechendi/wip-omap-out

os/filestore: add option to deploy omap to a separate device(path)

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6664 from ifed01/wip-13848

osd: more accurate disk space usage calculation when considering 'whiteout'/empty/small objects.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6971 from ukernel/jewel-14120

msg/simple/Pipe: avoid infinite loop in Pipe::do_recv()

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7050 from songbaisen/wipshowpg

mon: show the pg count in df output

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7872 from majianpeng/bluestore-write-buffer-rebuild

os/bluestore/KernelDevice: For buffer mode no need do bufferlist::rebuid

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8087 from songbaisen/song13

osd: copy blacklist in better way.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8800 from xiexingguo/xxg-wip-fixscrubwarn

mon/PGMonitor: fix inaccurate warn of unscrubbed pgs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8888 from liewegas/wip-mon-kv

mon: enable different backends (e.g., rocksdb)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #8905 from tchaikov/wip-15695

os/FileStore::sync_entry check for stop in after wait

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8923 from ceph/wip-selinux-relabel

rpm: Fix SELinux relabel on fedora

Reviewed-by: Petr Lautrbach <plautrba@redhat.com>

Merge pull request #8756 from Abhishekvrshny/fix-rgw-min-read-ahead

rgw: introduce rgw_list_bucket_min_readahead parameter for bucket list
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8726 from rzarzynski/wip-rgw-xstorageurl-fix

rgw: fix printing wrong X-Storage-Url in Swift's TempAuth.

Fixes: http://tracker.ceph.com/issues/15667
Reviewed-by: Orit Wasserman <owassern@redhat.com>

Merge pull request #8557 from SUSE/wip-15472

Test-enable XIO messenger and IB/RDMA support

Reviewed-by: Roi Dayan <roi.dayan@gmail.com>
Reviewed-by: Sage Weil <sweil@redhat.com>

Test-enable XIO messenger and IB/RDMA support

Fixes: http://tracker.ceph.com/issues/15472
Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8405 from dzafman/wip-15338

common: Fix ObjBencher clean_up when no prefix specified

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8935 from ceph/wip-jq

debian,rpm: add jq build dependency

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #8932 from dillaman/wip-15577

librbd: reduce log level for image format 1 warning

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rpm: Fix SELinux relabel on fedora

The SELinux userspace utilities stopped providing versions when they
switched to CIL language. We need to use a different technique to
relabel the files.

Fixes: #15725
Signed-off-by: Boris Ranto <branto@redhat.com>

debian,rpm: add jq build dependency

Needed since db517ba9176f1ee829453b016b1bd43c6054a555.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8720 from ceph/wip-13445

rpm: drop support for unsupported distros

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

librbd: reduce log level for image format 1 warning

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: fix printing wrong X-Storage-Url in Swift's TempAuth.

If the option "rgw_swift_account_in_url" is being set to true,
both user and subuser IDs will be present in the X-Storage-Url
header generated by the implementation of Swift API's TempAuth.
This patch rectifies this behavior and makes that only the user
identifier will be placed there.

Fixes: http://tracker.ceph.com/issues/15667
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Fixes for list-inconsistent-*

Improve rados scrub output
Add epoch for future repair-read feature
Add testing of rados scrub output

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kefu@redhat.com>

test: Add testing of new scrub commands in rados

Test list-inconsistent-pg
Test list-inconsistent-obj
Test inconsistent-snapset

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #8916 from linuxbox2/xio-cmake

cmake: <UPPERCASED_NAME>_FOUND in Findxio.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8768 from cbodley/wip-rgw-admin-zonegroup

radosgw-admin: add missing --zonegroup-id to usage

Merge pull request #8300 from jcsp/wip-standby-replay-reorg

mds: Fix standby replay MDS in upgrades and simplify standby replay logic (#15591)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rpm: do a full make check when --with-tests is given

The check-local target tests the CLI tools only.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop sysvinit bits from relabel_files function

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: replace literal paths with RPM macros

The only place we should write out literal paths is in the RPM scriptlets,
and there only for things that are not installed by this package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop udev/95-ceph-osd-alt.rules

This udev rules file was needed on older RHEL platforms, which are
unsupported as of jewel.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: global replace $RPM_BUILD_ROOT with %{buildroot}

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: put dependencies in alphabetical order

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: put /sbin/ldconfig into -p

This saves approximately 1 second per invocation.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop sysvinit-specific dependencies

Obsolete in jewel and above.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: move boost-random dependency to appropriate section

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: move unified python-sphinx build dependency

Now that the python-sphinx build dependency is unified, move it
to the proper section of the spec file.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop dead conditionals

This commit drops conditionals that no longer serve any purpose, since
jewel and above do not support the distro versions they are checking for.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop python-argparse dependency

This was only necessary for older (now unsupported) distro versions.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop _with_systemd

Ceph versions jewel and above only support systemd.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop init-ceph.in-fedora.patch

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: drop python_sitelib/sitearch conditional

This conditional was required to support older versions of RHEL/CentOS that are
no longer supported in infernalis and above.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8550 from xiexingguo/xxg-wip-cls

objclass: honour do_osd_ops() results before going any further

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8367 from Yan-waller/yj-wip-utthrottle

test/throttle: reassign the value of 'delay' in ThrottleTest.get

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8392 from dachary/wip-virtualenv-jewel

Makefile-env.am: set a default for CEPH_BUILD_VIRTUALENV (part 3)

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #8439 from ErwanAliasr1/evelu-file-leak

tests: Delete temporary files created by ceph_objectstore_tool.py

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8632 from xiexingguo/xxg-wip-deepscrub

osd: fix be_deep_scrub() can't catch omap error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8742 from majianpeng/bluestore-wal-transaction

os/bluestore: record released extents into bluestore_wal_transaction_t.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8763 from wjwithagen/patch-10

os/filestore: Make sure str-pointer is not 0x0 when referenced

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8808 from wjin/fast_heartbeat

osd: fast dispatch heartbeat message

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8811 from stiopaa1/log_includes

log: remove unnecessary includes from header file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8831 from wjin/clean

osd: clean heartbeat dispatch message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8821 from aderumier/deb-fix-default-ceph-location

debian: fix /etc/default/ceph location

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PG: update info.stats.* mappings on splitReviewed-by: Samuel Just <sjust@redhat.com>

osd/PG: update info.stats.* mappings on split

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8733 from xiexingguo/xxg-wip-fix-reweightlessutil

mon/OSDMonitor: improve reweight_by_utilization() logic

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8837 from liewegas/wip-reweight

mon/OSDMonitor: max_osds must be > 0 for reweight-by-*

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8926 from ceph/wip-ec-test-check

test/ec: build the libs only when 'make check'

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #8546 from jcsp/wip-15465

mds: fix auth caps with hyphen in path (#15465)

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #8484 from jcsp/wip-15399

mds: use epoch as incarnation counter (#15399)

Reviewed-by: Sage Weil <sage@redhat.com>

test/ec: build the libs only when 'make check'

Currently, we are always building the erasure code libraries while we
need them only when 'make check' is run. Moving the test libraries to
check_LTLIBRARIES should fix this for us.

We no longer need to remove the libec libs manually, remove the lines
that do that.

Signed-off-by: Boris Ranto <branto@redhat.com>

show the  total pg on each osd.
ceph pg dump osds
dumped osds in format plain
osdstat kbused kbavail kb hb in hb out pg sum
0 143304204 79623420 234882056 [1,2] [] 330
1 143304204 79623420 234882056 [0,2] [] 295
2 143304148 79623476 234882056 [0,1] [] 303
sum 429912556 238870316 704646168

Fixes:#14183
Signed-off-by: songbaisen song.baisen@zte.com.cn

Merge pull request #8876 from kmroz/wip-fix-rgw-lttng-segfault

global: don't link lttng into libglobal

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: <UPPERCASED_NAME>_FOUND in Findxio.cmake

For uniformity, upcase all these variables.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8637 from roidayan/xio-remove_rdma_and_ibverbs_from_configure

confiugre.ac: remove rdma and ibverbs dependency from xio

Merge pull request #8704 from ceph/wip-cmake-xio-libs

cmake: removed rdmacm & ibverbs parallel to PR#8637

Merge pull request #8914 from adamemerson/wip-grr-warnings

I have a hatred for warnings

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8917 from liewegas/wip-civetweb-fix

civetweb: combine various submodule changes

Reviewed-by: Adam Emerson <aemerson@redhat.com>

rados: Don't bother showing list-inconsistent-* errors that aren't set

Signed-off-by: David Zafman <dzafman@redhat.com>

osd, rados: Fixes for list-inconsistent-snapset

Signed-off-by: David Zafman <dzafman@redhat.com>

include, rados: Fixes for list-inconsistent-obj and librados

Signed-off-by: David Zafman <dzafman@redhat.com>

civetweb: combine various submodule changes

There were some accidental reverts, fixes, re-reverts,
and follow-on changes that are hard to follow and in the
end did not include what we want.  Merge it all together.

Signed-off-by: Sage Weil <sage@redhat.com>

global: don't link lttng into libglobal

Rely on dynamic initialization instead. Linking lttng in this way had
the unfortunate side effect of causing radosgw to segfault (when
daemonized) during sigterm processing (ie. during lttng_ust_exit()).

This was originally removed in 638738f, but accidentally re-added via
5f61d36.

Signed-off-by: Karol Mroz <kmroz@suse.com>

Merge pull request #8809 from trociny/wip-rbd-mirror-asok-restart

rbd-mirror: admin socket commands to start/stop/restart mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8775 from cy-lee/fix-15604-rbd-nbd-write-error

rbd-nbd: fix rbd-nbd aio callback error handling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: Fix warnings in chain_xattr tests

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #8910 from dillaman/wip-15709

librbd: unlock image if journal error encountered during lock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: Fix warnings in C write operations test

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #8907 from dillaman/wip-15690

librbd: possible race condition leads to use-after-free

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8793 from cbodley/wip-rgw-admin-modify

radosgw-admin: update usage for zone[group] modify

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

test/mon/mkfs.sh: clean up kv_backend file

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonitorDBStore: remember kv backend type

If it is specified, use it; otherwise, go by the config
option.

Record the type when creating the store.

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: unlock image if journal error encountered during lock

Explicitly unlock to prevent a client from accidentally blacklisting
itself when retrying the lock.

Fixes: http://tracker.ceph.com/issues/15709
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8893 from flyd1005/wip-fix-duplicated1

init-ceph.in: remove duplicated words

Merge pull request #8894 from flyd1005/wip-fix-duplicated2

os: remove duplicated words

Merge pull request #8895 from flyd1005/wip-fix-duplicated3

pybind/rados: remove duplicated words

Merge pull request #8896 from flyd1005/wip-fix-duplicated4

mon: remove duplicated words

Merge pull request #8898 from flyd1005/wip-fix-duplicated6

libcephfs.h: remove duplicated words

Merge pull request #8899 from flyd1005/wip-fix-duplicated7

erasure-code: remove duplicated words

Merge pull request #8900 from flyd1005/wip-fix-duplicated8

common: remove duplicated words

Merge pull request #8901 from flyd1005/wip-fix-duplicated9

cls_user: remove duplicated words

Reviewed-by: Sage Weil <sage@redhat.com>

mon/MonitorDBStore: {read,write}_meta

These are lifted from ObjectStore.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonitorDBStore: move backend instantiation out of ctor

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonitorDBStore: fix bug in dump output file

Signed-off-by: Sage Weil <sage@redhat.com>

mds: fix upgrades with replay daemons in map

In the new world, a standby replay daemon which
is actively replaying has its rank set.  In the old
world it did not: map this during the upgrade/downgrade
paths in decode/encode.

Fixes: http://tracker.ceph.com/issues/15591
Signed-off-by: John Spray <john.spray@redhat.com>

doc: add cephfs daemon management docs

Principally to tell people how to configure
standby, but more generally as a place to define
our terms for GIDs etc.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: omit fscid in single-filesystem status output

Signed-off-by: John Spray <john.spray@redhat.com>

mds: simplify standby/standby-replay logic

This used to use an arcane set of constants
in standby_for_rank, combined with daemons sometimes
sending requests to enter state STANDBY_REPLAY.

Simplify this so that there is only one place we
put daemons into standby replay, and that is
in tick->maybe_promote_staandby.

There is a behavioural change in here, which is
that we used to sometimes promote standbys to
be standby-replay, even if they didn't have
"standby replay = true", when they did have
a standby_for_rank or standby_for_name set.
I'm calling that a bug, and making it so that
daemons will only go into standby-replay if
"standby replay = true" is set.

Signed-off-by: John Spray <john.spray@redhat.com>

messages: add MMDSBeacon::standby_replay

So that daemons can explicitly say that they
want to be standby replay, instead of using
magic standby_for_rank values.

Because in some cases daemons would indicate this
with a beacon requesting STANDBY_REPLAY state, we
also munge these during decode to be normal STANDBY
requests with standby_replay=true.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: remove ONESHOT_REPLAY mode

This predated cephfs-journal-tool.  Don't bother
with any backwards-compatibility mechanisms here, as
it was a seriously niche undocumented thing.

On the off chance that someone ran an old ceph-mds against
a newer ceph-mon with oneshot enabled, it shouldn't break
anything, it just won't do what they want.

Signed-off-by: John Spray <john.spray@redhat.com>