]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agotest: Sort pool list because the order isn't guaranteed from "balancer pool ls" 31956/head
David Zafman [Fri, 20 Dec 2019 21:46:34 +0000 (13:46 -0800)]
test: Sort pool list because the order isn't guaranteed from "balancer pool ls"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c65d5c8d1421438da999698f1f734d6a61adb3bb)

5 years agomgr: Change default upmap_max_deviation to 5
David Zafman [Sat, 14 Dec 2019 01:43:44 +0000 (17:43 -0800)]
mgr: Change default upmap_max_deviation to 5

Fixes: https://tracker.ceph.com/issues/43312
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b0a1b758d012dfea40db3feca1a841c96f79defe)

5 years agoosdmaptool: Add --upmap-active to simulate active upmap balancing
David Zafman [Wed, 18 Dec 2019 19:27:02 +0000 (11:27 -0800)]
osdmaptool: Add --upmap-active to simulate active upmap balancing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 621acf8ce7f48253e9d2189a9a2ee432fa1d3ba1)

5 years agodoc: Add upmap options to osdmaptool man page and give example
David Zafman [Wed, 18 Dec 2019 03:38:51 +0000 (19:38 -0800)]
doc: Add upmap options to osdmaptool man page and give example

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 184e9d1ae3b5bcc332d5fe3330d46a5cb8fcacd6)

5 years agotools: osdmaptool document non-upmap options that were missing
David Zafman [Wed, 18 Dec 2019 01:35:14 +0000 (17:35 -0800)]
tools: osdmaptool document non-upmap options that were missing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e42a6ccb1819be4988e3ed7bd78fcf513f8d1589)

5 years agotest: Fix test case for pool based balancing instead of rule batched
David Zafman [Sat, 14 Dec 2019 01:14:51 +0000 (17:14 -0800)]
test: Fix test case for pool based balancing instead of rule batched

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8e46bbbf36e3d3795c1d3ba50c431ac80467da20)

5 years agoPartially revert "mgr/balancer: balance pools with same crush_rule in batch"
David Zafman [Fri, 13 Dec 2019 03:42:10 +0000 (19:42 -0800)]
Partially revert "mgr/balancer: balance pools with same crush_rule in batch"

This partially reverts commit 3a730d751deff892e7a0bddba87eba3dbb829c3e

Fixes: https://tracker.ceph.com/issues/43307
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7a1d6872c74a03b2493ecef0f2c4a5f9656a7c5e)

Conflicts:
src/pybind/mgr/balancer/module.py (No active+clean handling)

5 years agoRevert "tools: osdmaptool sync with balancer module behavior"
David Zafman [Tue, 10 Dec 2019 16:55:46 +0000 (08:55 -0800)]
Revert "tools: osdmaptool sync with balancer module behavior"

This was the rules batching version of osdmaptool

This reverts commit f165d4ca39b1edef4bf22a7597af79339c95026f.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a92f48773e9c59d49ee248de1d7951e7c9d3ac25)

5 years agorelease note: Add pending release notes for already merged code
David Zafman [Thu, 5 Dec 2019 17:52:04 +0000 (09:52 -0800)]
release note: Add pending release notes for already merged code

Follow on to https://github.com/ceph/ceph/pull/31774

Fix incorrect version number for Nautilus

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 65d03bae8b4f50cc3cbaa50640eaeab4cabd711f)

5 years agotools/osdmaptool.cc: do not use deprecated std::random_shuffle()
Kefu Chai [Wed, 4 Dec 2019 00:06:40 +0000 (16:06 -0800)]
tools/osdmaptool.cc: do not use deprecated std::random_shuffle()
the use of `std::random_shuffle()` was introduced by
b946308 .

in this change, it is replaced using `std::shuffle()`.

Fixes: https://tracker.ceph.com/issues/43084
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 30dfef208d859a67c1f1b13c314735861c536e85)

5 years agotest: Add test case based on Xie script in commit comment
David Zafman [Wed, 27 Nov 2019 18:50:01 +0000 (10:50 -0800)]
test: Add test case based on Xie script in commit comment

Other test fixes to reflect changes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9d2e0267e1f025a634ee4925c919008cacdd3d38)

5 years agoosd: ceph_pg_upmaps() use any overfull when there are still underfull
David Zafman [Wed, 27 Nov 2019 18:48:30 +0000 (10:48 -0800)]
osd: ceph_pg_upmaps() use any overfull when there are still underfull

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f38d38398f33f64bd49f017b1c8e737452157c53)

5 years agoosd: Create more_underfull with below target that aren't in underfull
David Zafman [Tue, 26 Nov 2019 23:19:01 +0000 (15:19 -0800)]
osd: Create more_underfull with below target that aren't in underfull

Use more_underfull for finding remaps for overfull OSDs

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fe03383e6691f6bd95189d3ba4d91fbb2ea9e9c8)

5 years agoosd/OSDMap.cc: don't output over/underfull messages to lderr
Neha Ojha [Mon, 11 Nov 2019 21:32:15 +0000 (13:32 -0800)]
osd/OSDMap.cc: don't output over/underfull messages to lderr

There can be cases where overfull and underfull(see example in
https://tracker.ceph.com/issues/42756) will be empty, which is not
necessarily an error. These error messages can end up spamming
the ceph-mgr log.

Fixes: https://tracker.ceph.com/issues/42756
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 3cf28f09a2ebdd922fc5c1d29c65469f8331e741)

5 years agoosd: calc_pg_upmaps() pick most overfull remap from try_pg_upmap()
David Zafman [Tue, 26 Nov 2019 05:38:16 +0000 (21:38 -0800)]
osd: calc_pg_upmaps() pick most overfull remap from try_pg_upmap()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f132061fd33c82dc131e18ae80cbec067b53cae)

5 years agoosd: ceph_pg_upmaps() use max_deviation to determine perfect distribution
David Zafman [Fri, 15 Nov 2019 05:10:28 +0000 (21:10 -0800)]
osd: ceph_pg_upmaps() use max_deviation to determine perfect distribution

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7f4ae9312314c329b63c850e09b403a95ad18bed)

5 years agotools: osdmaptool sync with balancer module behavior
David Zafman [Sat, 9 Nov 2019 00:23:14 +0000 (16:23 -0800)]
tools: osdmaptool sync with balancer module behavior

This can be backported only to nautilus

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f165d4ca39b1edef4bf22a7597af79339c95026f)

5 years agotools: osdmaptool: Perform upmap calculation as ceph-mgr does
David Zafman [Thu, 7 Nov 2019 04:32:48 +0000 (20:32 -0800)]
tools: osdmaptool: Perform upmap calculation as ceph-mgr does

This is the backportable commit that works with older balancer module.py

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b946308f03f1798915fafe7878f34b4a234c2ae4)

5 years agoosd: For balancer crush needs the rule passed to get_parent_of_type()
David Zafman [Wed, 20 Nov 2019 06:21:00 +0000 (22:21 -0800)]
osd: For balancer crush needs the rule passed to get_parent_of_type()

Fixes: https://tracker.ceph.com/issues/42718
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3eb69fd0cb15974e33e675babdd5d232d69c3c6d)

5 years agoosdmaptool: Match default max value of 10
David Zafman [Thu, 7 Nov 2019 04:31:21 +0000 (20:31 -0800)]
osdmaptool: Match default max value of 10

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 91bab14ea0e2ec7c94eb758cd76f9a620688725f)

5 years agotools: odsmaptool truncate target upmap file
David Zafman [Wed, 20 Nov 2019 19:55:15 +0000 (11:55 -0800)]
tools: odsmaptool truncate target upmap file

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a49324ceb9c322f792ea00720f438ea9ea09555f)

5 years agomgr: Fix balancer print
David Zafman [Wed, 20 Nov 2019 22:08:24 +0000 (14:08 -0800)]
mgr: Fix balancer print

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0af7e25620ae823d46eb842992570ddc1607cc4f)

5 years agoMerge pull request #32063 from smithfarm/wip-42878-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:54:46 +0000 (08:54 -0800)]
Merge pull request #32063 from smithfarm/wip-42878-nautilus

nautilus: common/admin_socket: Increase socket timeouts

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31850 from smithfarm/wip-43007-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:42:52 +0000 (08:42 -0800)]
Merge pull request #31850 from smithfarm/wip-43007-nautilus

nautilus: common/options: remove unused ms_msgr2_{sign,encrypt}

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32059 from smithfarm/wip-41954-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:42:03 +0000 (08:42 -0800)]
Merge pull request #32059 from smithfarm/wip-41954-nautilus

nautilus: tests/krbd: avoid hexdump skip and length options

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 years agoMerge pull request #31848 from smithfarm/wip-42989-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:41:34 +0000 (08:41 -0800)]
Merge pull request #31848 from smithfarm/wip-42989-nautilus

nautilus: tests: kernel.sh: update for read-only changes

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 years agoMerge pull request #31782 from rhcs-dashboard/wip-42809-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:41:01 +0000 (08:41 -0800)]
Merge pull request #31782 from rhcs-dashboard/wip-42809-nautilus

nautilus: mgr/dashboard: Handle always-on Ceph Manager modules correctly

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #31697 from k0ste/wip-42850-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:39:17 +0000 (08:39 -0800)]
Merge pull request #31697 from k0ste/wip-42850-nautilus

nautilus: pybind/mgr: Cancel output color control

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #32651 from tchaikov/nautilus-dts-8-on-arm64
Kefu Chai [Wed, 15 Jan 2020 15:37:19 +0000 (23:37 +0800)]
Merge pull request #32651 from tchaikov/nautilus-dts-8-on-arm64

nautilus: build/ops: install-deps,rpm: enable devtoolset-8 on aarch64 also

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #31791 from rhcs-dashboard/wip-42935-nautilus
Laura Paduano [Wed, 15 Jan 2020 15:29:05 +0000 (16:29 +0100)]
Merge pull request #31791 from rhcs-dashboard/wip-42935-nautilus

nautilus: mgr/dashboard: sort monitors by open sessions correctly.

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoinstall-deps,rpm: enable devtoolset-8 on aarch64 also 32651/head
Kefu Chai [Wed, 3 Apr 2019 03:08:04 +0000 (11:08 +0800)]
install-deps,rpm: enable devtoolset-8 on aarch64 also

in 5ae3b06e, we left aarch64 with devtoolset-7, but aarch64 builders are
also suffering from http://tracker.ceph.com/issues/38892. so i installed
devtoolset-8 on all aarch64 builders manually from the rpm packages
downloaded from koji builder:

- https://cbs.centos.org/koji/buildinfo?buildID=24923
- https://cbs.centos.org/koji/buildinfo?buildID=24931

libasan5, libubsan1 are not installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 00f19923b4225c25480ddb3c09a898a2444fbeb9)

5 years agoMerge pull request #31646 from batrick/i42818
Yuri Weinstein [Tue, 14 Jan 2020 20:35:30 +0000 (12:35 -0800)]
Merge pull request #31646 from batrick/i42818

nautilus: global: disable THP for Ceph daemons

5 years agoMerge pull request #31082 from smithfarm/wip-42279-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:34:39 +0000 (12:34 -0800)]
Merge pull request #31082 from smithfarm/wip-42279-nautilus

nautilus: tests/ceph.py: tolerate ECONNRESET errcode during logrotate

5 years agoMerge pull request #30820 from liewegas/wip-alerts-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:33:59 +0000 (12:33 -0800)]
Merge pull request #30820 from liewegas/wip-alerts-nautilus

nautilus: mgr/alerts: simple health alerts

5 years agoMerge pull request #30765 from smithfarm/wip-42129-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:33:17 +0000 (12:33 -0800)]
Merge pull request #30765 from smithfarm/wip-42129-nautilus

nautilus: doc/ceph-fuse: mention -k option in ceph-fuse man page

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #30739 from smithfarm/wip-41849-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:32:24 +0000 (12:32 -0800)]
Merge pull request #30739 from smithfarm/wip-41849-nautilus

nautilus: tools/setup-virtualenv.sh: do not default to python2.7

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30009 from smithfarm/wip-40126-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:14:40 +0000 (12:14 -0800)]
Merge pull request #30009 from smithfarm/wip-40126-nautilus

nautilus: mgr/zabbix: Adds possibility to send data to multiple zabbix servers.

Reviewed-by: Wido den Hollander <wido@widodh.nl>
5 years agomake-dist: drop Python 2/3 autoselect 30739/head
Nathan Cutler [Fri, 27 Dec 2019 09:55:21 +0000 (10:55 +0100)]
make-dist: drop Python 2/3 autoselect

virtualenv will prefer Python 3 over Python 2 if both are available.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 249bf81a32ff6d7046e525b947718d49b2314c13)

5 years agotools/setup-virtualenv.sh: do not default to python2.7
Nathan Cutler [Fri, 13 Sep 2019 14:12:52 +0000 (16:12 +0200)]
tools/setup-virtualenv.sh: do not default to python2.7

This commit changes the script's behavior when run without a --python option.
Before, we were running virtualenv with "--python python2.7" in that case.

That is not a reasonable default in September 2019, given that Python 2 EOL is
coming on January 1, 2020. Also, it breaks setup-virtualenv.sh in py3-only
environments.

With this patch, the script will not pass any "--python" option to virtualenv
(i.e., it will let virtualenv do whatever it thinks is best) in this case.

Fixes: https://tracker.ceph.com/issues/41821
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 78468e49ab2110d69bc34333c0ce90acb832de1a)

5 years agoMerge pull request #32593 from trociny/wip-43500-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:09:14 +0000 (17:09 -0800)]
Merge pull request #32593 from trociny/wip-43500-nautilus

nautilus: rbd-mirror: make logrotate work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32447 from trociny/wip-43428-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:08:33 +0000 (17:08 -0800)]
Merge pull request #32447 from trociny/wip-43428-nautilus

nautilus: rbd-mirror: fix 'rbd mirror status' asok command output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32086 from dillaman/wip-42630-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:07:51 +0000 (17:07 -0800)]
Merge pull request #32086 from dillaman/wip-42630-nautilus

nautilus: mgr: add 'rbd' profiles to support 'rbd_support' module commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agoMerge pull request #32167 from rzarzynski/wip-fips-zeroize-memset_bzero_nautilus
Neha Ojha [Mon, 13 Jan 2020 17:45:45 +0000 (09:45 -0800)]
Merge pull request #32167 from rzarzynski/wip-fips-zeroize-memset_bzero_nautilus

nautilus: common: FIPS: audit and switch some memset & bzero users

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31332 from joscollin/wip-41182-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:29:07 +0000 (08:29 -0800)]
Merge pull request #31332 from joscollin/wip-41182-nautilus

nautilus: mgr/volumes: fs subvolume resize command

5 years agoMerge pull request #31302 from smithfarm/wip-42142-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:28:18 +0000 (08:28 -0800)]
Merge pull request #31302 from smithfarm/wip-42142-nautilus

nautilus: mds: split the dir if the op makes it oversized, because some ops maybe in flight

5 years agoMerge pull request #31084 from smithfarm/wip-42424-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:27:47 +0000 (08:27 -0800)]
Merge pull request #31084 from smithfarm/wip-42424-nautilus

nautilus: tests: whitelist "Error recovering journal" for cephfs-data-scan

5 years agoMerge pull request #31083 from smithfarm/wip-42422-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:27:10 +0000 (08:27 -0800)]
Merge pull request #31083 from smithfarm/wip-42422-nautilus

nautilus: tests/cephfs.py: allow client mount to reset fully

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #31081 from smithfarm/wip-42158-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:25:59 +0000 (08:25 -0800)]
Merge pull request #31081 from smithfarm/wip-42158-nautilus

nautilus: cephfs: osdc/objecter: Fix last_sent in scientific format and add age to ops

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #30769 from smithfarm/wip-41888-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:19:41 +0000 (12:19 -0800)]
Merge pull request #30769 from smithfarm/wip-41888-nautilus

nautilus: cephfs: "client: fix lazyio_synchronize() to update file size" and "libcephfs: Add Tests for LazyIO"

5 years agoMerge pull request #30767 from smithfarm/wip-42147-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:18:54 +0000 (12:18 -0800)]
Merge pull request #30767 from smithfarm/wip-42147-nautilus

nautilus: cephfs: mds returns -5(EIO) error when the deleted file does not exist

5 years agoMerge pull request #30766 from smithfarm/wip-42145-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:18:13 +0000 (12:18 -0800)]
Merge pull request #30766 from smithfarm/wip-42145-nautilus

nautilus: cephfs: client: remove Inode.dir_contacts field and handle bad whence value to llseek gracefully

5 years agoMerge pull request #30764 from smithfarm/wip-42121-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:17:43 +0000 (12:17 -0800)]
Merge pull request #30764 from smithfarm/wip-42121-nautilus

nautilus: cephfs: client: add procession of SEEK_HOLE and SEEK_DATA in lseek

5 years agoMerge pull request #30763 from smithfarm/wip-42040-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:17:01 +0000 (12:17 -0800)]
Merge pull request #30763 from smithfarm/wip-42040-nautilus

nautilus: cephfs: client: _readdir_cache_cb() may use the readdir_cache already clear

5 years agoMerge pull request #30762 from smithfarm/wip-42035-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:16:26 +0000 (12:16 -0800)]
Merge pull request #30762 from smithfarm/wip-42035-nautilus

nautilus: cephfs: client: EINVAL may be returned when offset is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #30761 from smithfarm/wip-41899-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:15:42 +0000 (12:15 -0800)]
Merge pull request #30761 from smithfarm/wip-41899-nautilus

nautilus: mds: "apply configuration changes through MDSRank" and "recall caps from quiescent sessions" and "drive cap recall while dropping cache"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #31518 from trociny/wip-42726-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 16:13:59 +0000 (08:13 -0800)]
Merge pull request #31518 from trociny/wip-42726-nautilus

nautilus: rbd-mirror: clone v2 mirroring improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32133 from dillaman/wip-43212-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 16:13:03 +0000 (08:13 -0800)]
Merge pull request #32133 from dillaman/wip-43212-nautilus

nautilus: mgr/dashboard: block mirroring page results in internal server error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agologrotate: also sighup rbd-mirror 32593/head
Mykola Golub [Tue, 31 Dec 2019 10:01:31 +0000 (10:01 +0000)]
logrotate: also sighup rbd-mirror

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 86424fc3c895995d1d45f067c7852e6dce993027)

Conflicts:
src/cephadm/cephadm (does not exist)

5 years agorbd-mirror: reopen all contexts logs on SIGHUP
Mykola Golub [Tue, 31 Dec 2019 09:57:15 +0000 (09:57 +0000)]
rbd-mirror: reopen all contexts logs on SIGHUP

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 9ddf111506611a596c713ffe861a41aeda05e7a5)

Conflicts:
src/tools/rbd_mirror/Mirror.cc (std::lock_guard vs Mutex::Locker)
src/tools/rbd_mirror/PoolReplayer.cc (std::lock_guard vs Mutex::Locker)

5 years agorbd-mirror: delay local/remote rados initialization until context created
Mykola Golub [Sun, 5 Jan 2020 09:10:24 +0000 (09:10 +0000)]
rbd-mirror: delay local/remote rados initialization until context created

We rely on that if a rados ref is initialized it contains the
valid context.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit f3b49191771d2a3a20a7f55a14e0a7482ee96172)

5 years agoPendingReleaseNotes: reference new MGR profile cap support 32086/head
Jason Dillaman [Fri, 6 Dec 2019 15:16:38 +0000 (10:16 -0500)]
PendingReleaseNotes: reference new MGR profile cap support

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agopybind/mgr: use custom exception to handle authorization failures
Jason Dillaman [Sun, 27 Oct 2019 19:56:34 +0000 (15:56 -0400)]
pybind/mgr: use custom exception to handle authorization failures

Python2 does not include the PermissionError exception so use a
custom exception to support both versions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7683f5f259bbeb6fcfc6a18c8e1a9551e068e847)

5 years agoqa/workunits/rbd: add permission tests for mgr profile
Jason Dillaman [Tue, 22 Oct 2019 15:19:14 +0000 (11:19 -0400)]
qa/workunits/rbd: add permission tests for mgr profile

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7cccdef25e3c35a743414d5a7a6e0b10b8878b9b)

5 years agomgr: added placeholder 'osd' and 'mds' profiles
Jason Dillaman [Tue, 22 Oct 2019 13:11:34 +0000 (09:11 -0400)]
mgr: added placeholder 'osd' and 'mds' profiles

CephFS documents a previously unhandled profile named 'mds' that
needs to be accepted as valid (but currently provides no caps).
Similarly, the 'osd' cap is deployed in several scenarios as well.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2dc7a959509325f8da23d0baef2509174477398c)

5 years agodoc/rbd: add new 'profile rbd' mgr caps to examples
Jason Dillaman [Mon, 14 Oct 2019 16:45:56 +0000 (12:45 -0400)]
doc/rbd: add new 'profile rbd' mgr caps to examples

Fixes: https://tracker.ceph.com/issues/42265
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5137cd80db7c9b699378ce22a91fbe25aba321f1)

Conflicts:
doc/rbd/rbd-kubernetes.rst: doc does not exist

5 years agopybind/mgr: test session authorization against specific pools/namespaces
Jason Dillaman [Mon, 14 Oct 2019 16:34:35 +0000 (12:34 -0400)]
pybind/mgr: test session authorization against specific pools/namespaces

Use the optional "pool" and "namespace" constraints for the RBD cap
profiles to ensure the user has access to touch the specified
pool/namespace combo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cd71413f57fa2285e9349a07449e3fcf345eeb48)

5 years agomgr: python modules can now perform authorization tests
Jason Dillaman [Mon, 14 Oct 2019 14:39:54 +0000 (10:39 -0400)]
mgr: python modules can now perform authorization tests

In cases where the python service or individual python modules are
enabled via caps, the module might want to perform finer grained
tests to ensure specific commands are allowed. An example of this is
the 'rbd_support' module limiting access by pools and namespaces.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 282c31c383856b45caadcefb876a71e39fe4b219)

Conflicts:
src/mgr/ActivePyModules.cc: tweaked due to changes introduced in 4878509652
src/mgr/DaemonServer.cc: tweaked due to changes introduced in 4878509652
src/mgr/MgrSession.h: trivial resolition

5 years agomgr: validate that profile caps are actually valid
Jason Dillaman [Thu, 17 Oct 2019 13:37:13 +0000 (09:37 -0400)]
mgr: validate that profile caps are actually valid

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9193d8762347af65bebdaea673175113f7ccd01a)

5 years agomgr: added 'profile rbd/rbd-read-only' cap
Jason Dillaman [Mon, 14 Oct 2019 14:54:44 +0000 (10:54 -0400)]
mgr: added 'profile rbd/rbd-read-only' cap

These profiles can be further restricted via 'pool' and
'namespace' argument optionals.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b0d73aed19a9f3f242d5a13757016759835c164c)

Conflicts:
doc/rados/operations/user-management.rst: trivial resolution

5 years agomgr: support optional arguments for module and profile caps
Jason Dillaman [Fri, 11 Oct 2019 16:44:01 +0000 (12:44 -0400)]
mgr: support optional arguments for module and profile caps

This allows an optional, arbitrary key/value constraint clauses to
be appended to "profile XYZ" and "allow module XYZ" caps. A module
can then provide additional validatation against these meta-arguments.

Example:
profile rbd pool=rbd
allow module rbd_support with pool=rbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cb534e0049afc4f9259812a6624744bbdbaabafa)

5 years agomgr: add new 'allow module' cap to MgrCap
Jason Dillaman [Fri, 11 Oct 2019 15:25:56 +0000 (11:25 -0400)]
mgr: add new 'allow module' cap to MgrCap

This allows specific python add-on modules to be whitelisted instead
of manually adding each command exported by the module.

  allow module {module-name} {access-spec}

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3463613bd43d5fb7f8094c3b9b8a3163325ff196)

5 years agomon: dropped daemon type argument for MonCap
Jason Dillaman [Fri, 11 Oct 2019 14:45:15 +0000 (10:45 -0400)]
mon: dropped daemon type argument for MonCap

This was a placeholder for handling MGR caps within the MonCap
class. Now that the MGR has its own MgrCap class, this is no
longer required.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dbb1b54d492134b677b4e239415c0579e34032b4)

Conflicts:
src/mon/MonCap.h: trivial resolution
src/mon/Monitor.cc: trivial resolution

5 years agomgr: stop re-using MonCap for handling MGR caps
Jason Dillaman [Fri, 11 Oct 2019 14:30:03 +0000 (10:30 -0400)]
mgr: stop re-using MonCap for handling MGR caps

Use the existing MonCap as the basis for a new custom MgrCap
processor.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6350bee5c873cb1cef0ca1345b89bff66817e0c1)

5 years agorbd-mirror: fix 'rbd mirror status' asok command output 32447/head
Mykola Golub [Mon, 30 Dec 2019 15:44:32 +0000 (17:44 +0200)]
rbd-mirror: fix 'rbd mirror status' asok command output

This was broken by def50d04796, and implicitly fixed during
refactoring in the master (octopus) by adf1486e46c, hence it is a
direct commit to nautilus branch.

Fixes: https://tracker.ceph.com/issues/43429
Signed-off-by: Mykola Golub <mgolub@suse.com>
5 years agoMerge pull request #32520 from sseshasa/wip-43495-nautilus
Yuri Weinstein [Thu, 9 Jan 2020 16:56:51 +0000 (08:56 -0800)]
Merge pull request #32520 from sseshasa/wip-43495-nautilus

nautilus: mon/OSDMonitor: Don't update mon cache settings if rocksdb is not used

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32056 from smithfarm/wip-42735-nautilus
Yuri Weinstein [Thu, 9 Jan 2020 16:55:00 +0000 (08:55 -0800)]
Merge pull request #32056 from smithfarm/wip-42735-nautilus

nautilus: rgw: allow reshard log entries for non-existent buckets to be cancelled

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #32055 from smithfarm/wip-42733-nautilus
Yuri Weinstein [Thu, 9 Jan 2020 16:54:36 +0000 (08:54 -0800)]
Merge pull request #32055 from smithfarm/wip-42733-nautilus

nautilus: rgw: auto-clean reshard queue entries for non-existent buckets

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #31683 from smithfarm/wip-42840-nautilus
Yuri Weinstein [Thu, 9 Jan 2020 16:54:12 +0000 (08:54 -0800)]
Merge pull request #31683 from smithfarm/wip-42840-nautilus

nautilus: rgw: when resharding store progress json

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #32058 from smithfarm/wip-43161-nautilus
Yuri Weinstein [Thu, 9 Jan 2020 16:53:36 +0000 (08:53 -0800)]
Merge pull request #32058 from smithfarm/wip-43161-nautilus

nautilus: rgw: url encode common prefixes for List Objects response

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge remote-tracking branch 'origin/nautilus-saved' into nautilus
Yuri Weinstein [Thu, 9 Jan 2020 00:13:09 +0000 (00:13 +0000)]
Merge remote-tracking branch 'origin/nautilus-saved' into nautilus

5 years ago14.2.6 v14.2.6
Jenkins Build Slave User [Wed, 8 Jan 2020 18:36:54 +0000 (18:36 +0000)]
14.2.6

5 years agodoc: update inf/infinite option for subvolume resize 31332/head
Jos Collin [Sat, 26 Oct 2019 16:02:33 +0000 (21:32 +0530)]
doc: update inf/infinite option for subvolume resize

Fixes: https://tracker.ceph.com/issues/42479
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit e5bad7d4a6fa598042e718924fadbddf9ac64e55)

5 years agodoc: fs subvolume resize command
Jos Collin [Thu, 17 Oct 2019 14:25:47 +0000 (19:55 +0530)]
doc: fs subvolume resize command

Fixes: https://tracker.ceph.com/issues/41182
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 04b6b737153940c852c4b6771fb53a74a0c73bbe)

5 years agoqa/tasks/cbt: install python3 deps
Kefu Chai [Fri, 27 Dec 2019 11:03:24 +0000 (19:03 +0800)]
qa/tasks/cbt: install python3 deps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 299375852a4b3210cf5b95d3559d69080f14339f)

Conflicts:
qa/tasks/cbt.py: install python36-* instead of python3-*
packages. as in EL7, python packages is still versioned using minor
version, and they don't "Provide" python3-*.

5 years agomon/OSDMonitor: Don't update mon cache settings if rocksdb is not used 32520/head
Sridhar Seshasayee [Fri, 3 Jan 2020 09:26:13 +0000 (14:56 +0530)]
mon/OSDMonitor: Don't update mon cache settings if rocksdb is not used

Currently any update to mon cache config settings like 'mon_memory_target'
or 'rocksdb_cache_size' will cause a crash in case rocksdb and priority
cache manager is not being used.

If rocksdb (and therefore priority cache manager) is not being used, then
the update to the mon caches is ignored. Also, reduce the log levels of
some important log messages to help in debugging.

Fixes: https://tracker.ceph.com/issues/43454
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit b9e7a7af98ca284f4febc37fede58345ecb8a1d1)

5 years agoMerge pull request #32436 from tchaikov/nautilus-cbt-py3
Kefu Chai [Tue, 7 Jan 2020 01:34:25 +0000 (09:34 +0800)]
Merge pull request #32436 from tchaikov/nautilus-cbt-py3

nautilus: qa/tasks/cbt: install python3 deps

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoqa/tasks/cbt: install python3 deps 32436/head
Kefu Chai [Fri, 27 Dec 2019 11:03:24 +0000 (19:03 +0800)]
qa/tasks/cbt: install python3 deps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 299375852a4b3210cf5b95d3559d69080f14339f)

Conflicts:
qa/tasks/cbt.py: install python36-* instead of python3-*
packages. as in EL7, python packages is still versioned using minor
version, and they don't "Provide" python3-*.

5 years agomon/PGMap.h: disable network stats in dump_osd_stats
Neha [Mon, 23 Dec 2019 18:12:59 +0000 (18:12 +0000)]
mon/PGMap.h: disable network stats in dump_osd_stats

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ffb8a2a904426c8a3d6fc2fd70266418a98623c5)

 Conflicts:
src/mon/PGMap.h - trivial resolution

5 years agoosd_stat_t::dump: Add option for ceph-mgr pythonn callers to skip ping network
David Zafman [Fri, 20 Dec 2019 23:53:49 +0000 (15:53 -0800)]
osd_stat_t::dump: Add option for ceph-mgr pythonn callers to skip ping network

Due to the overhead of this on large clusters, we removed the network
information was wasn't getting used anyway.

Fixes: https://tracker.ceph.com/issues/43364
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1e2fe722c41d4cc34094afb157b3eb06b4a50972)

 Conflicts:
src/mon/PGMap.cc - trivial resolution
src/mon/PGMap.h - trivial resolution
src/osd/osd_types.h  - trivial resolution

5 years agoMerge pull request #32466 from neha-ojha/wip-43364-2-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:24:37 +0000 (15:24 -0800)]
Merge pull request #32466 from neha-ojha/wip-43364-2-nautilus

nautilus: mon/PGMap.h: disable network stats in dump_osd_stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #32064 from smithfarm/wip-42997-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:18:18 +0000 (15:18 -0800)]
Merge pull request #32064 from smithfarm/wip-42997-nautilus

nautilus: osd/PeeringState: do not exclude up from acting_recovery_backfill

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32062 from smithfarm/wip-42853-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:17:53 +0000 (15:17 -0800)]
Merge pull request #32062 from smithfarm/wip-42853-nautilus

nautilus: mon/OSDMonitor: fix format error ceph osd stat --format json

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31740 from smithfarm/wip-42885-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:17:27 +0000 (15:17 -0800)]
Merge pull request #31740 from smithfarm/wip-42885-nautilus

nautilus: core: mon/OSDMonitor: Fix pool set target_size_bytes (etc) with unit suffix

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31736 from SUSE/wip-42846-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:15:57 +0000 (15:15 -0800)]
Merge pull request #31736 from SUSE/wip-42846-nautilus

nautilus: msg: async/net_handler.cc: Fix compilation

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #32067 from smithfarm/wip-42899-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 23:15:24 +0000 (15:15 -0800)]
Merge pull request #32067 from smithfarm/wip-42899-nautilus

nautilus: mgr/pg_autoscaler: only generate target_* health warnings if targets set

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31862 from smithfarm/wip-43012-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 20:26:21 +0000 (12:26 -0800)]
Merge pull request #31862 from smithfarm/wip-43012-nautilus

nautilus: rgw: crypt: permit RGW-AUTO/default with SSE-S3 headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #31684 from smithfarm/wip-42841-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 20:25:54 +0000 (12:25 -0800)]
Merge pull request #31684 from smithfarm/wip-42841-nautilus

nautilus: rgw: fix an endless loop error when to show usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #31298 from smithfarm/wip-42555-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 20:25:23 +0000 (12:25 -0800)]
Merge pull request #31298 from smithfarm/wip-42555-nautilus

nautilus: rgw: prevent bucket reshard scheduling if bucket is resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #31289 from smithfarm/wip-42386-nautilus
Yuri Weinstein [Mon, 6 Jan 2020 20:25:04 +0000 (12:25 -0800)]
Merge pull request #31289 from smithfarm/wip-42386-nautilus

nautilus: rgw: fix compile errors with boost 1.70

5 years agoMerge pull request #31182 from croit/nautilus-rgw-bucket-stats-num-shards
Yuri Weinstein [Mon, 6 Jan 2020 20:24:18 +0000 (12:24 -0800)]
Merge pull request #31182 from croit/nautilus-rgw-bucket-stats-num-shards

nautilus: rgw: add num_shards to radosgw-admin bucket stats