]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocephadm: Revert "mgr/cephadm: fixing call method timeout handling"
Adam King [Thu, 12 May 2022 14:09:43 +0000 (10:09 -0400)]
cephadm: Revert "mgr/cephadm: fixing call method timeout handling"

This reverts commit 55d42d0c31a5414cb2beac9273faa7123efb8709.

Fixes: https://tracker.ceph.com/issues/55623
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit c6e63de4fc51ca54ca2f60ef800222a1b3cfde87)

3 years agocephadm: fix adoption of osds from custom name clusters
Adam King [Fri, 13 May 2022 16:53:09 +0000 (12:53 -0400)]
cephadm: fix adoption of osds from custom name clusters

Fixes: https://tracker.ceph.com/issues/55654
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 361c71c7321929898a9cc381b05f4cd65aba36f7)

3 years agomgr/cephadm: Disable external snapshots in grafana
Aashish Sharma [Thu, 5 May 2022 08:22:33 +0000 (13:52 +0530)]
mgr/cephadm: Disable external snapshots in grafana

This PR intends to disable the external snapshots in grafana to reduce security threats

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 334f4e141c8176df09d1820a1396d434e87781ae)

3 years agomgr/nfs: add test for access type validation func
John Mulligan [Thu, 21 Apr 2022 19:34:25 +0000 (15:34 -0400)]
mgr/nfs: add test for access type validation func

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit bb2b45d7036cd6fe03c45f349c86302d7fbfbbf1)

3 years agomgr/nfs: add test for squash validation func
John Mulligan [Thu, 21 Apr 2022 19:32:33 +0000 (15:32 -0400)]
mgr/nfs: add test for squash validation func

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2bcd911e3146ed6429a84cccf87f447525c124bc)

3 years agomgr/nfs: lift validate access type method to file level function
John Mulligan [Thu, 21 Apr 2022 19:27:40 +0000 (15:27 -0400)]
mgr/nfs: lift validate access type method to file level function

More reusable across the entire file and made private, shrinking
the public surface area of the type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 3d17b9ee74b01a0dec3d4f2b50ab242a91259faf)

3 years agomgr/nfs: lift validate squash method to file level function
John Mulligan [Thu, 21 Apr 2022 19:17:40 +0000 (15:17 -0400)]
mgr/nfs: lift validate squash method to file level function

More reusable across the entire file and made private, shrinking
the public surface area of the type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2587c29c8a8fe496b9b8fd705aeee6fb6333a71f)

3 years agomgr/nfs: move write_block func to file level format_block
John Mulligan [Fri, 4 Mar 2022 20:04:56 +0000 (15:04 -0500)]
mgr/nfs: move write_block func to file level format_block

The method was an unnecessary use of staticmethod as well as a
confusingly named function. Move the file and rename it for clarity.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 42e0a9ad9f813f64978f60dda25984ec2760d066)

3 years agomgr/nfs: lift _format_val function to file level
John Mulligan [Fri, 4 Mar 2022 19:53:38 +0000 (14:53 -0500)]
mgr/nfs: lift _format_val function to file level

There's no advantage to having this function as a closure and it makes
reading the code (and testing it) more challenging.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 1a7b7e35b261c9e227f0540ae5310e68a7e8bfe4)

3 years agomgr/nfs: simplify implementation of _indentation function
John Mulligan [Thu, 21 Apr 2022 19:05:13 +0000 (15:05 -0400)]
mgr/nfs: simplify implementation of _indentation function

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 6b33cec65724db7c0894d4aa2bdf320448eb2dc4)

3 years agomgr/nfs: lift generic indentation function to file level
John Mulligan [Fri, 4 Mar 2022 19:40:33 +0000 (14:40 -0500)]
mgr/nfs: lift generic indentation function to file level

Tucking a private method away as a staticmethod makes it harder to reuse
and less obvious that it can be resused.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d791d5c2291970b750ec73aa8862dd9369b92da0)

3 years agomgr/nfs: rename export_utils.py to ganesha_conf.py
John Mulligan [Fri, 4 Mar 2022 19:28:43 +0000 (14:28 -0500)]
mgr/nfs: rename export_utils.py to ganesha_conf.py

This new name is clearer about what topic the code in the file covers.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit bc895b6a7f3955d59ebbf5ca55a3cb6d2b56f439)

3 years agocephfs/test_nfs: Add test for dynamic update export
Ramana Raja [Fri, 11 Mar 2022 19:49:42 +0000 (14:49 -0500)]
cephfs/test_nfs: Add test for dynamic update export

Add test to verify that the NFS servers don't restart when the
access type of a CephFS NFS export is updated.

And check the NFS servers are restarted when the pseudo path of
a CephFS NFS export is updated.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit d7904e78d13c24b305b4e83f15ee2b86ab2085e5)

3 years agocephfs/test_nfs: fix _write_to_read_only_export()
Ramana Raja [Fri, 11 Mar 2022 21:36:44 +0000 (16:36 -0500)]
cephfs/test_nfs: fix _write_to_read_only_export()

... to fail the test if write to read only export succeeded.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 3c22f6eb716b4b4f2817566a5b14350f7dd3310e)

3 years agomgr/nfs: Don't notify ganesha rados object
Ramana Raja [Wed, 26 Jan 2022 04:30:26 +0000 (23:30 -0500)]
mgr/nfs: Don't notify ganesha rados object

... to update export if the ganesha server is going to be
restarted. The ganesha server restart will reload the exports.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 4df294b7fafc308611b75c4656a40c42d1d95361)

3 years agomgr/nfs: allow dynamic update of cephfs nfs export
Ramana Raja [Tue, 25 Jan 2022 01:06:11 +0000 (20:06 -0500)]
mgr/nfs: allow dynamic update of cephfs nfs export

mgr/nfs module's apply_export() method is used to update an existing
CephFS NFS export. The method always restarted the ganesha service (
ganesha server cluster) after updating the export object and notifying
the ganesha servers to reload their exports. The restart temporarily
affected the clients connections of all the exports served by the
ganesha servers.

It is not always necessary to restart the ganesha servers. Only
updating the export ID, path, or FSAL block of a CephFS NFS export
requires a restart. So modify apply_export() to only restart the
ganesha servers for such export updates.

The mgr/nfs module creates a FSAL ceph user with read-only or
read-write path restricted MDS caps for each export. To change the
access type of the CephFS NFS export, the MDS caps of the export's FSAL
ceph user must also be changed. Ganesha can dynamically enforce an
export's access type changes, but Ceph server daemons can't dynamically
enforce changes in caps of the Ceph clients. To allow dynamic updates
of CephFS NFS exports, always create a FSAL Ceph user with read-write
path restricted MDS caps per export. Rely on the ganesha servers to
enforce the export access type changes for the NFS clients.

Fixes: https://tracker.ceph.com/issues/54025
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 2415e03217b4afe9e430863da0b4503be254c425)

3 years agomgr/cephadm: adding logic to close ports when removing a daemon
Redouane Kachach [Tue, 26 Apr 2022 10:30:38 +0000 (12:30 +0200)]
mgr/cephadm: adding logic to close ports when removing a daemon
Fixes: https://tracker.ceph.com/issues/52906
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 4deb546ffd67ac8f05d2788150764a26b5671b87)

3 years agodoc/cephadm/services: Add missing ceph command to orch apply
bluikko [Thu, 28 Apr 2022 06:42:31 +0000 (13:42 +0700)]
doc/cephadm/services: Add missing ceph command to orch apply

In cephadm service management documentation several of the
ceph orch commands are missing the ceph part, mostly in
ceph orch apply commands but not all of them.

Add ceph in the front of the command to make them consistent
with all other commands.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 4d96410826f31ea9c3bf5f3322a9391e257ffb76)

3 years agoMerge pull request #46311 from b-ranto/wip-alerts-quincy
Boris Ranto [Thu, 19 May 2022 11:27:11 +0000 (13:27 +0200)]
Merge pull request #46311 from b-ranto/wip-alerts-quincy

[quincy] mgr/alerts: Add Message-Id and Date header to sent emails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge pull request #46325 from zdover23/wip-pr-46315-backport-to-quincy
zdover23 [Wed, 18 May 2022 23:21:48 +0000 (09:21 +1000)]
Merge pull request #46325 from zdover23/wip-pr-46315-backport-to-quincy

quincy: doc/start: s/3/three/ in intro.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agodoc/start: s/3/three/ in intro.rst 46325/head
Zac Dover [Wed, 18 May 2022 10:36:53 +0000 (20:36 +1000)]
doc/start: s/3/three/ in intro.rst

I'm changing "3" to "three" for two reasons:

1. It's correct.
2. This allows me to test backports into Octopus, Pacific, and Quincy.
   I am particularly interested to see what happens when I attempt
   the backport into Octopus, because backports into Octopus have
   failed. This will provide me with another unit of data.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 28efcec2d65e85ff2fa54e62b5b134e63ace853b)

3 years agoMerge pull request #46212 from idryomov/wip-use-system-libpmem-on-c9s-quincy
Yuri Weinstein [Wed, 18 May 2022 15:13:59 +0000 (08:13 -0700)]
Merge pull request #46212 from idryomov/wip-use-system-libpmem-on-c9s-quincy

quincy: rpm: use system libpmem on Centos 9 Stream

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #46094 from idryomov/wip-55389-quincy
Yuri Weinstein [Wed, 18 May 2022 15:13:01 +0000 (08:13 -0700)]
Merge pull request #46094 from idryomov/wip-55389-quincy

quincy: librbd/cache/pwl: fix bit field endianness issue

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agomgr/alerts: Add Date header to mails sent by alerts module 46311/head
Lorenz Bausch [Tue, 8 Mar 2022 13:45:03 +0000 (14:45 +0100)]
mgr/alerts: Add Date header to mails sent by alerts module

Signed-off-by: Lorenz Bausch <info@lorenzbausch.de>
(cherry picked from commit 4031917739c1e17d909a3fd53fb74fd0fc86030a)

3 years agomgr/alerts: Generate Message-Id for mails sent by alerts module
Lorenz Bausch [Tue, 8 Mar 2022 13:36:20 +0000 (14:36 +0100)]
mgr/alerts: Generate Message-Id for mails sent by alerts module

Signed-off-by: Lorenz Bausch <info@lorenzbausch.de>
(cherry picked from commit ef5975e701cfeb58f2dd8fb9f0410d08692fa757)

3 years agoMerge pull request #46113 from Matan-B/wip-55513-quincy
Yuri Weinstein [Tue, 17 May 2022 22:58:13 +0000 (15:58 -0700)]
Merge pull request #46113 from Matan-B/wip-55513-quincy

quincy: mount/conf: Fix IPv6 parsing

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #45871 from idryomov/wip-stretch-last-force-resend-quincy
Yuri Weinstein [Tue, 17 May 2022 22:56:33 +0000 (15:56 -0700)]
Merge pull request #45871 from idryomov/wip-stretch-last-force-resend-quincy

quincy: mon/OSDMonitor: properly set last_force_op_resend in stretch mode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
3 years agoMerge pull request #46287 from zdover23/wip-doc-tracker-55676-backport-quincy
zdover23 [Tue, 17 May 2022 15:07:31 +0000 (01:07 +1000)]
Merge pull request #46287 from zdover23/wip-doc-tracker-55676-backport-quincy

quincy: doc/dev: update basic-workflow.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agodoc/dev: update basic-workflow.rst 46287/head
Zac Dover [Wed, 13 Apr 2022 14:09:38 +0000 (00:09 +1000)]
doc/dev: update basic-workflow.rst

This PR updates the basic-workflow.rst file
to serve the needs of people in 2022 who were not
present at jump street.

The text has been refined up to the section called
"Integration Tests" (non-inclusive).

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit a227e4007a5ce66b63e42facf97f89655edf2169)

3 years agoMerge pull request #45814 from p-se/wip-55222-quincy
Ernesto Puerta [Fri, 13 May 2022 09:09:24 +0000 (11:09 +0200)]
Merge pull request #45814 from p-se/wip-55222-quincy

quincy: mgr/dashboard: Compare values of MTU alert by device

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #46220 from rhcs-dashboard/wip-55600-quincy
Ernesto Puerta [Fri, 13 May 2022 09:04:48 +0000 (11:04 +0200)]
Merge pull request #46220 from rhcs-dashboard/wip-55600-quincy

quincy: mgr/dashboard: introduce memory and cpu usage for daemons

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: rkachach <NOT@FOUND>
3 years agoMerge pull request #46229 from rhcs-dashboard/wip-55416-quincy
Ernesto Puerta [Thu, 12 May 2022 19:59:28 +0000 (21:59 +0200)]
Merge pull request #46229 from rhcs-dashboard/wip-55416-quincy

quincy: mgr/dashboard: fix wrong pg status processing

Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agomgr/dashboard: fix wrong pg status processing 46229/head
Ernesto Puerta [Fri, 11 Mar 2022 16:29:07 +0000 (17:29 +0100)]
mgr/dashboard: fix wrong pg status processing

Fixes: https://tracker.ceph.com/issues/54481
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 206dd9d4a71a70c46972597a838fda05ceec03da)

3 years agomgr/dashboard: fix linting errors and add test 46220/head
Aashish Sharma [Fri, 8 Apr 2022 05:19:04 +0000 (10:49 +0530)]
mgr/dashboard: fix linting errors and add test

Fixes: https://tracker.ceph.com/issues/55218
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit b496f5837803a915daac98b0fad56eac3174db85)

 Conflicts:
src/pybind/mgr/dashboard/frontend/cypress/integration/cluster/services.po.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-daemon-list/service-daemon-list.component.html
     - Neglecting e2e changes as they aren't adopted fully to master yet- no regression. Adopting daemon-list html with master.

3 years agomgr/dashboard: introduce memory and cpu usage for daemons
Avan Thakkar [Thu, 7 Apr 2022 11:01:20 +0000 (16:31 +0530)]
mgr/dashboard: introduce memory and cpu usage for daemons

Fixes: https://tracker.ceph.com/issues/55218
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Co-authored-by: Aashish Sharma <aasharma@redhat.com>
Introducing 2 new columns in Cluster->Host->Daemons table for Memory and CPU usage.

(cherry picked from commit 263940502bdd9858c97923f394cd3d918e86e921)

3 years agoMerge pull request #46156 from nmshelke/wip-55413-quincy
Yuri Weinstein [Tue, 10 May 2022 14:47:02 +0000 (07:47 -0700)]
Merge pull request #46156 from nmshelke/wip-55413-quincy

quincy: mds: add a perf counter to record slow replies

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46147 from neesingh-rh/wip-55540-quincy
Yuri Weinstein [Tue, 10 May 2022 14:45:21 +0000 (07:45 -0700)]
Merge pull request #46147 from neesingh-rh/wip-55540-quincy

quincy: cephfs-top: Multiple filesystem support

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
3 years agoMerge pull request #45994 from nmshelke/wip-55376-quincy
Yuri Weinstein [Tue, 10 May 2022 14:43:42 +0000 (07:43 -0700)]
Merge pull request #45994 from nmshelke/wip-55376-quincy

quincy: mgr/volumes: set, get, list and remove metadata of subvolume

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #45939 from nmshelke/wip-55039-quincy
Yuri Weinstein [Tue, 10 May 2022 14:43:09 +0000 (07:43 -0700)]
Merge pull request #45939 from nmshelke/wip-55039-quincy

quincy: ceph-fuse: ignore fuse mount failure if path is already mounted

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agorpm: use system libpmem on Centos 9 Stream 46212/head
Ilya Dryomov [Thu, 10 Mar 2022 17:32:30 +0000 (18:32 +0100)]
rpm: use system libpmem on Centos 9 Stream

We need libpmem 1.10 and Centos 9 Stream has it.  On top of sticking
to distro-provided packages being generally a good thing, this fixes
a build failure: libpmem 1.10 doesn't build with LTO which is enabled
by default in Centos 9 Stream.  The distro package works around it.

libpmem 1.10 is also there in Fedora 34 and Fedora 35.

Fixes: https://tracker.ceph.com/issues/54473
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 09440364731b35d5b47922a196df173d306e11dc)

3 years agoMerge pull request #46144 from cbodley/wip-55545
Yuri Weinstein [Mon, 9 May 2022 21:59:12 +0000 (14:59 -0700)]
Merge pull request #46144 from cbodley/wip-55545

quincy: cmake/rgw: add missing dependency on Arrow::Arrow

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agoMerge pull request #46041 from cbodley/wip-55458
Yuri Weinstein [Mon, 9 May 2022 20:34:11 +0000 (13:34 -0700)]
Merge pull request #46041 from cbodley/wip-55458

quincy: rgw: RGWCoroutine::set_sleeping() checks for null stack

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #46038 from cbodley/wip-55455
Yuri Weinstein [Mon, 9 May 2022 20:33:21 +0000 (13:33 -0700)]
Merge pull request #46038 from cbodley/wip-55455

quincy: rgw: OpsLogFile::stop() signals under mutex

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45679 from rhcs-dashboard/wip-54587-quincy
Ernesto Puerta [Fri, 6 May 2022 08:23:08 +0000 (10:23 +0200)]
Merge pull request #45679 from rhcs-dashboard/wip-54587-quincy

quincy: mgr/dashboard: highlight the search text in cluster logs

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #45957 from votdev/wip-55379-quincy
Ernesto Puerta [Fri, 6 May 2022 08:16:12 +0000 (10:16 +0200)]
Merge pull request #45957 from votdev/wip-55379-quincy

quincy: mgr/dashboard: Imrove error message of '/api/grafana/validation' API endpoint

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #45862 from rhcs-dashboard/wip-55274-quincy
Ernesto Puerta [Fri, 6 May 2022 08:15:26 +0000 (10:15 +0200)]
Merge pull request #45862 from rhcs-dashboard/wip-55274-quincy

quincy: mgr/dashboard: datatable in Cluster Host page hides wrong column on selection

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #45867 from votdev/wip-55275-quincy
Ernesto Puerta [Fri, 6 May 2022 08:14:29 +0000 (10:14 +0200)]
Merge pull request #45867 from votdev/wip-55275-quincy

quincy: mgr/dashboard: RGW users and buckets tables are empty if the selected gateway is down

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
3 years agoMerge pull request #45619 from votdev/wip-55007-quincy
Ernesto Puerta [Fri, 6 May 2022 08:13:58 +0000 (10:13 +0200)]
Merge pull request #45619 from votdev/wip-55007-quincy

quincy: mgr/dashboard: Language dropdown box is partly hidden on login page

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #46055 from adk3798/quincy-backport-april
Adam King [Thu, 5 May 2022 19:04:27 +0000 (15:04 -0400)]
Merge pull request #46055 from adk3798/quincy-backport-april

quincy: Cephadm Batch Backport April

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #46061 from ljflores/wip-55439-quincy
Yuri Weinstein [Thu, 5 May 2022 15:22:17 +0000 (08:22 -0700)]
Merge pull request #46061 from ljflores/wip-55439-quincy

quincy: osd: return appropriate error if the object is not manifest

Reviewed-by: Myoungwon Oh <omwmw@sk.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45624 from ljflores/wip-55047-quincy
Yuri Weinstein [Thu, 5 May 2022 15:20:13 +0000 (08:20 -0700)]
Merge pull request #45624 from ljflores/wip-55047-quincy

quincy: test: fix TierFlushDuringFlush to wait until dedup_tier is set on base pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <omwmw@sk.com>
3 years agomds: add a perf counter to record slow replies 46156/head
haoyixing [Fri, 25 Mar 2022 03:02:13 +0000 (03:02 +0000)]
mds: add a perf counter to record slow replies

Though we have MDS_HEALTH_SLOW_METADATA_IO and MDS_HEALTH_SLOW_REQUEST health alert, but those are not
precise nor accumulated. With slow reply counter compared to reply counter, we can find out the ratio
of slow requests through perf dump.

Fixes: https://tracker.ceph.com/issues/55126
Signed-off-by: haoyixing <haoyixing@kuaishou.com>
(cherry picked from commit e8e3b307c87dc9eec2d087b396c0e7a0248b4f1d)

3 years agoMerge pull request #46116 from zdover23/wip-doc-pr-46109-backport-to-quincy
Laura Flores [Wed, 4 May 2022 16:06:58 +0000 (11:06 -0500)]
Merge pull request #46116 from zdover23/wip-doc-pr-46109-backport-to-quincy

Wip doc pr 46109 backport to quincy

3 years agoqa: test for verification of client metrics and metadata. 46147/head
Neeraj Pratap Singh [Mon, 21 Mar 2022 15:44:57 +0000 (21:14 +0530)]
qa: test for verification of client metrics and metadata.

Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 9855e15de2af20be49079271af23f7bfe24d56c4)

3 years agomgr/stats: fetches client_metadata for multiple filesystems.
Neeraj Pratap Singh [Thu, 17 Mar 2022 04:33:58 +0000 (10:03 +0530)]
mgr/stats: fetches client_metadata for multiple filesystems.

Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 5a1c4875c1b95248c050fb1f6aff48af4553156d)

3 years agocmake/rgw: add missing dependency on Arrow::Arrow 46144/head
Casey Bodley [Thu, 28 Apr 2022 15:43:15 +0000 (11:43 -0400)]
cmake/rgw: add missing dependency on Arrow::Arrow

when WITH_SYSTEM_ARROW is disabled, BuildArrow.cmake creates an
Arrow::Parquet target that depends on Arrow::Arrow:

  target_link_libraries(Arrow::Parquet INTERFACE Arrow::Arrow)

but when WITH_SYSTEM_ARROW is enabled, the targets we get from
find_package() do not carry this dependency. so rgw's cmake needs to
depend on both targets

Fixes: https://tracker.ceph.com/issues/55420
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3f12d21418963f2a33b124185a2842a9c39e9cfa)

3 years agoMerge pull request #45700 from tserong/wip-55100-quincy
Deepika Upadhyay [Wed, 4 May 2022 12:23:26 +0000 (17:53 +0530)]
Merge pull request #45700 from tserong/wip-55100-quincy

quincy: ceph.spec.in: Use libthrift-devel on SUSE distros

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #46075 from bluikko/patch-5
Anthony D'Atri [Wed, 4 May 2022 06:41:25 +0000 (23:41 -0700)]
Merge pull request #46075 from bluikko/patch-5

doc/mgr/dashboard: Fix typo and double slash missing from URL

3 years agomgr/cephadm: retry mgr fail over in case of transient failure 46055/head
Adam King [Mon, 11 Apr 2022 20:57:51 +0000 (16:57 -0400)]
mgr/cephadm: retry mgr fail over in case of transient failure

Fixes: https://tracker.ceph.com/issues/55279
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3fe2d7d553d475f1fe3840c98ee31d71f6188a1a)

3 years agoApply sysctl.d migration from /usr/lib to /etc
windgmbh [Fri, 12 Nov 2021 15:51:03 +0000 (16:51 +0100)]
Apply sysctl.d migration from /usr/lib to /etc
A fix regarding the SYSCTL_DIR location (#53130) requires to migrate
sysctl.d/*.conf files from /usr/lib to /etc.
Signed-off-by: Lukas Mayer <lmayer@wind.gmbh>
(cherry picked from commit a167a27f30536958e0f2c513d351642e81ba06d5)

3 years agoFix sysctl.d location FHS compliance
windgmbh [Wed, 3 Nov 2021 17:16:53 +0000 (18:16 +0100)]
Fix sysctl.d location FHS compliance
This fixes #53130
Containers should not write to '/usr/lib'.
That location could be read-only or overwritten.
Signed-off-by: Lukas Mayer <lmayer@wind.gmbh>
(cherry picked from commit 77afa812ea8b7e1e802246e4aa3a31e7b644a502)

3 years agocephadm: add comment explaining docker.io grep test
Ken Dreyer [Wed, 16 Feb 2022 14:48:59 +0000 (09:48 -0500)]
cephadm: add comment explaining docker.io grep test

Add a comment explain this docker.io grep count test, since it's
difficult to understand the purpose from simply reading the code.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 14c5538a311542e1466a9426c35108ce3e394338)

3 years agocephadm: pass "--security-opt label=disable" to node-exporter container
Adam King [Thu, 24 Mar 2022 13:59:10 +0000 (09:59 -0400)]
cephadm: pass "--security-opt label=disable" to node-exporter container

in order to support setting '--path.procfs=/host/proc','--path.sysfs=/host/sys',
'--path.rootfs=/rootfs' for node-exporter we need to disable selinux separation
between the node-exporter container and the host to avoid selinux denials

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 6d4591723ba89dada9814118e2c14e08d4e4179a)

3 years agocephadm: Specify proc/sys path for node-exporter to use
Adam King [Wed, 23 Mar 2022 17:22:51 +0000 (13:22 -0400)]
cephadm: Specify proc/sys path for node-exporter to use

Fixes: https://tracker.ceph.com/issues/55023
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 97373de71e080423a2321e2c889e6681b47bfc74)

3 years agomgr/cephadm: offline host watcher
Adam King [Fri, 4 Mar 2022 02:47:47 +0000 (21:47 -0500)]
mgr/cephadm: offline host watcher

To be able to detect if certain offline hosts go
offline quicker. Could be useful for the NFS
HA feature as this requires moving nfs daemons from
offline hosts within 90 seconds.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit bd9eb596570cfcc7fea793c2b380bc66dd719439)

3 years agomgr/cephadm: fallback to normal sorted if cannot import natsorted
Redouane Kachach [Tue, 29 Mar 2022 16:37:10 +0000 (18:37 +0200)]
mgr/cephadm: fallback to normal sorted if cannot import natsorted
Fixes: https://tracker.ceph.com/issues/55113
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 19c07de8207de5038df6f510a3c2ff41b10f7e08)

3 years agomgr/cephadm: Reschedule nfs daemons from offline hosts
Adam King [Tue, 22 Mar 2022 22:57:21 +0000 (18:57 -0400)]
mgr/cephadm: Reschedule nfs daemons from offline hosts

In order to improve nfs availability, if there are other
hosts we can place an nfs daemon on or if there is a host
with a lower rank nfs daemon when a higher rank one is on
an offline host, we should reschedule the nfs daemons

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 9febc21c14b7ad26e6d811444e7daf0b0a292afb)

3 years agomgr/cephadm: fixing natsort dependencies on debian
Redouane Kachach [Thu, 17 Mar 2022 08:49:26 +0000 (09:49 +0100)]
mgr/cephadm: fixing natsort dependencies on debian

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit dadbd7299640b50949c4f5f37509831ee4824937)

Conflicts:
debian/control

3 years agomgr/cephadm: Adding support for natural sorting
Redouane Kachach [Fri, 4 Feb 2022 11:22:50 +0000 (12:22 +0100)]
mgr/cephadm: Adding support for natural sorting
Fixes: https://tracker.ceph.com/issues/54026
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 28314cebef328a1e822dbc4c348995433af25fc2)

3 years agomgr/cephadm: fix cephadm errors
Aashish Sharma [Wed, 2 Mar 2022 11:23:02 +0000 (16:53 +0530)]
mgr/cephadm: fix cephadm errors

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 46902381306fd31a054f58c49af7f817833f8cc4)

3 years agomgr/dashboard: fixing cephadm errors
Aashish Sharma [Fri, 11 Feb 2022 09:10:17 +0000 (14:40 +0530)]
mgr/dashboard: fixing cephadm errors

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit ba6efa121ab0e7b6949cf41219b68c38d1a77641)

3 years agomgr/cephadm: adding loki and promtail monitoring services testing
Avan Thakkar [Thu, 10 Feb 2022 10:29:03 +0000 (15:59 +0530)]
mgr/cephadm: adding loki and promtail monitoring services testing

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit cf2734a74cde261d7c44830c8668622901ed18cc)

3 years agomgr/cephadm: fix loki grpc port and liniting errors
Avan Thakkar [Thu, 10 Feb 2022 09:23:15 +0000 (14:53 +0530)]
mgr/cephadm: fix loki grpc port and liniting errors

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit a25738e49030ae77a105ea244843a7ab0b83aee6)

3 years agomgr/dashboard: add grps port to loki
Aashish Sharma [Thu, 3 Feb 2022 08:43:56 +0000 (14:13 +0530)]
mgr/dashboard: add grps port to loki

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Co-authored-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit f0fa000edc8af2896f66a5a95ec3e5f06cedf8e4)

3 years agomgr/dashboard: add Loki to grafana datasources
Avan Thakkar [Mon, 31 Jan 2022 12:05:31 +0000 (17:35 +0530)]
mgr/dashboard: add Loki to grafana datasources

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Co-authored-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 66b2f03e8a028dd692ec95a0f97f3183a4bc8b56)

3 years ago mgr/cephadm: introducing orch apply daemon promtail
Avan Thakkar [Tue, 25 Jan 2022 18:56:20 +0000 (00:26 +0530)]
 mgr/cephadm: introducing orch apply daemon promtail

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Co-authored-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit b576d95f8cf288cb9ca242375a99b6bc460ecb8b)

Conflicts:
src/cephadm/cephadm

3 years agomgr/cephadm: introducing orch apply daemon loki
Avan Thakkar [Mon, 24 Jan 2022 22:37:45 +0000 (04:07 +0530)]
mgr/cephadm: introducing orch apply daemon loki

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 5dc96cb62165109338e6e13a259e5e6dc2b5a8ce)

Conflicts:
src/cephadm/cephadm
src/pybind/mgr/cephadm/module.py

3 years agomgr/cephadm: introduce loki and promtail containers
Avan Thakkar [Mon, 24 Jan 2022 09:38:02 +0000 (15:08 +0530)]
mgr/cephadm: introduce loki and promtail containers

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit ddccfdb5aa108631a6ce0bc9435a4ded1c12af7f)

Conflicts:
src/cephadm/cephadm

3 years agomgr/cephadm: fixing call method timeout handling
Redouane Kachach [Mon, 11 Apr 2022 14:13:05 +0000 (16:13 +0200)]
mgr/cephadm: fixing call method timeout handling
Fixes: https://tracker.ceph.com/issues/55265
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 55d42d0c31a5414cb2beac9273faa7123efb8709)

3 years agomgr/cephadm: Adding support to store ceph conf per cluster fsid
Redouane Kachach [Tue, 12 Apr 2022 12:27:02 +0000 (14:27 +0200)]
mgr/cephadm: Adding support to store ceph conf per cluster fsid
Fixes: https://tracker.ceph.com/issues/55185
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 2ea76173a163a93bbfbf69d0faa732d46eaf05ba)

3 years agoqa/suites/orch/cephadm: stop upgrade tests if failures are seen
Adam King [Sun, 17 Apr 2022 16:21:44 +0000 (12:21 -0400)]
qa/suites/orch/cephadm: stop upgrade tests if failures are seen

Otherwise the tests may run forever. This was already done for
mds upgrade sequence, justadding it in the other two places here

Related to: https://tracker.ceph.com/issues/53939

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 017aa9cfe8362e8512a581e39850ce70bd1ce82f)

3 years agomgr/cephadm: fixing logging output for rm-cluster operation
Redouane Kachach [Mon, 18 Apr 2022 13:42:39 +0000 (15:42 +0200)]
mgr/cephadm: fixing logging output for rm-cluster operation
Fixes: https://tracker.ceph.com/issues/51671
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 9756e3bf8ce6976d12245e1e6724838b03d70135)

3 years agomgr/cephadm: do not add _admin label when no-minimize-config is provided
Redouane Kachach [Thu, 21 Apr 2022 10:01:44 +0000 (12:01 +0200)]
mgr/cephadm: do not add _admin label when no-minimize-config is provided
Fixes: https://tracker.ceph.com/issues/52727
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 01c8999d0354a71a7ef8526aab9b39e30d67c1bb)

3 years agomgr/cephadm: fixing upgrade candidate verions listing
Redouane Kachach [Wed, 13 Apr 2022 14:22:53 +0000 (16:22 +0200)]
mgr/cephadm: fixing upgrade candidate verions listing
Fixes: https://tracker.ceph.com/issues/53527
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 671442c4b445c4a0547978a6298645a5c2f5725f)

3 years agomgr/cephadm: Using '.ID' instead of '.Id' which is only supported by podman
Redouane Kachach [Fri, 22 Apr 2022 13:53:34 +0000 (15:53 +0200)]
mgr/cephadm: Using '.ID' instead of '.Id' which is only supported by podman

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 6e7c0488389e7e8b85f0dbd6b71c8627a44b349e)

3 years agomgr/cephadm: Adding image tag and date to cephadm startup messages
Redouane Kachach [Wed, 23 Mar 2022 17:24:01 +0000 (18:24 +0100)]
mgr/cephadm: Adding image tag and date to cephadm startup messages
Fixes: https://tracker.ceph.com/issues/55008
Fixes: https://tracker.ceph.com/issues/54373
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 92ecb58d46b6f75265a664f3165f4b3a0dd4993a)

3 years agomgr/cephadm: allow setting insecure_skip_verify for alertmanager
Adam King [Wed, 6 Apr 2022 14:32:22 +0000 (10:32 -0400)]
mgr/cephadm: allow setting insecure_skip_verify for alertmanager

Add a "secure" parameter to alertmanager spec that will cause it
to deploy alertmanagers with insecure_skip_verify as true or false
depending on the value given for "secure".

NOTE: alertmanager must still be reconfigured after applying a yaml
with this option changed.

Fixes: https://tracker.ceph.com/issues/55272
Fixes: https://tracker.ceph.com/issues/55333
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e583d4ef1ac23a7473d50d253e0edf70580542ae)

3 years agocephadm: avoid crashing on expected non-zero exit
Moritz Röhrich [Mon, 21 Mar 2022 16:32:25 +0000 (17:32 +0100)]
cephadm: avoid crashing on expected non-zero exit

- Avoid crashing when a call out to an external program expectedly does
  not return exit status zero.

There are programs that communicate other information than error/no
error through exit status. E.g. `systemctl status` will return different
exit codes depending on the actual status of the units in question.
In cases where this is expected crashing with a RuntimeError exception
is inappropriate and should be avoided.

Fixes: https://tracker.ceph.com/issues/55117
Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
(cherry picked from commit a02be6f22fa18094cd8758700ab74581b6ce1701)

3 years agomgr/cephadm: skip loopback devices when gathering facts
Redouane Kachach [Mon, 11 Apr 2022 11:04:13 +0000 (13:04 +0200)]
mgr/cephadm: skip loopback devices when gathering facts
Fixes: https://tracker.ceph.com/issues/53528
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit a9eec9e3a7a53bd63d3bf9ced96e224fb752d9f7)

3 years agoceph cephadm set-user does not reflect the user change in ssh-config
Teoman ONAY [Wed, 6 Apr 2022 09:32:17 +0000 (11:32 +0200)]
ceph cephadm set-user does not reflect the user change in ssh-config

Fixes: https://tracker.ceph.com/issues/54618
Signed-off-by: Teoman ONAY <tonay@redhat.com>
(cherry picked from commit 071f72a734ce207e5cb2ff6d3d996e45396f5c7a)

3 years agomgr/cephadm: Adding cephadm networking configuration checks+refactoring
Redouane Kachach [Fri, 1 Apr 2022 16:03:42 +0000 (18:03 +0200)]
mgr/cephadm: Adding cephadm networking configuration checks+refactoring
Fixes: https://tracker.ceph.com/issues/55174
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit e0bafe6b1da104782b29edf7035d7bc93f89e12f)

3 years agomgr/cephadm: Making default cephadm shell cmd easier
Redouane Kachach [Thu, 17 Feb 2022 12:48:08 +0000 (13:48 +0100)]
mgr/cephadm: Making default cephadm shell cmd easier
Fixes: https://tracker.ceph.com/issues/52042
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit dc201197639dcab471611ac3c4fefda74a74a94f)

3 years agocephadm: show error message if private registry credentials not provided
Melissa Li [Wed, 23 Mar 2022 15:38:37 +0000 (11:38 -0400)]
cephadm: show error message if private registry credentials not provided

Raise UnauthorizedRegistryError in `_pull_image` if user tries to pull from a private registry without authentication, handle error in `command_bootstrap`, `commond_adopt`, `command_pull`

Fixes: https://tracker.ceph.com/issues/55015
Signed-off-by: Melissa Li <melissali@redhat.com>
(cherry picked from commit 4de0803ba893abf341ab634d1382208370de7c98)

3 years agomgr/cephadm: improving logging to send errors to stderr
Redouane Kachach [Tue, 29 Mar 2022 11:30:37 +0000 (13:30 +0200)]
mgr/cephadm: improving logging to send errors to stderr
Fixes: https://tracker.ceph.com/issues/47905
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 7f09307a614b313908a545a1d26e28e3e704e321)

3 years agomgr/cephadm: support non-root ssh-user w permissions
Christoph Glaubitz [Thu, 31 Mar 2022 12:57:53 +0000 (12:57 +0000)]
mgr/cephadm: support non-root ssh-user w permissions

Restructured code, so that in case of non-root, the resulting file will
be created with permissions set to the ssh-user. This allows the
subsequent scp to be able to write the file. The remaining code kept the
same, so that file permissions are restored to the expected ones, but
just runs after the scp.

Fixes: https://tracker.ceph.com/issues/54620
Signed-off-by: Christoph Glaubitz <c.glaubitz@syseleven.de>
(cherry picked from commit 452e52a7e39409e3409d59940133333416b830bc)

3 years agomgr/cephadm: fixing public network conf parsing
Redouane Kachach [Wed, 30 Mar 2022 13:48:40 +0000 (15:48 +0200)]
mgr/cephadm: fixing public network conf parsing
Fixes: https://tracker.ceph.com/issues/55132
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 3ef6341e8ef5fe6a01f15c847f6bc9e2205d4d97)

3 years agoceph-volume/tests: reject loop devices in lvm.conf
Guillaume Abrioux [Wed, 30 Mar 2022 14:18:26 +0000 (16:18 +0200)]
ceph-volume/tests: reject loop devices in lvm.conf

The current task doesn't works (typo?).
Otherwise api/lvm.py can't work properly, functions such as
`get_single_lv()` and many other don't return the expected results.
Indeed, lvm is confused because of the nvme_loop setup.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit a5fab15e44517ac63f3fd257989e81b8127b86d9)

3 years agoceph-volume: do not leave pv when zapping osds
Guillaume Abrioux [Mon, 28 Mar 2022 22:01:39 +0000 (00:01 +0200)]
ceph-volume: do not leave pv when zapping osds

when zapping a device and no vg/lv are left, the pv should be
removed too.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7f007e7fc75b4d6e7465c684f7e5b2458883dcc5)

3 years agoorchestrator: support complex osd creation
Guillaume Abrioux [Wed, 23 Mar 2022 09:04:45 +0000 (10:04 +0100)]
orchestrator: support complex osd creation

This adds the support of complex OSD creation with command
`orch daemon add osd`.
Any argument supported by `DriveGroupSpec()` can be passed on the command line.

Usage:
```
ceph orch daemon add osd host:data_devices=device1,device2,db_devices=device3,osds_per_device=2,...
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8aa2f4745adff0ba3c7a0731cf48ccc1c85b33f3)

3 years agoDriveSelection: skip unavailable devices
Guillaume Abrioux [Tue, 22 Mar 2022 15:35:58 +0000 (16:35 +0100)]
DriveSelection: skip unavailable devices

Cephadm shouldn't try to deploy a disk reported as unavailable by ceph-volume.
The idea here is to check the rejection reason so we can still use DB devices
in case of OSD replacement.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3a88547559769f4dd438f6557cef22ef9004fa2a)