]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agodashboard: Convert FQDN to hostname in grafana panels 34495/head
Kristoffer Grönlund [Fri, 27 Mar 2020 10:06:58 +0000 (11:06 +0100)]
dashboard: Convert FQDN to hostname in grafana panels

The $ceph_hosts variable contained the FQDN for hosts
while the instance label created by ceph only has
the hostname.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit b7abaab5bd2b72f48f41b3d2a9732743afdcecfc)

5 years agodashboard: Resolve FQDN / hostname mismatch in hosts overview panel
Kristoffer Grönlund [Thu, 26 Mar 2020 19:38:04 +0000 (20:38 +0100)]
dashboard: Resolve FQDN / hostname mismatch in hosts overview panel

In the AVG Disk Utilization panel, the result is calculated
by combining the output of node_disk_io_time_seconds_total
with the output of ceph_disk_occupation. However, the
first vector encodes the instance label with the full FQDN
while the ceph label only contains the hostname:port. In
order for these to match correctly, the domain name and port
has to be stripped from the labels.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 136d21e21dc3c05ec8c586a47eed1904ffbda578)

5 years agodashboard: Use exported_instance to identify OSDs
Kristoffer Grönlund [Thu, 26 Mar 2020 18:12:55 +0000 (19:12 +0100)]
dashboard: Use exported_instance to identify OSDs

When moving to LVM-based ceph-volume setups, several
grafana dashboards stopped working. The problem is that
(device, instance) no longer results in unique labels
which causes errors like:

"many-to-many matching not allowed: matching labels must be unique on one side"

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 8b61b8d3d781b9d762098cf45b41b596a32509ab)

5 years agodashboard: AVG RAM Utilization panel always showed "N/A"
Kristoffer Grönlund [Tue, 24 Mar 2020 16:57:15 +0000 (17:57 +0100)]
dashboard: AVG RAM Utilization panel always showed "N/A"

The references to `$osd_hosts` etc. were encoded as
`[[osd_hosts]]` in the PromQL expression divisor, and
the panel always displayed N/A as the result of the
query.

Replacing the `[[...]]` with `$...` makes the expression
work again.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 4444333243aaa81de1d7347bf0c57e039b657d43)

5 years agoMerge pull request #34482 from ceph/octopus-fixes
Abhishek L [Wed, 8 Apr 2020 16:46:45 +0000 (18:46 +0200)]
Merge pull request #34482 from ceph/octopus-fixes

Octopus fixes

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Radoslaw Zarynski <rzarynski@redhat.com>
Reviewed-By: Josh Durgin <jdurgin@redhat.com>
5 years agorgw: reject control characters in response-header actions 34482/head
Robin H. Johnson [Fri, 27 Mar 2020 19:48:13 +0000 (20:48 +0100)]
rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
5 years agorgw: EPERM to ERR_INVALID_REQUEST
Abhishek Lekshmanan [Fri, 27 Mar 2020 18:29:01 +0000 (19:29 +0100)]
rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: reject unauthenticated response-header actions
Matt Benjamin [Fri, 27 Mar 2020 17:13:48 +0000 (18:13 +0100)]
rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

5 years agomsg/async/crypto_onwire: fix endianness of nonce_t
Ilya Dryomov [Fri, 6 Mar 2020 19:16:45 +0000 (20:16 +0100)]
msg/async/crypto_onwire: fix endianness of nonce_t

As a AES-GCM IV, nonce_t is implicitly shared between server and
client.  Currently, if their endianness doesn't match, they are unable
to communicate in secure mode because each gets its own idea of what
the next nonce should be after the counter is incremented.

Several RFCs state that the nonce counter should be BE, but since we
use LE for everything on-disk and on-wire, make it LE.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agomsg/async/ProtocolV2: avoid AES-GCM nonce reuse vulnerabilities
Ilya Dryomov [Fri, 6 Mar 2020 19:16:45 +0000 (20:16 +0100)]
msg/async/ProtocolV2: avoid AES-GCM nonce reuse vulnerabilities

The secure mode uses AES-128-GCM with 96-bit nonces consisting of a
32-bit counter followed by a 64-bit salt.  The counter is incremented
after processing each frame, the salt is fixed for the duration of
the session.  Both are initialized from the session key generated
during session negotiation, so the counter starts with essentially
a random value.  It is allowed to wrap, and, after 2**32 frames, it
repeats, resulting in nonce reuse (the actual sequence numbers that
the messenger works with are 64-bit, so the session continues on).

Because of how GCM works, this completely breaks both confidentiality
and integrity aspects of the secure mode.  A single nonce reuse reveals
the XOR of two plaintexts and almost completely reveals the subkey
used for producing authentication tags.  After a few nonces get used
twice, all confidentiality and integrity goes out the window and the
attacker can potentially encrypt-authenticate plaintext of their
choice.

We can't easily change the nonce format to extend the counter to
64 bits (and possibly XOR it with a longer salt).  Instead, just
remember the initial nonce and cut the session before it repeats,
forcing renegotiation.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Conflicts:
src/msg/async/ProtocolV2.h [ context: commit ed3ec4c01d17
  ("msg: Build target 'common' without using namespace in
  headers") not in octopus ]

5 years agoMerge pull request #34372 from rhcs-dashboard/wip-44697-octopus
Lenz Grimmer [Wed, 8 Apr 2020 08:47:08 +0000 (10:47 +0200)]
Merge pull request #34372 from rhcs-dashboard/wip-44697-octopus

octopus: mgr/dashboard: add popover list of Stand-by Managers & Metadata Servers (MDS) in landing page

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34358 from smithfarm/wip-44786-octopus
Lenz Grimmer [Wed, 8 Apr 2020 08:46:02 +0000 (10:46 +0200)]
Merge pull request #34358 from smithfarm/wip-44786-octopus

octopus: qa/tasks/mgr/dashboard/test_health: update mdsmap schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34402 from rhcs-dashboard/wip-44932-octopus
Lenz Grimmer [Wed, 8 Apr 2020 08:45:18 +0000 (10:45 +0200)]
Merge pull request #34402 from rhcs-dashboard/wip-44932-octopus

octopus: mgr/dashboard: fix error when enabling SSO with cert. file

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34403 from rhcs-dashboard/wip-44933-octopus
Lenz Grimmer [Tue, 7 Apr 2020 16:18:28 +0000 (18:18 +0200)]
Merge pull request #34403 from rhcs-dashboard/wip-44933-octopus

octopus: mgr/dashboard: fix notifications E2E tests

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #34397 from votdev/wip-44921-octopus
Laura Paduano [Tue, 7 Apr 2020 13:53:41 +0000 (15:53 +0200)]
Merge pull request #34397 from votdev/wip-44921-octopus

octopus: mgr/dashboard: Add more debug information to Dashboard RGW backend

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34428 from badone/wip-octopus-specify-rule-for-pool-creation
Brad Hubbard [Tue, 7 Apr 2020 07:03:23 +0000 (17:03 +1000)]
Merge pull request #34428 from badone/wip-octopus-specify-rule-for-pool-creation

octopus: specify rule for pool creation

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoqa/ceph-ansible: CA master requires Ansible 2.9 34428/head
Brad Hubbard [Mon, 30 Mar 2020 09:46:19 +0000 (19:46 +1000)]
qa/ceph-ansible: CA master requires Ansible 2.9

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit c9317ea2aeab9e038b1664af36dc3404daf9f3bd)

5 years agoqa/ceph-ansible: rule_name is required for pool creation
Brad Hubbard [Mon, 30 Mar 2020 06:21:14 +0000 (16:21 +1000)]
qa/ceph-ansible: rule_name is required for pool creation

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit a12353815dac3db5c8452fb2d7649a032d4832f4)

5 years agoMerge pull request #34401 from sebastian-philipp/octopus-backport-34290-34318-34319
Nathan Cutler [Mon, 6 Apr 2020 16:10:27 +0000 (18:10 +0200)]
Merge pull request #34401 from sebastian-philipp/octopus-backport-34290-34318-34319

octopus: cephadm: docs batch 1

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agomgr/dashboard: fix notifications E2E tests 34403/head
Alfonso Martínez [Mon, 16 Mar 2020 15:11:10 +0000 (16:11 +0100)]
mgr/dashboard: fix notifications E2E tests

Fixes: https://tracker.ceph.com/issues/44626
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 418b21ac6f2314edb3fda093db3ef694fa9c7ffe)

5 years agomgr/dashboard: fix error when enabling SSO with cert. file 34402/head
Alfonso Martínez [Wed, 18 Mar 2020 16:19:13 +0000 (17:19 +0100)]
mgr/dashboard: fix error when enabling SSO with cert. file

Also:
* Disabled security setting 'wantNameIdEncrypted': not all Identity Providers support this and we are already requiring encrypted assertions (which is the default).

Fixes: https://tracker.ceph.com/issues/44666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 8b0b9277ca3aa0dc5313ac635a06c0f80000b6ac)

5 years agodoc/mgr/orchestrator: update cephadm shell proposed alias entry 34401/head
clyso [Tue, 31 Mar 2020 12:50:07 +0000 (14:50 +0200)]
doc/mgr/orchestrator: update cephadm shell proposed alias entry

prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
(cherry picked from commit 4a850e1dca911ba17a9645c982742443be547021)

5 years agodoc/mgr/orchestrator: add path to ssh-copy-id instructions
clyso [Tue, 31 Mar 2020 12:48:05 +0000 (14:48 +0200)]
doc/mgr/orchestrator: add path to ssh-copy-id instructions

by default ssh key will be placed under /etc/ceph - so it should be included in examples

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
(cherry picked from commit 87440ab4d26b6cd1576eacdbbe47f28f90bacc1c)

5 years agodoc/mgr/orchestrator: Update `Placement by pattern matching`
Sebastian Wagner [Wed, 1 Apr 2020 09:33:51 +0000 (11:33 +0200)]
doc/mgr/orchestrator: Update `Placement by pattern matching`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b740ae12e54e69e21fa06081d47b0ed4f90fac41)

5 years agodoc/mgr/orchestrator: use yaml syntax highlighting
Sebastian Wagner [Tue, 31 Mar 2020 13:21:45 +0000 (15:21 +0200)]
doc/mgr/orchestrator: use yaml syntax highlighting

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit bc162005a92d92dc5e84bda5a7034057af2436b6)

5 years agodoc/cephadm/drivegroup: Don't introduce DriveGroups
Sebastian Wagner [Mon, 30 Mar 2020 14:48:16 +0000 (16:48 +0200)]
doc/cephadm/drivegroup: Don't introduce DriveGroups

We already have a name for it: a Service Specification of
type `osd`. We don't need to introduce a new name for it.

Well, they are "DriveGroups", but users don't need to know it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8d4487527e17824cd1ad18ad52d8f3449899cf21)

5 years agodoc/cephadm/drivegroups: Update to new yaml
Sebastian Wagner [Mon, 30 Mar 2020 14:43:44 +0000 (16:43 +0200)]
doc/cephadm/drivegroups: Update to new yaml

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f5e3e60e0eef840a709e2d118c421d409b58b957)

5 years agodoc/mgr/orchestrator: Add "Service Specification"
Sebastian Wagner [Mon, 30 Mar 2020 14:36:46 +0000 (16:36 +0200)]
doc/mgr/orchestrator: Add "Service Specification"

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0c6759750a95083081cd2b701d2d60b3966428c8)

5 years agomgr/dashboard: Add more debug information to Dashboard RGW backend 34397/head
Volker Theile [Thu, 2 Apr 2020 14:01:53 +0000 (16:01 +0200)]
mgr/dashboard: Add more debug information to Dashboard RGW backend

Fixes: https://tracker.ceph.com/issues/44914
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 4febdd6ddaddebd2a8d07f44c8d0f79524305766)

5 years agoMerge pull request #34355 from smithfarm/wip-44710-octopus
Sebastian Wagner [Fri, 3 Apr 2020 10:51:29 +0000 (12:51 +0200)]
Merge pull request #34355 from smithfarm/wip-44710-octopus

octopus: doc/cephadm: replace `osd create` with `apply osd`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/dashboard: add popover list of managers in landing page 34372/head
Avan Thakkar [Tue, 25 Feb 2020 08:49:10 +0000 (14:19 +0530)]
mgr/dashboard: add popover list of managers in landing page
Fixes: https://tracker.ceph.com/issues/42979
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit cdfeb1d196c7d47340baae2be5910b90c889e778)

5 years agoqa/tasks/mgr/dashboard/test_health: ignore fsmap.standbys 34358/head
Kiefer Chang [Wed, 25 Mar 2020 03:17:17 +0000 (11:17 +0800)]
qa/tasks/mgr/dashboard/test_health: ignore fsmap.standbys

Fixes: https://tracker.ceph.com/issues/44725
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit afce413014086ab7cab5de133dae91e30635198a)

5 years agoqa/tasks/mgr/dashboard/test_health: update mdsmap schema
Kiefer Chang [Tue, 24 Mar 2020 10:14:05 +0000 (18:14 +0800)]
qa/tasks/mgr/dashboard/test_health: update mdsmap schema

Fixes: https://tracker.ceph.com/issues/44725
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 933df97cab1c1042dc350d8c08d26cb01ea240cf)

5 years agodoc/cephadm: replace `osd create` with `apply osd` 34355/head
Sebastian Wagner [Fri, 20 Mar 2020 11:33:27 +0000 (12:33 +0100)]
doc/cephadm: replace `osd create` with `apply osd`

Fixes: https://tracker.ceph.com/issues/44692
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit cb12a3177cb51ea37d3930a40f58c681ddefd2ac)

5 years agoMerge PR #34225 into octopus
Sage Weil [Wed, 1 Apr 2020 19:26:47 +0000 (14:26 -0500)]
Merge PR #34225 into octopus

* refs/pull/34225/head:
doc/start: Update hardware recommendations.

Reviewed-by: David Galloway <dgallowa@redhat.com>
5 years agoMerge PR #34280 into octopus
Sage Weil [Wed, 1 Apr 2020 18:56:01 +0000 (13:56 -0500)]
Merge PR #34280 into octopus

* refs/pull/34280/head:
mgr/dashboard: do not fail on user creation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge PR #34313 into octopus
Sage Weil [Wed, 1 Apr 2020 18:54:27 +0000 (13:54 -0500)]
Merge PR #34313 into octopus

* refs/pull/34313/head:
qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #34339 into octopus
Sage Weil [Wed, 1 Apr 2020 18:53:48 +0000 (13:53 -0500)]
Merge PR #34339 into octopus

* refs/pull/34339/head:
qa/tasks/mgr/dashboard/test_rbd: wait longer when purging

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoqa/tasks/mgr/dashboard/test_rbd: wait longer when purging 34339/head
Kiefer Chang [Fri, 27 Mar 2020 05:14:09 +0000 (13:14 +0800)]
qa/tasks/mgr/dashboard/test_rbd: wait longer when purging

Fixes: https://tracker.ceph.com/issues/44743
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
Co-authored-by: Kefu Chai <kefu@redhat.com>
Co-authored-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 1c3d6539d93961f9f7bd5308771725410bde32ec)

5 years agoMerge pull request #34323 from dillaman/wip-44861-octopus
Yuri Weinstein [Wed, 1 Apr 2020 00:45:50 +0000 (00:45 +0000)]
Merge pull request #34323 from dillaman/wip-44861-octopus

octopus: librbd: fix client backwards compatibility issues

5 years agolibrbd: re-add support for nautilus clients talking to jewel clusters 34323/head
Jason Dillaman [Thu, 2 May 2019 20:55:44 +0000 (16:55 -0400)]
librbd: re-add support for nautilus clients talking to jewel clusters

We want to support N - 3 client backward compatibility (special case
to support Jewel since it was a LTS release). The "get_snapshot_timestamp"
cls method does not exist in Jewel clusters so librbd should fallback
to excluding the op if it fails.

Note that this N - 3 also needs to apply for downstream releases as well,
which implies we still need Jewel for the time being.

Fixes: http://tracker.ceph.com/issues/39450
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c644121820b83c97e68f9896393a45cd34787672)

Conflicts:
src/test/librbd/image/test_mock_RefreshRequest.cc: tweaked to support pool configs
(cherry picked from commit d1a67c3bda09430bb21f768fb743b3e41c7d411e)

5 years agolibrbd: ignore -EOPNOTSUPP errors when retrieving image group membership
Jason Dillaman [Wed, 20 Mar 2019 18:40:50 +0000 (14:40 -0400)]
librbd: ignore -EOPNOTSUPP errors when retrieving image group membership

The Luminous release did not support adding images to a group (it only
included the bare-minimum support for creating groups). Commit f76df32666b
incorrectly dropped support for ignoring this possible failure. This
prevents Nautilus-release clients from opening images contained within
a Luminous-release cluster.

Fixes: http://tracker.ceph.com/issues/38834
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6f29dc69a0db3b6d982c95ab9d3b9b00a7029e37)
(cherry picked from commit ab720d4c0026d596dcea8782fbdd1640c93ec384)

5 years agoMerge pull request #34202 from tspmelo/wip-44741-octopus
Lenz Grimmer [Tue, 31 Mar 2020 13:59:37 +0000 (15:59 +0200)]
Merge pull request #34202 from tspmelo/wip-44741-octopus

octopus: mgr/dashboard: Check for missing npm resolutions

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoqa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length 34313/head
Neha [Sun, 22 Mar 2020 20:01:23 +0000 (20:01 +0000)]
qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length

It is possible for the pg dump to not be the latest when we check for newprimary
in _common_test(). This is because mgr_stats_period is 5 seconds, and we may not
have fetched the latest stats just yet. This causes the test to look at the same
stats before and after wait_for_clean.

Fixes: https://tracker.ceph.com/issues/43807 (2)
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit cfebec1b128bc2191d2bbe90f99191ae7750b203)

5 years agoMerge PR #34132 into octopus
Sage Weil [Mon, 30 Mar 2020 13:34:39 +0000 (08:34 -0500)]
Merge PR #34132 into octopus

* refs/pull/34132/head:
vstart_runner.py: fix OSError when checking if non-existent path is mounted

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #34255 into octopus
Sage Weil [Mon, 30 Mar 2020 13:33:32 +0000 (08:33 -0500)]
Merge PR #34255 into octopus

* refs/pull/34255/head:
qa/tasks/ceph.py: fail test if osd devices not found
qa/tasks/ceph.py: use .format to log dicts
qa/tasks/ceph.py: drop roles_to_journals and remote_to_roles_to_journals
qa/tasks/ceph.py: drop block_journal, tmpfs_journal
qa/tasks/ceph.py: cleanup: stop calling get_wwn_id_map()

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34212 from s0nea/wip-44709-octopus
Lenz Grimmer [Mon, 30 Mar 2020 10:54:16 +0000 (12:54 +0200)]
Merge pull request #34212 from s0nea/wip-44709-octopus

octopus: mgr/dashboard: correct Orchestrator documentation link

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agomgr/dashboard: do not fail on user creation 34280/head
Tatjana Dehler [Fri, 27 Mar 2020 14:58:42 +0000 (15:58 +0100)]
mgr/dashboard: do not fail on user creation

Accordingly to other Ceph commands do not fail on user
creation with a non-zero error code if the user already
exists.
Instead succeed and return the message 'User <username>
already exists'.

Fixes: https://tracker.ceph.com/issues/44502
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 9992faa9479b62f05f36352daf89f6e7541f29c9)

5 years agoMerge PR #34192 into octopus
Sage Weil [Sun, 29 Mar 2020 22:38:01 +0000 (17:38 -0500)]
Merge PR #34192 into octopus

* refs/pull/34192/head:
mgr/cephadm: add watch_url to NFS bootstrap config
mgr/cephadm: check for an existing NFS config object
mgr/cephadm: combine `get_daemon_name` and `get_rados_user` funcs
mon: allow `auth caps` for `mgr` profile
mgr/cephadm: add deployment example to NFS docs
mgr/cephadm: add utils.py
mgr/cephadm: create/update keyring during nfs config
mgr/cephadm: pass NFSServiceSpec instead of pool/namespace args
mgr/cephadm: add nfs apply test
mgr/cephadm: fix nfs redeploy/reconfig
mgr/cephadm: ensure nfs uses the correct entity
mgr/cephadm: fix service name for nfs type
mgr/cephadm: apply nfs service spec
mgr/cephadm: don't create a unique nfs daemon name
mgr/cephadm: update docs
mgr/cephadm: add minimal caps for nfs client keyring
mgr/cephadm: add logging to NFSGanesha class
mgr/cephadm: create empty nfs rados config obj
mgr/cephadm: add nfs tests
mgr/cephadm: add a minimal ganesha.conf
mgr/cephadm: deploy nfs-ganesha service
mgr/cephadm: split out `config-and-keyrings` json
mgr/orchestrator: rearrange interface to match impl. order

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #34258 into octopus
Sage Weil [Sun, 29 Mar 2020 22:37:46 +0000 (17:37 -0500)]
Merge PR #34258 into octopus

* refs/pull/34258/head:
cephadm: Fix trucated output of `mgr dump`

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #34199 into octopus
Sage Weil [Sun, 29 Mar 2020 22:37:30 +0000 (17:37 -0500)]
Merge PR #34199 into octopus

* refs/pull/34199/head:
mgr/rook: Blinking lights

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #34204 into octopus
Sage Weil [Sat, 28 Mar 2020 00:59:14 +0000 (19:59 -0500)]
Merge PR #34204 into octopus

* refs/pull/34204/head:
osd/PeeringState: drop mimic assert

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #34178 into octopus
Sage Weil [Sat, 28 Mar 2020 00:57:45 +0000 (19:57 -0500)]
Merge PR #34178 into octopus

* refs/pull/34178/head:
pybind/mgr/*: fix config_notify handling of default values

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #34123 into octopus
Sage Weil [Sat, 28 Mar 2020 00:57:13 +0000 (19:57 -0500)]
Merge PR #34123 into octopus

* refs/pull/34123/head:
osd/PeeringState: fix pending want_acting vs osd offline race

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agocephadm: Fix trucated output of `mgr dump` 34258/head
Sebastian Wagner [Fri, 27 Mar 2020 22:24:58 +0000 (23:24 +0100)]
cephadm: Fix trucated output of `mgr dump`

In case the process has terminated, we
have to make sure to read till EOF, otherwise
we might skip some bytes.

Fixes: https://tracker.ceph.com/issues/44642
Co-authored-by: Tim Serong <tserong@suse.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6a6759eef06c32efe1a10190925da6bc7e5ec4b0)

5 years agoqa/tasks/ceph.py: fail test if osd devices not found 34255/head
Nathan Cutler [Tue, 22 Oct 2019 14:45:04 +0000 (16:45 +0200)]
qa/tasks/ceph.py: fail test if osd devices not found

Fixes: https://tracker.ceph.com/issues/42357
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit bc76b39a30ba1301132b14dd65f0f9ee935c508b)

5 years agoqa/tasks/ceph.py: use .format to log dicts
Nathan Cutler [Tue, 22 Oct 2019 14:35:27 +0000 (16:35 +0200)]
qa/tasks/ceph.py: use .format to log dicts

The ".format" builtin logs dicts nicely right out of the box.

Also, some of the log messages were too cryptic - fixed them in this commit as
well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 9abebf28a8c519053bf2e89a944a0b1fdc167ddd)

5 years agoqa/tasks/ceph.py: drop roles_to_journals and remote_to_roles_to_journals
Nathan Cutler [Tue, 22 Oct 2019 14:31:14 +0000 (16:31 +0200)]
qa/tasks/ceph.py: drop roles_to_journals and remote_to_roles_to_journals

These do not seem to get any use anymore.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ad477be286e373aa00cbf0ab02f9bc9b1321663e)

5 years agoqa/tasks/ceph.py: drop block_journal, tmpfs_journal
Nathan Cutler [Tue, 22 Oct 2019 13:59:12 +0000 (15:59 +0200)]
qa/tasks/ceph.py: drop block_journal, tmpfs_journal

I looked, but did not find any tests that actually use these options.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 13933171297da0e35461e0c7420751867a645b5b)

5 years agoqa/tasks/ceph.py: cleanup: stop calling get_wwn_id_map()
Nathan Cutler [Tue, 22 Oct 2019 13:42:48 +0000 (15:42 +0200)]
qa/tasks/ceph.py: cleanup: stop calling get_wwn_id_map()

Nowadays, get_wwn_id_map is essentially a noop - it does:

    return dict((d, d) for d in devs)

This reverts another bit of 8f720454cbb536ac6f410fb6c18b2b0945b70160 from 2013.

References: https://tracker.ceph.com/issues/42313
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 51c714d9b29402f21b25b5754d42f917964079e2)

5 years agoMerge pull request #34214 from neha-ojha/wip-34183-octopus
Josh Durgin [Fri, 27 Mar 2020 19:39:25 +0000 (12:39 -0700)]
Merge pull request #34214 from neha-ojha/wip-34183-octopus

octopus: doc/releases/octopus.rst: fix min_mon_release string

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #34207 from liewegas/bp-34188-octopus
Nathan Cutler [Fri, 27 Mar 2020 10:06:32 +0000 (11:06 +0100)]
Merge pull request #34207 from liewegas/bp-34188-octopus

octopus: doc: do not encourage readers to run multiple clusters on same HW

Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #34237 from smithfarm/wip-sle-15.2-octopus
Nathan Cutler [Fri, 27 Mar 2020 09:32:30 +0000 (10:32 +0100)]
Merge pull request #34237 from smithfarm/wip-sle-15.2-octopus

octopus: qa/distros: add SLE-15-SP2

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
5 years agodoc: fix cephadm link 34207/head
Nathan Cutler [Thu, 26 Mar 2020 13:09:59 +0000 (14:09 +0100)]
doc: fix cephadm link

Fixes: 8fe460e7562b28e007179eeb533ef68a3e99b4c8
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7dd012252ce3be13d8d44a5c3829198fc72b5754)

5 years agoqa/distros: add SLE-15-SP2 34237/head
Thomas Bechtold [Fri, 27 Mar 2020 07:16:28 +0000 (08:16 +0100)]
qa/distros: add SLE-15-SP2

Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
(cherry picked from commit 3261e24636474a3e274c65b859e9325299470f11)

5 years agodoc/start: Update hardware recommendations. 34225/head
Mark Nelson [Thu, 26 Mar 2020 20:07:50 +0000 (15:07 -0500)]
doc/start: Update hardware recommendations.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 3305a0732403ba7fe58fe6248868b7711bbccc72)

5 years agodoc/releases/octopus.rst: fix min_mon_release string 34214/head
Neha [Wed, 25 Mar 2020 17:29:48 +0000 (17:29 +0000)]
doc/releases/octopus.rst: fix min_mon_release string

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c1c6f8f8c191d206659116bbc04695999478b450)

5 years agoMerge PR #34194 into octopus
Sage Weil [Thu, 26 Mar 2020 15:04:16 +0000 (10:04 -0500)]
Merge PR #34194 into octopus

* refs/pull/34194/head:
CMakeLists: 15.2.0
ceph_release: stable, not rc

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #33939 into octopus
Sage Weil [Thu, 26 Mar 2020 14:59:31 +0000 (09:59 -0500)]
Merge PR #33939 into octopus

* refs/pull/33939/head:
mgr/dashboard: NoRebalance flag is added to the Dashboard

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge PR #34163 into octopus
Sage Weil [Thu, 26 Mar 2020 14:59:19 +0000 (09:59 -0500)]
Merge PR #34163 into octopus

* refs/pull/34163/head:
mgr: ceph_pg_* metrics contains last value instead of sum all of them

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #34185 into octopus
Sage Weil [Thu, 26 Mar 2020 14:59:07 +0000 (09:59 -0500)]
Merge PR #34185 into octopus

* refs/pull/34185/head:
mgr/cephadm: DriveGroupSpec needs to know about unmanaged attribute
mgr/cephadm: DriveGroupSpec needs to support/ignore _unmanaged_

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
5 years agoMerge PR #34186 into octopus
Sage Weil [Thu, 26 Mar 2020 14:58:55 +0000 (09:58 -0500)]
Merge PR #34186 into octopus

* refs/pull/34186/head:
mgr/cephadm: use docker.io in default container image name
mgr/cephadm: fix 'upgrade start' message when specifying a version

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #34190 into octopus
Sage Weil [Thu, 26 Mar 2020 14:58:42 +0000 (09:58 -0500)]
Merge PR #34190 into octopus

* refs/pull/34190/head:
mgr/cephadm: fix 'orch ps --refresh'

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #34191 into octopus
Sage Weil [Thu, 26 Mar 2020 14:58:29 +0000 (09:58 -0500)]
Merge PR #34191 into octopus

* refs/pull/34191/head:
mgr/cephadm: bind grafana to all interfaces

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agomgr/dashboard: correct Orchestrator documentation link 34212/head
Tatjana Dehler [Mon, 23 Mar 2020 10:45:53 +0000 (11:45 +0100)]
mgr/dashboard: correct Orchestrator documentation link

Fixes: https://tracker.ceph.com/issues/44708
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 6e91edb287188f64694f82ba55d35fc2d589bbe6)

5 years agodoc: do not encourage readers to run multiple clusters on same HW
Nathan Cutler [Wed, 25 Mar 2020 22:41:28 +0000 (23:41 +0100)]
doc: do not encourage readers to run multiple clusters on same HW

This material was first added on December 22, 2012 by
46d0334456c14da2bc8e012f121c545b1b33b95f

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 8fe460e7562b28e007179eeb533ef68a3e99b4c8)

5 years agoMerge pull request #34176 from theanalyst/backports/oct-distro-fix
Abhishek L [Thu, 26 Mar 2020 12:10:46 +0000 (13:10 +0100)]
Merge pull request #34176 from theanalyst/backports/oct-distro-fix

octopus: doc: releases: add a note on debian packages & those built by the community

Reviewed-By: Sage Weil <sweil@redhat.com>
5 years agoosd/PeeringState: drop mimic assert 34204/head
Sage Weil [Wed, 25 Mar 2020 22:13:46 +0000 (17:13 -0500)]
osd/PeeringState: drop mimic assert

This code can still execute for a cluster that has require_osd_release >=
nautilus because a PG is processing an older OSDMap.  Drop this assert
since the osd boot checks on require_osd_release are sufficient.

Fixes: https://tracker.ceph.com/issues/44759
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 29b7469fb7196f0ce7b2ae5d0a21226ab15ba306)

5 years agomgr/dashboard: Update Angular and fix npm audits 34202/head
Tiago Melo [Thu, 12 Mar 2020 14:34:56 +0000 (13:34 -0100)]
mgr/dashboard: Update Angular and fix npm audits

This fixes a new audit problem found in older versions of "acorn"
and updates Angular to latest v8 version.

Fixes: https://tracker.ceph.com/issues/44580
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit e96cfbd8c5e56b061af59a11850b064147a9f759)

5 years agomgr/dashboard: Update prettier
Tiago Melo [Thu, 5 Mar 2020 18:37:23 +0000 (17:37 -0100)]
mgr/dashboard: Update prettier

Fix all new linting errors.

Fixes: https://tracker.ceph.com/issues/44451
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit bd08daad2e380c1df2d9ba49609210095a28470d)

5 years agomgr/dashboard: Check for missing npm resolutions
Tiago Melo [Mon, 23 Mar 2020 19:54:06 +0000 (18:54 -0100)]
mgr/dashboard: Check for missing npm resolutions

Check if it's necessary to run "npm run fix:audit" before committing the changes.

This is important to resolve some issues with building and
to fix some npm vulnerabilities.

Fixes: https://tracker.ceph.com/issues/44723
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit b7b189c9d1308335efb42b18c45d58a1e8469b82)

5 years agomgr/rook: Blinking lights 34199/head
Juan Miguel Olmo Martínez [Mon, 17 Feb 2020 09:59:46 +0000 (10:59 +0100)]
mgr/rook: Blinking lights

Blinking lights implementation

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit a35c1b25933ba5583e9c6b8752d2949462f92025)

5 years agoCMakeLists: 15.2.0 34194/head
Sage Weil [Thu, 26 Mar 2020 02:02:06 +0000 (21:02 -0500)]
CMakeLists: 15.2.0

I'm not actually sure if this is used for much of anything...

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph_release: stable, not rc
Sage Weil [Thu, 26 Mar 2020 02:01:20 +0000 (21:01 -0500)]
ceph_release: stable, not rc

This should have happened before v15.2.0.. oops!

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: add watch_url to NFS bootstrap config 34192/head
Michael Fritch [Mon, 23 Mar 2020 23:41:09 +0000 (17:41 -0600)]
mgr/cephadm: add watch_url to NFS bootstrap config

watch for changes to the per-daemon RADOS conf file

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ff9265104f69c702630e5069b1e1a2105d1f74fa)

5 years agomgr/cephadm: check for an existing NFS config object
Michael Fritch [Wed, 18 Mar 2020 19:57:18 +0000 (13:57 -0600)]
mgr/cephadm: check for an existing NFS config object

assume an existing NFS config if the named rados object already exists

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 75b988f314ab8792cccad70bb9894ff1a50ae3f3)

5 years agomgr/cephadm: combine `get_daemon_name` and `get_rados_user` funcs
Michael Fritch [Tue, 17 Mar 2020 21:14:55 +0000 (15:14 -0600)]
mgr/cephadm: combine `get_daemon_name` and `get_rados_user` funcs

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fdf90bf530edbb9608fb8a5d273f7ce38147f534)

5 years agomon: allow `auth caps` for `mgr` profile
Michael Fritch [Tue, 17 Mar 2020 17:17:17 +0000 (11:17 -0600)]
mon: allow `auth caps` for `mgr` profile

cephadm orchestrator needs to update the caps during reconfig/redeploy

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ef8a46a3b13ea6753b8c03121da4c6daff370fd2)

5 years agomgr/cephadm: add deployment example to NFS docs
Michael Fritch [Tue, 17 Mar 2020 04:48:31 +0000 (22:48 -0600)]
mgr/cephadm: add deployment example to NFS docs

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 77440d91c1be7c872fc91042a0d723b9198f224f)

5 years agomgr/cephadm: add utils.py
Michael Fritch [Tue, 17 Mar 2020 04:47:36 +0000 (22:47 -0600)]
mgr/cephadm: add utils.py

move `name_to_config_section` and `assert_valid_host` into utils.py

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 322eea8099616b25a091ac66a1253d1991a0da2c)

5 years agomgr/cephadm: create/update keyring during nfs config
Michael Fritch [Mon, 16 Mar 2020 18:05:56 +0000 (12:05 -0600)]
mgr/cephadm: create/update keyring during nfs config

the keyring might already exist from a prior config/reconfig/redeploy.

attempt to get_or_create the keyring first and than update the keyring
caps afterward

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 084fd4a91ae781ab7bed9e06ceca17b1a5ca5be2)

5 years agomgr/cephadm: pass NFSServiceSpec instead of pool/namespace args
Michael Fritch [Sun, 15 Mar 2020 18:14:05 +0000 (12:14 -0600)]
mgr/cephadm: pass NFSServiceSpec instead of pool/namespace args

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 8bb79a5f275ecf79428dc35a8fc1a76c36569df3)

5 years agomgr/cephadm: add nfs apply test
Michael Fritch [Sun, 15 Mar 2020 17:08:23 +0000 (11:08 -0600)]
mgr/cephadm: add nfs apply test

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 78ce6d431806c1f3e8e55e946eff64d450821db1)

5 years agomgr/cephadm: fix nfs redeploy/reconfig
Michael Fritch [Sun, 15 Mar 2020 16:53:32 +0000 (10:53 -0600)]
mgr/cephadm: fix nfs redeploy/reconfig

search for the `NFSServiceSpec` instead of passing empty
`cephadm_args` during `_create_daemon`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 107885801bb5d7401dead52cbc2abadaa2865a91)

5 years agomgr/cephadm: ensure nfs uses the correct entity
Michael Fritch [Fri, 13 Mar 2020 20:19:23 +0000 (14:19 -0600)]
mgr/cephadm: ensure nfs uses the correct entity

mon entity needs to start with `client.`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fd6322f7ec51f3dc57168bbd08ffab3c832a4598)

5 years agomgr/cephadm: fix service name for nfs type
Michael Fritch [Mon, 9 Mar 2020 23:21:11 +0000 (17:21 -0600)]
mgr/cephadm: fix service name for nfs type

the `nfs` service was incorrectly reported as `mds`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d9f04ec1d73948e915dda0f51592306542314238)

5 years agomgr/cephadm: apply nfs service spec
Michael Fritch [Mon, 9 Mar 2020 18:44:06 +0000 (12:44 -0600)]
mgr/cephadm: apply nfs service spec

orch apply nfs <svc_id> [<placement>]

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 76f27894caba2b3bc146c79d86c4335c9c33dcde)

5 years agomgr/cephadm: don't create a unique nfs daemon name
Michael Fritch [Mon, 9 Mar 2020 18:38:25 +0000 (12:38 -0600)]
mgr/cephadm: don't create a unique nfs daemon name

only one nfs daemon will be deployed per host

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 81823176af101864524fc611fff346bf9798f0e0)

5 years agomgr/cephadm: update docs
Michael Fritch [Sun, 8 Mar 2020 23:27:54 +0000 (17:27 -0600)]
mgr/cephadm: update docs

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 530c62d76d79c4dd90a966302cd07d1017261686)

5 years agomgr/cephadm: add minimal caps for nfs client keyring
Michael Fritch [Sun, 8 Mar 2020 16:53:17 +0000 (10:53 -0600)]
mgr/cephadm: add minimal caps for nfs client keyring

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 94600c69e2f547c02017204c909d11d626649066)