]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge pull request #37369 from smithfarm/wip-47255-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:37:33 +0000 (11:37 -0700)]
Merge pull request #37369 from smithfarm/wip-47255-octopus

octopus: cephfs: client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37368 from smithfarm/wip-47253-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:37:03 +0000 (11:37 -0700)]
Merge pull request #37368 from smithfarm/wip-47253-octopus

octopus: cephfs: osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37090 from shyukri/wip-46961-octopus
Lenz Grimmer [Fri, 9 Oct 2020 12:00:09 +0000 (14:00 +0200)]
Merge pull request #37090 from shyukri/wip-46961-octopus

octopus: mgr/dashboard: Improve notification badge

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #37300 from alvistack/octopus-aio_graceful_reboot
Yuri Weinstein [Thu, 8 Oct 2020 19:38:37 +0000 (12:38 -0700)]
Merge pull request #37300 from alvistack/octopus-aio_graceful_reboot

octopus: systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37533 from smithfarm/wip-47739-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:58:00 +0000 (11:58 -0700)]
Merge pull request #37533 from smithfarm/wip-47739-octopus

octopus: mgr/devicehealth: device_health_metrics pool gets created even without any OSDs in the cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37225 from smithfarm/wip-47531-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:56:20 +0000 (11:56 -0700)]
Merge pull request #37225 from smithfarm/wip-47531-octopus

octopus: ceph.in: ignore failures to flush stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37128 from k0ste/wip-47426-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:55:54 +0000 (11:55 -0700)]
Merge pull request #37128 from k0ste/wip-47426-octopus

octopus: mgr/zabbix: indent the output of "zabbix config-show"

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37048 from trociny/wip-47363-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:54:02 +0000 (11:54 -0700)]
Merge pull request #37048 from trociny/wip-47363-octopus

octopus: bluestore: fix collection_list ordering

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929...
Nathan Cutler [Thu, 8 Oct 2020 17:18:29 +0000 (19:18 +0200)]
Merge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929-36995-36946-36432-36903-36766-37046-37071-36980-3693

octopus: cephadm batch backport September (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
4 years agoMerge PR #37345 into octopus
Jan Fajerski [Wed, 7 Oct 2020 13:32:57 +0000 (15:32 +0200)]
Merge PR #37345 into octopus

* refs/pull/37345/head:
ceph-volume: avoid format strings for now

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agodoc: replace luminous with |stable-release| 37436/head
Kefu Chai [Fri, 28 Aug 2020 10:26:26 +0000 (18:26 +0800)]
doc: replace luminous with |stable-release|

"luminous" is a magic number in these contexts, and we should use a
constant for representing it.

the "sphinx_substitution_extensions" sphinx extension is introduced for
performing the global subsitution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7ca85e77d040e990fc60d7fe99447d3fb1e7f69e)

4 years agoMerge pull request #37337 from smithfarm/wip-47319-octopus
Yuri Weinstein [Tue, 6 Oct 2020 18:44:26 +0000 (11:44 -0700)]
Merge pull request #37337 from smithfarm/wip-47319-octopus

octopus: rgw: RGWObjVersionTracker tracks version over increments

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37332 from callithea/wip-47569-octopus
Lenz Grimmer [Tue, 6 Oct 2020 13:31:44 +0000 (15:31 +0200)]
Merge pull request #37332 from callithea/wip-47569-octopus

octopus: mgr/dashboard: table detail rows overflow

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
4 years agoadmin/build-doc: use "ceph" for prefix of functions defined by libcephfs
Kefu Chai [Wed, 11 Mar 2020 14:41:29 +0000 (22:41 +0800)]
admin/build-doc: use "ceph" for prefix of functions defined by libcephfs

if some python module tries to `import cephfs`, it should not fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b9e282f0e3eed2cacf1aac0853dc6607053607a3)

4 years agoadmin/build-doc: pass -rpath to linker
Kefu Chai [Wed, 11 Mar 2020 14:39:59 +0000 (22:39 +0800)]
admin/build-doc: pass -rpath to linker

to make sure the dummy cythonized python binding is able to find the dummy C binding.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1183f267192681df2148d44e51ad56a531b5a45b)

4 years agomgr/cephadm: fixup expected extra ceph conf test result
Michael Fritch [Fri, 18 Sep 2020 14:54:00 +0000 (08:54 -0600)]
mgr/cephadm: fixup expected extra ceph conf test result

fix test failure introduced by:
ff7e76348e5457fa6acb23545fcef56d6640c50a

```
E           AssertionError: expected call not found.
E           Expected: _run_cephadm('test', 'mon.test', 'deploy', ['--name', 'mon.test', '--reconfig', '--config-json', '-'], stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}')
E           Actual: _run_cephadm('test', 'mon.test', 'deploy',
['--name', 'mon.test', '--reconfig', '--config-json', '-'],
stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}', image='')
```

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b3205210a18c7270b93551925d0dc91e050969cb)

4 years agocephadm: capadd and privileged are mutex
Joshua Schmid [Mon, 14 Sep 2020 08:38:07 +0000 (10:38 +0200)]
cephadm: capadd and privileged are mutex

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 76e5020b106e14284f63bd7cee81822ad6b1fbf0)

4 years agodocs: s/svc_id/osd_id
Joshua Schmid [Fri, 18 Sep 2020 12:32:33 +0000 (14:32 +0200)]
docs: s/svc_id/osd_id

Fixes: https://tracker.ceph.com/issues/47402
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d7d1267686474ca9d16410ebe21ba89038785c56)

4 years agocephadm: support multiple mounts when running shell
Guillaume Abrioux [Wed, 9 Sep 2020 08:14:33 +0000 (10:14 +0200)]
cephadm: support multiple mounts when running shell

This commit adds the multiple mounts support when running the
interactive shell.

ie:
```
--mount /foo /bar:/bar:z
```

Keeping default destination `/mnt` when no destination is passed for
backward compatibility. In the above example `/foo` will be mounted in
`/mnt/foo` and `/bar` in `/bar`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3ed3cff0d0ea2744dff816270d49539fd73c1691)

4 years agomgr/cephadm: Add extra-ceph-conf
Sebastian Wagner [Sat, 29 Aug 2020 20:16:42 +0000 (22:16 +0200)]
mgr/cephadm: Add extra-ceph-conf

Due to limitations of `config generate-minimal-conf`,
we need to provide a way for users to modify the
ceph.conf for the daemons.

This now provides a way to append config to
the daemons config. Like. e.g. `cluster_network`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ff7e76348e5457fa6acb23545fcef56d6640c50a)

4 years agomgr/cephadm: Refactor (to,from) datetime
Sebastian Wagner [Sat, 29 Aug 2020 20:30:29 +0000 (22:30 +0200)]
mgr/cephadm: Refactor (to,from) datetime

`datetime.datetime.strptime(input, DATEFMT)` just
looks too ugly to have it everywhere.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d65092f0fa616e623993a422176fb51ecf1245bf)

4 years agomgr/cephadm: Refactor `get_minimal_ceph_conf()`
Sebastian Wagner [Sat, 29 Aug 2020 19:36:09 +0000 (21:36 +0200)]
mgr/cephadm: Refactor `get_minimal_ceph_conf()`

Create a single source for the minimal ceph.conf

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fd4611836d3aa36882a18167d27af1b06715a0a6)

4 years agomgr/cephadm: CephadmDaemonSpec: add ceph_conf attribute
Sebastian Wagner [Sat, 29 Aug 2020 19:21:56 +0000 (21:21 +0200)]
mgr/cephadm: CephadmDaemonSpec: add ceph_conf attribute

* `extra_config` should actually be called `extra_files`
* Special treatment for `ceph_conf` to ease refactorization

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 75aa97ee3fec290682afde9eb5a3254db2104ece)

4 years agomgr/cephadm: Fix race between host_add and _apply_all_specs
Sebastian Wagner [Mon, 14 Sep 2020 10:17:43 +0000 (12:17 +0200)]
mgr/cephadm: Fix race between host_add and _apply_all_specs

This mitigates a potential race, where new host was added *after*
``_refresh_host_daemons()`` was called, but *before*
``_apply_all_specs()`` was called. thus we end up with a hosts
where daemons might be running, but we have not yet detected them.

Imagine:

1. the initial MGR was deployed manually
2. `serve()` is called
3. `_refresh_hosts_and_daemons()` is called
4. `add_host()` is called with the initial host
5. `apply_all_specs()` now deploys an additional MGR on the initial host

Fixes: https://tracker.ceph.com/issues/45093
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 72e3990382d65205de7871cc86b5063ed16732e5)

4 years agopython-common: deprecate filter_matching_hosts
Sebastian Wagner [Mon, 14 Sep 2020 09:34:32 +0000 (11:34 +0200)]
python-common: deprecate filter_matching_hosts

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7b7aeafad1bb4e3f0818417008f5df616de8f90c)

4 years agomgr/cephadm: Simplified test_scheduling
Sebastian Wagner [Mon, 14 Sep 2020 09:29:51 +0000 (11:29 +0200)]
mgr/cephadm: Simplified test_scheduling

Removed `get_hosts_func`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5750f32e9ddfdbe148b60d98f1196c90e134478a)

4 years agomgr/cephadm: Simplify HostAssignment arguments
Sebastian Wagner [Mon, 14 Sep 2020 09:27:02 +0000 (11:27 +0200)]
mgr/cephadm: Simplify HostAssignment arguments

`get_hosts_func` -> `hosts`, as `get_hosts_func` is
only called once in __init__

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 908bd443f335979f428833059161673b5330e745)

4 years agomgr/cephadm: Scheduler: fetch hosts only once
Sebastian Wagner [Mon, 14 Sep 2020 09:18:38 +0000 (11:18 +0200)]
mgr/cephadm: Scheduler: fetch hosts only once

Otherwise we might be prone to races, e.g.
a new hosts gets added in between calls to
`get_hosts_func`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3b841fc7fa5c13dbc212053aeb8deda012a3de0d)

4 years agocephadm: Introduce 'container' specification to deploy custom containers
Volker Theile [Wed, 16 Sep 2020 10:06:36 +0000 (12:06 +0200)]
cephadm: Introduce 'container' specification to deploy custom containers

Fixes: https://tracker.ceph.com/issues/46666
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7219377cfcef7ccec89b34e934388b8753608e15)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.ts

4 years agomgr/cephadm: fix RemoveUtil.load_from_store()
Sebastian Wagner [Mon, 14 Sep 2020 13:15:12 +0000 (15:15 +0200)]
mgr/cephadm: fix RemoveUtil.load_from_store()

* Run `json.loads` only once
* `nodename` -> `hostname`
* Remove network calls from `repr(OSD(...)`

Fixes: https://tracker.ceph.com/issues/47438
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0bcd29a247f4a01ba69c6d43db668ef3dcc1610d)

4 years agoqa: spawn MDS daemons before creating fs
Patrick Donnelly [Thu, 17 Sep 2020 16:01:33 +0000 (09:01 -0700)]
qa: spawn MDS daemons before creating fs

This avoids unnecessary MDS_ALL_DOWN messages because the MDS daemons
have not yet been spawned.

Fixes: https://tracker.ceph.com/issues/47518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 61db7a9c2e47169017f92df05c843f5b8d0526ee)

4 years agodoc/cephadm: use appropriate directive for formatting codeblocks
Kefu Chai [Fri, 11 Sep 2020 23:04:14 +0000 (07:04 +0800)]
doc/cephadm: use appropriate directive for formatting codeblocks

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 967104103c6c611b122e3e37227a04486cd6d2b1)

4 years agomgr/rook: Pass pod namespace to list_namespaced_pod()
Varsha Rao [Thu, 17 Sep 2020 06:59:40 +0000 (12:29 +0530)]
mgr/rook: Pass pod namespace to list_namespaced_pod()

As list_namespaced_pod method requires pod namespace instead of cluster name.

Fixes: https://tracker.ceph.com/issues/47511
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit a6c6c114afa9cdc89508ae90bbe837421914d342)

4 years agomgr/rook: fix listing daemons
Kiefer Chang [Thu, 17 Sep 2020 03:37:15 +0000 (11:37 +0800)]
mgr/rook: fix listing daemons

Pass the parameters to the `_list_daemons` method explicitly to avoid missing
parameters.

Fixes: https://tracker.ceph.com/issues/47387
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 8098ca765fadce18c3738df04f9eb5ddf786c113)

4 years agocephadm: filter dangling images in get_last_local_ceph_image()
Guillaume Abrioux [Wed, 16 Sep 2020 08:50:50 +0000 (10:50 +0200)]
cephadm: filter dangling images in get_last_local_ceph_image()

if `podman images --filter label=ceph=True --format {{.Repository}} {{.Tag}}`
returns something like following:

```
<none>                <none>
docker.io/ceph/ceph   v15
```

it will end up throwing an error:

```
    r = args.func()
  File "./cephadm", line 1147, in _infer_fsid
    return func()
  File "./cephadm", line 1184, in _infer_config
    return func()
  File "./cephadm", line 1230, in _infer_image
    return func()
  File "./cephadm", line 3283, in command_shell
    make_log_dir(args.fsid)
  File "./cephadm", line 1325, in make_log_dir
    uid, gid = extract_uid_gid()
  File "./cephadm", line 1913, in extract_uid_gid
    raise RuntimeError('uid/gid not found')
RuntimeError: uid/gid not found
```

adding `--filter dangling=false` can help to avoid this.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 01880cebcd64bc83ea722217e2925ab2b6304c65)

4 years agodoc/mgr: fix outdated `orch apply` docs.
Sebastian Wagner [Mon, 31 Aug 2020 08:14:20 +0000 (10:14 +0200)]
doc/mgr: fix outdated `orch apply` docs.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f3bd8f2569a8315755fe89fb16199eacc9bcb6c7)

4 years agomgr/cephadm: CephadmService.create -> CephadmService.prepare_create
Sebastian Wagner [Fri, 11 Sep 2020 17:49:57 +0000 (19:49 +0200)]
mgr/cephadm: CephadmService.create -> CephadmService.prepare_create

Refactored this to simplify the control flow. It became obvious that

* `CephadmService.make_daemon_spec`
* `CephadmService.prepare_create`
* `CephadmService.generate_config`

are basically the same thing. They're all called directly
before the deployment of a daemon. All of them should be
unified. This PR makes this refactorization simpler

By renaming `create` to `prepare_create`, we make `create`
no longer being the entrypoint to call
`create_daemon`. Thus all the functions above
return some data structures.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 245cf50132e1731982d19e1114116e4150b9ea97)

4 years agopython-common/deployment: respect existing osds
Joshua Schmid [Tue, 14 Jul 2020 15:01:34 +0000 (17:01 +0200)]
python-common/deployment: respect existing osds

Fixes: https://tracker.ceph.com/issues/44888
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 6bfc4589f55c76d50cc87f75c4018860167f7708)

4 years agomgr/cephadm: allow to map existing daemons to osdspecs
Joshua Schmid [Tue, 14 Jul 2020 15:01:13 +0000 (17:01 +0200)]
mgr/cephadm: allow to map existing daemons to osdspecs

Fixes: https://tracker.ceph.com/issues/44888
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 6a374c583c2963a0bfc28fd6b54f5f91b6f8d26f)

4 years agomgr/cephadm: allow iscsi to be named by teuthology
Sebastian Wagner [Fri, 11 Sep 2020 12:18:54 +0000 (14:18 +0200)]
mgr/cephadm: allow iscsi to be named by teuthology

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1bd86d33166246d0b49997aabe6d146be2171c1a)

4 years agoqa/cephadm: Also fetch iscsi logs
Sebastian Wagner [Wed, 1 Jul 2020 09:03:20 +0000 (11:03 +0200)]
qa/cephadm: Also fetch iscsi logs

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 130c7a438ec4201638bb182e026cc359d1a4cbd9)

4 years agoqa/cephadm: Add iSCSI
Sebastian Wagner [Thu, 11 Jun 2020 14:11:29 +0000 (16:11 +0200)]
qa/cephadm: Add iSCSI

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 430f2c51320ed0092c17273ea12e6b527ae735ed)

4 years agomgr/cephadm: merge update_keyring_caps -> create_keyring
Michael Fritch [Fri, 11 Sep 2020 14:10:44 +0000 (08:10 -0600)]
mgr/cephadm: merge update_keyring_caps -> create_keyring

keyring is removed during daemon remove, which obsoletes the need to
update the keyring caps after get-or-create

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit eea25b5ac722422369653daebb669749b9cf1bba)

4 years agomgr/cephadm: disallow removing the mon keyring
Michael Fritch [Fri, 11 Sep 2020 14:10:38 +0000 (08:10 -0600)]
mgr/cephadm: disallow removing the mon keyring

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2266cc1e75a25ee46bbe8964af077b83f256a864)

4 years agomgr/cephadm: remove keyring during daemon post-remove
Michael Fritch [Fri, 11 Sep 2020 14:10:32 +0000 (08:10 -0600)]
mgr/cephadm: remove keyring during daemon post-remove

Fixes: https://tracker.ceph.com/issues/47298
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 11bbfd5836150083d97371d82482dec2eb14bb4a)

4 years agomgr/cephadm: pass `DaemonDescription` during pre/post remove
Michael Fritch [Fri, 11 Sep 2020 14:10:26 +0000 (08:10 -0600)]
mgr/cephadm: pass `DaemonDescription` during pre/post remove

add more context than simply passing the `daemon_id`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 183f7c34a303333fa5fdfbfa79f18bed725c3a83)

4 years agomgr/cephadm: add `post_remove` daemon action
Michael Fritch [Fri, 11 Sep 2020 14:10:17 +0000 (08:10 -0600)]
mgr/cephadm: add `post_remove` daemon action

invoked after a daemon has been removed

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit a0cefab9e5c443abaddecdab3da4189fca4f8144)

4 years agomgr/cephadm: make j2 teamplates overwritable
Sebastian Wagner [Fri, 11 Sep 2020 13:04:49 +0000 (15:04 +0200)]
mgr/cephadm: make j2 teamplates overwritable

This adds an escape hatch, if the configs generated
by cephadm are not flexible enough.

Downside is obviously that future upgrades might
break the daemons. Use at your own risk.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit cfc09e4d7903c404901344b4a7bdd6c5499603fb)

4 years agocephadm: remove z option from iSCSI dev and configfs mount
Adam King [Fri, 4 Sep 2020 12:06:37 +0000 (08:06 -0400)]
cephadm: remove z option from iSCSI dev and configfs mount

selinux complains if you try to deploy iSCSI with the z option
on the /dev and /sys/kernel/config mounts

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 351b92acdacd2bea0c7c0e7c08b0b82f277621f8)

4 years agomgr/cephadm: Call cephadm with --container-image
Sebastian Wagner [Wed, 26 Aug 2020 12:53:09 +0000 (14:53 +0200)]
mgr/cephadm: Call cephadm with --container-image

The kernel treats any process with PID 1 different. Especially
it does not generate a core dump. Call podman / docker with
--init in order to get core dumps.

In addition, we can now properly reap zombies processes.

Fixes: https://tracker.ceph.com/issues/44231
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 533931505272d215a1b4fa40b3bf235cf2110102)

4 years agocephadm: Add --container-image
Sebastian Wagner [Wed, 26 Aug 2020 12:45:34 +0000 (14:45 +0200)]
cephadm: Add --container-image

The kernel treats any process with PID 1 different. Especially
it does not generate a core dump. Call podman / docker with
--init in order to get core dumps.

In addition, we can now properly reap zombies processes.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 786e1c175b41d07d8eb3d86156d77e7769ae92ac)

4 years agomgr/cephadm: introduce `CephService` class
Michael Fritch [Tue, 1 Sep 2020 20:39:17 +0000 (14:39 -0600)]
mgr/cephadm: introduce `CephService` class

- split the cephx related keyring functions into a `CephService` class
- move _get_config_and_keyring and get_keyring_entity to this new class
- monitoring stack (prom, grafana, etc) have no use for cephx keyrings

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit c133705892cce3bd9ccd981e8080511bdbc12a0e)

4 years agomgr/cephadm: refactor `get_auth_entity()`
Michael Fritch [Tue, 1 Sep 2020 19:32:44 +0000 (13:32 -0600)]
mgr/cephadm: refactor `get_auth_entity()`

- convert to f-strings
- mgr keyring format is same as osd, mds
- crash client keyring is same as nfs, iscsi, etc

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 9d2031f01de4542f9a03f1234d5794395ae9162b)

4 years agomgr/cephadm: use `get_auth_entity()`
Michael Fritch [Tue, 1 Sep 2020 19:15:49 +0000 (13:15 -0600)]
mgr/cephadm: use `get_auth_entity()`

- some services have hard-coded auth entity names
- some services are incorrectly using `utils.name_to_config_section()`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b41bb98a98f853b32b3cdaf56d0c40a3229d2854)

4 years agomgr/cephadm: move `name_to_auth_entity` to `get_auth_entity`
Michael Fritch [Tue, 1 Sep 2020 18:32:04 +0000 (12:32 -0600)]
mgr/cephadm: move `name_to_auth_entity` to `get_auth_entity`

refactor the logic for getting the keyring auth entity from the utils
module into the CephadmService class

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 5e4140944f23f5cf90ee2dab91857f7b6f7a8bfc)

4 years agocephadm: fix iSCSI unit.run file
Adam King [Thu, 3 Sep 2020 16:30:33 +0000 (12:30 -0400)]
cephadm: fix iSCSI unit.run file

Current unit.run file generated for iSCSI when using podman
tries to have two containers use the same conmon pidfile and
cidfile for both containers which is invalid
Fixes: https://tracker.ceph.com/issues/47291
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0d993640af3f3ff9572e62af9811d94a992df0dd)

4 years agocephadm: Remove assignment to member variable in ServiceSpecs
Volker Theile [Wed, 9 Sep 2020 16:08:17 +0000 (18:08 +0200)]
cephadm: Remove assignment to member variable in ServiceSpecs

Remove unnecessary assignment to member variable 'preview_only', this is done in the constructor of the derived ServiceSpec class.

Fixes: https://tracker.ceph.com/issues/47384
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 6443ac5efa9816da580fe8546942af0cdf4ed271)

4 years agocephadm: unit.run: create /var/run/ceph/$FSID before doing anything else
Tim Serong [Tue, 8 Sep 2020 10:50:35 +0000 (20:50 +1000)]
cephadm: unit.run: create /var/run/ceph/$FSID before doing anything else

This ensures the /var/run/ceph/$FSID directory exists before any
other commands are run.

Fixes: https://tracker.ceph.com/issues/47360
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 721b4de3490c739635d8cf6623f36d142acb8fac)

4 years agocephadm: log to file
Juan Miguel Olmo Martínez [Mon, 24 Aug 2020 11:05:31 +0000 (13:05 +0200)]
cephadm: log to file

Log configuration improved adding a log file with debug level.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 81a7df0498dc649c68971ca31b9ae60874f61133)

4 years agopython-common: Make HostPlacementSpec.to_json() more friendly
Sebastian Wagner [Mon, 31 Aug 2020 09:12:47 +0000 (11:12 +0200)]
python-common: Make HostPlacementSpec.to_json() more friendly

We will now generate

```yaml
placement:
  hosts:
  - ceph-001
```

instead of

```yaml
placement:
  hosts:
  - hostname: ceph-001
    name: ''
    network: ''
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 642a6f13130f5ae154679188233c1e03fdca5e38)

4 years agomgr/cephadm: upgrade: Make use of repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 13:51:53 +0000 (15:51 +0200)]
mgr/cephadm: upgrade: Make use of repo_digest

* Automatically convert tags like `:latest` to the digest
* Use the digest instead of the tag

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 303bbdfe3d42db390c936334f3bde9f7cd7baac5)

4 years agoqa/cephadm: Add mgr/cephadm/use_repo_digest true to upgrade test
Sebastian Wagner [Mon, 3 Aug 2020 12:54:22 +0000 (14:54 +0200)]
qa/cephadm: Add mgr/cephadm/use_repo_digest true to upgrade test

As this is the most interesting test suite

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d390f4623530300cf4d8b12569eb6a98797d074b)

4 years agomgr/cephadm: convert tags to repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 12:36:56 +0000 (14:36 +0200)]
mgr/cephadm: convert tags to repo_digest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 19e23bef288de590d47bb4147603b422aa96ef06)

4 years agomgr/cephadm: extract set_continer_image(entity, image)
Sebastian Wagner [Fri, 7 Aug 2020 14:51:26 +0000 (16:51 +0200)]
mgr/cephadm: extract set_continer_image(entity, image)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8da1f38e92c3fc8d15d2a7d364b7455a26d9fe9b)

4 years agocephadm: inspect_image also returns repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 12:00:47 +0000 (14:00 +0200)]
cephadm: inspect_image also returns repo_digest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 30d8787c577727a3146c7ca32488936fc972c802)

Conflicts:
src/pybind/mgr/cephadm/module.py

4 years agodoc/cephadm: managing ceph.conf
Zac Dover [Wed, 2 Sep 2020 09:19:40 +0000 (19:19 +1000)]
doc/cephadm: managing ceph.conf

This commit repairs PR#36796, which was about
updating the configuration settings in ceph.conf
without touching the file directly.

That commit wasn't building because of a couple of
back quotes that were touching the word that followed
them. So much like the princess from The Princess and
The Pea is Sphinx. So careful must we always be.

Fixes: https://tracker.ceph.com/issues/47265
Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 29e8cbc49b3d904e2cf5d5eb2d632f49ba604052)

4 years agopython-common: s/unicode/str/
Kefu Chai [Tue, 1 Sep 2020 13:25:53 +0000 (21:25 +0800)]
python-common: s/unicode/str/

we've dropped python2 support. so it's safe to replace unicde with str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b9492d46d8460d6818e13d730f6a0bc3a0e4bdaa)

4 years agomgr/orchestrator: add test command testcase for "device ls"
Paul Cuzner [Thu, 3 Sep 2020 02:53:22 +0000 (14:53 +1200)]
mgr/orchestrator: add test command testcase for "device ls"

Quick test to ensure changes that break wide mode are picked
up

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 8691f94104bc90b7b19faaa334377c6f03eeeb83)

4 years agomgr/orchestrator: fix typing on lsm_data components
Paul Cuzner [Tue, 1 Sep 2020 04:39:35 +0000 (16:39 +1200)]
mgr/orchestrator: fix typing on lsm_data components

Resolves mypy issues

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 3a2430fed9846251926a179989829812f9d74d99)

4 years agoorchestrator: intergrate LSM information with device ls
Paul Cuzner [Mon, 31 Aug 2020 02:20:15 +0000 (14:20 +1200)]
orchestrator: intergrate LSM information with device ls

libstoragemgmt metatdata is now available with ceph-volume,
so this patch reformats the plain text output mode of the
orch device ls command to include device health and LED
states.

In addition ; the headings have been camel cased for readability,
the device size is given in dec (since you buy drives in GB not GiB),
a new wide mode has been added and the default output focuses
on the most frequently referred to fields to keep it brief.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit dea7ac0fbfc0330964868c4d5c838b9fd34f5ddd)

4 years agocephadm: Change service type to forking via systemd
Adam King [Tue, 25 Aug 2020 16:34:12 +0000 (12:34 -0400)]
cephadm: Change service type to forking via systemd

Using Type=Forking and setting the PIDFile=
directive is the supported configuration for
running podman containers with systemd as it
more accuractely models what is actually
happening. Type=Simple assumes podman is the
direct parent of the container processes which
is not true.

Fixes: https://tracker.ceph.com/issues/46654
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e6792f306ab4d07251588fdca6ed3876ae3a092a)

4 years agoMerge pull request #37308 from rhcs-dashboard/wip-47349-octopus
Lenz Grimmer [Tue, 6 Oct 2020 09:09:05 +0000 (11:09 +0200)]
Merge pull request #37308 from rhcs-dashboard/wip-47349-octopus

octopus: mgr/dashboard: REST API returns 500 when no Content-Type is specified

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
4 years agoMerge pull request #37493 from votdev/wip-47695-octopus
Lenz Grimmer [Tue, 6 Oct 2020 09:00:52 +0000 (11:00 +0200)]
Merge pull request #37493 from votdev/wip-47695-octopus

octopus: mgr/dashboard: Copy to clipboard does not work in Firefox

4 years agoMerge pull request #36862 from tspmelo/wip-46091-octopus
Lenz Grimmer [Tue, 6 Oct 2020 08:18:41 +0000 (10:18 +0200)]
Merge pull request #36862 from tspmelo/wip-46091-octopus

octopus: mgr/dashboard: Auto close table column dropdown on click outside

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #37549 from smithfarm/wip-preextended-release-note-octopus
Nathan Cutler [Tue, 6 Oct 2020 08:01:28 +0000 (10:01 +0200)]
Merge pull request #37549 from smithfarm/wip-preextended-release-note-octopus

octopus: doc/PendingReleaseNotes: mention bluefs_preextend_wal_files

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37034 from smithfarm/wip-46520-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:32:48 +0000 (16:32 -0700)]
Merge pull request #37034 from smithfarm/wip-46520-octopus

octopus: mds: do not submit omap_rm_keys if the dir is the basedir of merge

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37020 from Vicente-Cheng/wip-46522-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:32:10 +0000 (16:32 -0700)]
Merge pull request #37020 from Vicente-Cheng/wip-46522-octopus

octopus: mds: fix hang issue when accessing a file under a lost parent directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37017 from Vicente-Cheng/wip-46516-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:31:33 +0000 (16:31 -0700)]
Merge pull request #37017 from Vicente-Cheng/wip-46516-octopus

octopus: cephfs: client: fix directory inode can not call release callback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge pull request #36813 from smithfarm/wip-47080-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:30:50 +0000 (16:30 -0700)]
Merge pull request #36813 from smithfarm/wip-47080-octopus

octopus: messages,mds: Fix decoding of enum types on big-endian systems

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36713 from kotreshhr/wip-46947-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:30:17 +0000 (16:30 -0700)]
Merge pull request #36713 from kotreshhr/wip-46947-octopus

octopus: qa: Fix traceback during fs cleanup between tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agodoc/PendingReleaseNotes: mention bluefs_preextend_wal_files 37549/head
Nathan Cutler [Mon, 5 Oct 2020 15:16:52 +0000 (17:16 +0200)]
doc/PendingReleaseNotes: mention bluefs_preextend_wal_files

Signed-off-by: Nathan Cutler <ncutler@suse.com>
4 years agoMerge pull request #37452 from callithea/wip-47606-octopus
Lenz Grimmer [Mon, 5 Oct 2020 07:54:27 +0000 (09:54 +0200)]
Merge pull request #37452 from callithea/wip-47606-octopus

octopus: mgr/dashboard: log in non-admin users successfully if the telemetry notification is shown

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
4 years agomgr/devicehealth: device_health_metrics pool gets created even without any OSDs in... 37533/head
Sunny Kumar [Thu, 10 Sep 2020 13:13:27 +0000 (14:13 +0100)]
mgr/devicehealth: device_health_metrics pool gets created even without any OSDs in the cluster

This patch checks if the cluster has any OSD before the 'device_health_metrics'
pool creation by devicehealth, which prevents the cluster entering HEALTH_WARN
immediately after bootstrap.
   
Fixes: https://tracker.ceph.com/issues/46743
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit 9041adae9e7ce9e6c1855818079b128b6af267ba)

4 years agoMerge pull request #37217 from callithea/wip-47462-octopus
Yuri Weinstein [Sat, 3 Oct 2020 16:06:18 +0000 (09:06 -0700)]
Merge pull request #37217 from callithea/wip-47462-octopus

octopus: mgr: PyModuleRegistry::unregister_client() can run endlessly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge pull request #37022 from Vicente-Cheng/wip-46524-octopus
Yuri Weinstein [Sat, 3 Oct 2020 16:05:33 +0000 (09:05 -0700)]
Merge pull request #37022 from Vicente-Cheng/wip-46524-octopus

octopus: mds: add request to batch_op before taking auth pins and locks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36970 from Vicente-Cheng/wip-46473-octopus
Yuri Weinstein [Sat, 3 Oct 2020 15:58:39 +0000 (08:58 -0700)]
Merge pull request #36970 from Vicente-Cheng/wip-46473-octopus

octopus: mds: make threshold for MDS_TRIM configurable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37401 from smithfarm/wip-47644-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:29:14 +0000 (11:29 -0700)]
Merge pull request #37401 from smithfarm/wip-47644-octopus

octopus: rpm,deb: drop /etc/sudoers.d/cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37349 from smithfarm/wip-47346-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:26:47 +0000 (11:26 -0700)]
Merge pull request #37349 from smithfarm/wip-47346-octopus

octopus: qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37348 from smithfarm/wip-47251-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:25:31 +0000 (11:25 -0700)]
Merge pull request #37348 from smithfarm/wip-47251-octopus

octopus: tools/osdmaptool.cc: add ability to clean_temps

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37341 from smithfarm/wip-47466-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:25:03 +0000 (11:25 -0700)]
Merge pull request #37341 from smithfarm/wip-47466-octopus

octopus: common/admin_socket: always validate the parameters

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37302 from bk201/wip-47192-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:23:40 +0000 (11:23 -0700)]
Merge pull request #37302 from bk201/wip-47192-octopus

octopus: mgr/telemetry: fix device id splitting when anonymizing serial

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
4 years agoMerge pull request #37264 from ifed01/wip-ifed-finer-mempool-granularity-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:21:57 +0000 (11:21 -0700)]
Merge pull request #37264 from ifed01/wip-ifed-finer-mempool-granularity-octopus

octopus: bluestore: mempool's finer granularity + adding missed structs

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37257 from smithfarm/wip-47545-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:20:58 +0000 (11:20 -0700)]
Merge pull request #37257 from smithfarm/wip-47545-octopus

octopus: ceph.spec, debian: add smartmontools, nvme-cli dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37253 from bstillwell/wip-47424-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:20:16 +0000 (11:20 -0700)]
Merge pull request #37253 from bstillwell/wip-47424-octopus

octopus: compressor: Add a config option to specify Zstd compression level

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37412 from jan--f/wip-47649-octopus
Jan Fajerski [Fri, 2 Oct 2020 07:18:58 +0000 (09:18 +0200)]
Merge pull request #37412 from jan--f/wip-47649-octopus

octopus: [ceph-volume]: remove unneeded call to get_devices()

4 years agoMerge pull request #37344 from jan--f/wip-47284-octopus
Jan Fajerski [Fri, 2 Oct 2020 07:18:20 +0000 (09:18 +0200)]
Merge pull request #37344 from jan--f/wip-47284-octopus

octopus: ceph-volume: fix journal size argument not work

4 years agoMerge pull request #36921 from tspmelo/wip-npm-octopus
Lenz Grimmer [Wed, 30 Sep 2020 15:09:13 +0000 (17:09 +0200)]
Merge pull request #36921 from tspmelo/wip-npm-octopus

octopus: mgr/dashboard: Fix npm package's vulnerabilities

Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #37390 from tspmelo/wip-46999-octopus
Lenz Grimmer [Wed, 30 Sep 2020 12:00:33 +0000 (14:00 +0200)]
Merge pull request #37390 from tspmelo/wip-46999-octopus

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agomgr/dashboard: Copy to clipboard does not work in Firefox 37493/head
Volker Theile [Tue, 22 Sep 2020 12:09:52 +0000 (14:09 +0200)]
mgr/dashboard: Copy to clipboard does not work in Firefox

Fixes: https://tracker.ceph.com/issues/47578
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 8b9d881a63aac8ed3d7d1a7fc80be5e3d3d0b3b1)

Conflicts:
  src/pybind/mgr/dashboard/frontend/src/app/shared/directives/copy2clipboard-button.directive.spec.ts

4 years agoMerge pull request #37277 from tspmelo/wip-47559-octopus
Lenz Grimmer [Wed, 30 Sep 2020 09:50:58 +0000 (11:50 +0200)]
Merge pull request #37277 from tspmelo/wip-47559-octopus

octopus: mgr/dashboard: Allow editing iSCSI targets with initiators logged-in

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>