]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agomgr/cephadm: make j2 teamplates overwritable
Sebastian Wagner [Fri, 11 Sep 2020 13:04:49 +0000 (15:04 +0200)]
mgr/cephadm: make j2 teamplates overwritable

This adds an escape hatch, if the configs generated
by cephadm are not flexible enough.

Downside is obviously that future upgrades might
break the daemons. Use at your own risk.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit cfc09e4d7903c404901344b4a7bdd6c5499603fb)

4 years agocephadm: remove z option from iSCSI dev and configfs mount
Adam King [Fri, 4 Sep 2020 12:06:37 +0000 (08:06 -0400)]
cephadm: remove z option from iSCSI dev and configfs mount

selinux complains if you try to deploy iSCSI with the z option
on the /dev and /sys/kernel/config mounts

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 351b92acdacd2bea0c7c0e7c08b0b82f277621f8)

4 years agomgr/cephadm: Call cephadm with --container-image
Sebastian Wagner [Wed, 26 Aug 2020 12:53:09 +0000 (14:53 +0200)]
mgr/cephadm: Call cephadm with --container-image

The kernel treats any process with PID 1 different. Especially
it does not generate a core dump. Call podman / docker with
--init in order to get core dumps.

In addition, we can now properly reap zombies processes.

Fixes: https://tracker.ceph.com/issues/44231
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 533931505272d215a1b4fa40b3bf235cf2110102)

4 years agocephadm: Add --container-image
Sebastian Wagner [Wed, 26 Aug 2020 12:45:34 +0000 (14:45 +0200)]
cephadm: Add --container-image

The kernel treats any process with PID 1 different. Especially
it does not generate a core dump. Call podman / docker with
--init in order to get core dumps.

In addition, we can now properly reap zombies processes.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 786e1c175b41d07d8eb3d86156d77e7769ae92ac)

4 years agomgr/cephadm: introduce `CephService` class
Michael Fritch [Tue, 1 Sep 2020 20:39:17 +0000 (14:39 -0600)]
mgr/cephadm: introduce `CephService` class

- split the cephx related keyring functions into a `CephService` class
- move _get_config_and_keyring and get_keyring_entity to this new class
- monitoring stack (prom, grafana, etc) have no use for cephx keyrings

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit c133705892cce3bd9ccd981e8080511bdbc12a0e)

4 years agomgr/cephadm: refactor `get_auth_entity()`
Michael Fritch [Tue, 1 Sep 2020 19:32:44 +0000 (13:32 -0600)]
mgr/cephadm: refactor `get_auth_entity()`

- convert to f-strings
- mgr keyring format is same as osd, mds
- crash client keyring is same as nfs, iscsi, etc

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 9d2031f01de4542f9a03f1234d5794395ae9162b)

4 years agomgr/cephadm: use `get_auth_entity()`
Michael Fritch [Tue, 1 Sep 2020 19:15:49 +0000 (13:15 -0600)]
mgr/cephadm: use `get_auth_entity()`

- some services have hard-coded auth entity names
- some services are incorrectly using `utils.name_to_config_section()`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b41bb98a98f853b32b3cdaf56d0c40a3229d2854)

4 years agomgr/cephadm: move `name_to_auth_entity` to `get_auth_entity`
Michael Fritch [Tue, 1 Sep 2020 18:32:04 +0000 (12:32 -0600)]
mgr/cephadm: move `name_to_auth_entity` to `get_auth_entity`

refactor the logic for getting the keyring auth entity from the utils
module into the CephadmService class

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 5e4140944f23f5cf90ee2dab91857f7b6f7a8bfc)

4 years agocephadm: fix iSCSI unit.run file
Adam King [Thu, 3 Sep 2020 16:30:33 +0000 (12:30 -0400)]
cephadm: fix iSCSI unit.run file

Current unit.run file generated for iSCSI when using podman
tries to have two containers use the same conmon pidfile and
cidfile for both containers which is invalid
Fixes: https://tracker.ceph.com/issues/47291
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0d993640af3f3ff9572e62af9811d94a992df0dd)

4 years agocephadm: Remove assignment to member variable in ServiceSpecs
Volker Theile [Wed, 9 Sep 2020 16:08:17 +0000 (18:08 +0200)]
cephadm: Remove assignment to member variable in ServiceSpecs

Remove unnecessary assignment to member variable 'preview_only', this is done in the constructor of the derived ServiceSpec class.

Fixes: https://tracker.ceph.com/issues/47384
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 6443ac5efa9816da580fe8546942af0cdf4ed271)

4 years agocephadm: unit.run: create /var/run/ceph/$FSID before doing anything else
Tim Serong [Tue, 8 Sep 2020 10:50:35 +0000 (20:50 +1000)]
cephadm: unit.run: create /var/run/ceph/$FSID before doing anything else

This ensures the /var/run/ceph/$FSID directory exists before any
other commands are run.

Fixes: https://tracker.ceph.com/issues/47360
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 721b4de3490c739635d8cf6623f36d142acb8fac)

4 years agocephadm: log to file
Juan Miguel Olmo Martínez [Mon, 24 Aug 2020 11:05:31 +0000 (13:05 +0200)]
cephadm: log to file

Log configuration improved adding a log file with debug level.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 81a7df0498dc649c68971ca31b9ae60874f61133)

4 years agopython-common: Make HostPlacementSpec.to_json() more friendly
Sebastian Wagner [Mon, 31 Aug 2020 09:12:47 +0000 (11:12 +0200)]
python-common: Make HostPlacementSpec.to_json() more friendly

We will now generate

```yaml
placement:
  hosts:
  - ceph-001
```

instead of

```yaml
placement:
  hosts:
  - hostname: ceph-001
    name: ''
    network: ''
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 642a6f13130f5ae154679188233c1e03fdca5e38)

4 years agomgr/cephadm: upgrade: Make use of repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 13:51:53 +0000 (15:51 +0200)]
mgr/cephadm: upgrade: Make use of repo_digest

* Automatically convert tags like `:latest` to the digest
* Use the digest instead of the tag

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 303bbdfe3d42db390c936334f3bde9f7cd7baac5)

4 years agoqa/cephadm: Add mgr/cephadm/use_repo_digest true to upgrade test
Sebastian Wagner [Mon, 3 Aug 2020 12:54:22 +0000 (14:54 +0200)]
qa/cephadm: Add mgr/cephadm/use_repo_digest true to upgrade test

As this is the most interesting test suite

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d390f4623530300cf4d8b12569eb6a98797d074b)

4 years agomgr/cephadm: convert tags to repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 12:36:56 +0000 (14:36 +0200)]
mgr/cephadm: convert tags to repo_digest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 19e23bef288de590d47bb4147603b422aa96ef06)

4 years agomgr/cephadm: extract set_continer_image(entity, image)
Sebastian Wagner [Fri, 7 Aug 2020 14:51:26 +0000 (16:51 +0200)]
mgr/cephadm: extract set_continer_image(entity, image)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8da1f38e92c3fc8d15d2a7d364b7455a26d9fe9b)

4 years agocephadm: inspect_image also returns repo_digest
Sebastian Wagner [Mon, 3 Aug 2020 12:00:47 +0000 (14:00 +0200)]
cephadm: inspect_image also returns repo_digest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 30d8787c577727a3146c7ca32488936fc972c802)

Conflicts:
src/pybind/mgr/cephadm/module.py

4 years agodoc/cephadm: managing ceph.conf
Zac Dover [Wed, 2 Sep 2020 09:19:40 +0000 (19:19 +1000)]
doc/cephadm: managing ceph.conf

This commit repairs PR#36796, which was about
updating the configuration settings in ceph.conf
without touching the file directly.

That commit wasn't building because of a couple of
back quotes that were touching the word that followed
them. So much like the princess from The Princess and
The Pea is Sphinx. So careful must we always be.

Fixes: https://tracker.ceph.com/issues/47265
Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 29e8cbc49b3d904e2cf5d5eb2d632f49ba604052)

4 years agopython-common: s/unicode/str/
Kefu Chai [Tue, 1 Sep 2020 13:25:53 +0000 (21:25 +0800)]
python-common: s/unicode/str/

we've dropped python2 support. so it's safe to replace unicde with str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b9492d46d8460d6818e13d730f6a0bc3a0e4bdaa)

4 years agomgr/orchestrator: add test command testcase for "device ls"
Paul Cuzner [Thu, 3 Sep 2020 02:53:22 +0000 (14:53 +1200)]
mgr/orchestrator: add test command testcase for "device ls"

Quick test to ensure changes that break wide mode are picked
up

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 8691f94104bc90b7b19faaa334377c6f03eeeb83)

4 years agomgr/orchestrator: fix typing on lsm_data components
Paul Cuzner [Tue, 1 Sep 2020 04:39:35 +0000 (16:39 +1200)]
mgr/orchestrator: fix typing on lsm_data components

Resolves mypy issues

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 3a2430fed9846251926a179989829812f9d74d99)

4 years agoorchestrator: intergrate LSM information with device ls
Paul Cuzner [Mon, 31 Aug 2020 02:20:15 +0000 (14:20 +1200)]
orchestrator: intergrate LSM information with device ls

libstoragemgmt metatdata is now available with ceph-volume,
so this patch reformats the plain text output mode of the
orch device ls command to include device health and LED
states.

In addition ; the headings have been camel cased for readability,
the device size is given in dec (since you buy drives in GB not GiB),
a new wide mode has been added and the default output focuses
on the most frequently referred to fields to keep it brief.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit dea7ac0fbfc0330964868c4d5c838b9fd34f5ddd)

4 years agocephadm: Change service type to forking via systemd
Adam King [Tue, 25 Aug 2020 16:34:12 +0000 (12:34 -0400)]
cephadm: Change service type to forking via systemd

Using Type=Forking and setting the PIDFile=
directive is the supported configuration for
running podman containers with systemd as it
more accuractely models what is actually
happening. Type=Simple assumes podman is the
direct parent of the container processes which
is not true.

Fixes: https://tracker.ceph.com/issues/46654
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e6792f306ab4d07251588fdca6ed3876ae3a092a)

4 years agoMerge pull request #36862 from tspmelo/wip-46091-octopus
Lenz Grimmer [Tue, 6 Oct 2020 08:18:41 +0000 (10:18 +0200)]
Merge pull request #36862 from tspmelo/wip-46091-octopus

octopus: mgr/dashboard: Auto close table column dropdown on click outside

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #37549 from smithfarm/wip-preextended-release-note-octopus
Nathan Cutler [Tue, 6 Oct 2020 08:01:28 +0000 (10:01 +0200)]
Merge pull request #37549 from smithfarm/wip-preextended-release-note-octopus

octopus: doc/PendingReleaseNotes: mention bluefs_preextend_wal_files

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37034 from smithfarm/wip-46520-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:32:48 +0000 (16:32 -0700)]
Merge pull request #37034 from smithfarm/wip-46520-octopus

octopus: mds: do not submit omap_rm_keys if the dir is the basedir of merge

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37020 from Vicente-Cheng/wip-46522-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:32:10 +0000 (16:32 -0700)]
Merge pull request #37020 from Vicente-Cheng/wip-46522-octopus

octopus: mds: fix hang issue when accessing a file under a lost parent directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37017 from Vicente-Cheng/wip-46516-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:31:33 +0000 (16:31 -0700)]
Merge pull request #37017 from Vicente-Cheng/wip-46516-octopus

octopus: cephfs: client: fix directory inode can not call release callback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge pull request #36813 from smithfarm/wip-47080-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:30:50 +0000 (16:30 -0700)]
Merge pull request #36813 from smithfarm/wip-47080-octopus

octopus: messages,mds: Fix decoding of enum types on big-endian systems

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36713 from kotreshhr/wip-46947-octopus
Yuri Weinstein [Mon, 5 Oct 2020 23:30:17 +0000 (16:30 -0700)]
Merge pull request #36713 from kotreshhr/wip-46947-octopus

octopus: qa: Fix traceback during fs cleanup between tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agodoc/PendingReleaseNotes: mention bluefs_preextend_wal_files 37549/head
Nathan Cutler [Mon, 5 Oct 2020 15:16:52 +0000 (17:16 +0200)]
doc/PendingReleaseNotes: mention bluefs_preextend_wal_files

Signed-off-by: Nathan Cutler <ncutler@suse.com>
4 years agoMerge pull request #37452 from callithea/wip-47606-octopus
Lenz Grimmer [Mon, 5 Oct 2020 07:54:27 +0000 (09:54 +0200)]
Merge pull request #37452 from callithea/wip-47606-octopus

octopus: mgr/dashboard: log in non-admin users successfully if the telemetry notification is shown

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
4 years agoMerge pull request #37217 from callithea/wip-47462-octopus
Yuri Weinstein [Sat, 3 Oct 2020 16:06:18 +0000 (09:06 -0700)]
Merge pull request #37217 from callithea/wip-47462-octopus

octopus: mgr: PyModuleRegistry::unregister_client() can run endlessly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge pull request #37022 from Vicente-Cheng/wip-46524-octopus
Yuri Weinstein [Sat, 3 Oct 2020 16:05:33 +0000 (09:05 -0700)]
Merge pull request #37022 from Vicente-Cheng/wip-46524-octopus

octopus: mds: add request to batch_op before taking auth pins and locks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36970 from Vicente-Cheng/wip-46473-octopus
Yuri Weinstein [Sat, 3 Oct 2020 15:58:39 +0000 (08:58 -0700)]
Merge pull request #36970 from Vicente-Cheng/wip-46473-octopus

octopus: mds: make threshold for MDS_TRIM configurable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37401 from smithfarm/wip-47644-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:29:14 +0000 (11:29 -0700)]
Merge pull request #37401 from smithfarm/wip-47644-octopus

octopus: rpm,deb: drop /etc/sudoers.d/cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37349 from smithfarm/wip-47346-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:26:47 +0000 (11:26 -0700)]
Merge pull request #37349 from smithfarm/wip-47346-octopus

octopus: qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37348 from smithfarm/wip-47251-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:25:31 +0000 (11:25 -0700)]
Merge pull request #37348 from smithfarm/wip-47251-octopus

octopus: tools/osdmaptool.cc: add ability to clean_temps

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37341 from smithfarm/wip-47466-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:25:03 +0000 (11:25 -0700)]
Merge pull request #37341 from smithfarm/wip-47466-octopus

octopus: common/admin_socket: always validate the parameters

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37302 from bk201/wip-47192-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:23:40 +0000 (11:23 -0700)]
Merge pull request #37302 from bk201/wip-47192-octopus

octopus: mgr/telemetry: fix device id splitting when anonymizing serial

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
4 years agoMerge pull request #37264 from ifed01/wip-ifed-finer-mempool-granularity-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:21:57 +0000 (11:21 -0700)]
Merge pull request #37264 from ifed01/wip-ifed-finer-mempool-granularity-octopus

octopus: bluestore: mempool's finer granularity + adding missed structs

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37257 from smithfarm/wip-47545-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:20:58 +0000 (11:20 -0700)]
Merge pull request #37257 from smithfarm/wip-47545-octopus

octopus: ceph.spec, debian: add smartmontools, nvme-cli dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37253 from bstillwell/wip-47424-octopus
Yuri Weinstein [Fri, 2 Oct 2020 18:20:16 +0000 (11:20 -0700)]
Merge pull request #37253 from bstillwell/wip-47424-octopus

octopus: compressor: Add a config option to specify Zstd compression level

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37412 from jan--f/wip-47649-octopus
Jan Fajerski [Fri, 2 Oct 2020 07:18:58 +0000 (09:18 +0200)]
Merge pull request #37412 from jan--f/wip-47649-octopus

octopus: [ceph-volume]: remove unneeded call to get_devices()

4 years agoMerge pull request #37344 from jan--f/wip-47284-octopus
Jan Fajerski [Fri, 2 Oct 2020 07:18:20 +0000 (09:18 +0200)]
Merge pull request #37344 from jan--f/wip-47284-octopus

octopus: ceph-volume: fix journal size argument not work

4 years agoMerge pull request #36921 from tspmelo/wip-npm-octopus
Lenz Grimmer [Wed, 30 Sep 2020 15:09:13 +0000 (17:09 +0200)]
Merge pull request #36921 from tspmelo/wip-npm-octopus

octopus: mgr/dashboard: Fix npm package's vulnerabilities

Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #37390 from tspmelo/wip-46999-octopus
Lenz Grimmer [Wed, 30 Sep 2020 12:00:33 +0000 (14:00 +0200)]
Merge pull request #37390 from tspmelo/wip-46999-octopus

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #37277 from tspmelo/wip-47559-octopus
Lenz Grimmer [Wed, 30 Sep 2020 09:50:58 +0000 (11:50 +0200)]
Merge pull request #37277 from tspmelo/wip-47559-octopus

octopus: mgr/dashboard: Allow editing iSCSI targets with initiators logged-in

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
4 years agomgr/dashboard: check config opt permissions 37452/head
Tatjana Dehler [Mon, 7 Sep 2020 14:56:10 +0000 (16:56 +0200)]
mgr/dashboard: check config opt permissions

Getting the user object fails for a non-admin user. Check
the permissions directory if the user is allowed to access the
config options instead.

Fixes: https://tracker.ceph.com/issues/47331
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 0942b022d0515d53e2e8742851b6aeb60f3379f9)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/components/telemetry-notification/telemetry-notification.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/components/telemetry-notification/telemetry-notification.component.ts
- Resolved import conflicts in both files; Resolved TestBed conflict (we have to stick with TestBed.get(); in octopus vs. master
  TestBed.inject(); which is related to Angular 9 being used in master and is not backported to octopus

4 years agoMerge pull request #37270 from badone/wip-octopus-enable-mgr-client-debug
Yuri Weinstein [Tue, 29 Sep 2020 21:52:15 +0000 (14:52 -0700)]
Merge pull request #37270 from badone/wip-octopus-enable-mgr-client-debug

octopus: qa: Enable debug_client for mgr tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoMerge pull request #37422 from tchaikov/octopus-37388
Yuri Weinstein [Tue, 29 Sep 2020 21:51:13 +0000 (14:51 -0700)]
Merge pull request #37422 from tchaikov/octopus-37388

octopus: debian: Add missing Python dependency for ceph-mgr

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37373 from smithfarm/wip-46350-octopus
Yuri Weinstein [Tue, 29 Sep 2020 21:50:31 +0000 (14:50 -0700)]
Merge pull request #37373 from smithfarm/wip-46350-octopus

octopus: bluestore: remove preextended WAL support.

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37347 from smithfarm/wip-47091-octopus
Yuri Weinstein [Tue, 29 Sep 2020 21:49:45 +0000 (14:49 -0700)]
Merge pull request #37347 from smithfarm/wip-47091-octopus

octopus: mon: mark pgtemp messages as no_reply more consistenly in preprocess_…

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37032 from smithfarm/wip-47351-octopus
Yuri Weinstein [Tue, 29 Sep 2020 21:48:40 +0000 (14:48 -0700)]
Merge pull request #37032 from smithfarm/wip-47351-octopus

octopus: core: include/encoding: Fix encode/decode of float types on big-endian systems

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36945 from vumrao/wip-vumrao-47258
Yuri Weinstein [Tue, 29 Sep 2020 21:46:54 +0000 (14:46 -0700)]
Merge pull request #36945 from vumrao/wip-vumrao-47258

octopus: mon/PGMap: add pg count for pools in the ceph df command

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36047 from smithfarm/wip-46479-octopus
Yuri Weinstein [Tue, 29 Sep 2020 21:45:49 +0000 (14:45 -0700)]
Merge pull request #36047 from smithfarm/wip-46479-octopus

octopus: mds: send scrub status to ceph-mgr only when scrub is running

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37346 from smithfarm/wip-47460-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:15:44 +0000 (09:15 -0700)]
Merge pull request #37346 from smithfarm/wip-47460-octopus

octopus: qa/workunits/mon: fixed excessively large pool PG count

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37343 from smithfarm/wip-47416-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:14:52 +0000 (09:14 -0700)]
Merge pull request #37343 from smithfarm/wip-47416-octopus

octopus: rbd: include RADOS namespace in krbd symlinks

4 years agoMerge pull request #37342 from smithfarm/wip-47082-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:14:31 +0000 (09:14 -0700)]
Merge pull request #37342 from smithfarm/wip-47082-octopus

octopus: rbd-mirror: peer setup can still race and fail creation of peer

4 years agoMerge pull request #37340 from smithfarm/wip-47412-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:13:48 +0000 (09:13 -0700)]
Merge pull request #37340 from smithfarm/wip-47412-octopus

octopus: rgw: urlencode bucket name when forwarding request

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #37339 from smithfarm/wip-47348-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:12:57 +0000 (09:12 -0700)]
Merge pull request #37339 from smithfarm/wip-47348-octopus

octopus: rgw: Swift API anonymous access should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #37338 from smithfarm/wip-47321-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:12:13 +0000 (09:12 -0700)]
Merge pull request #37338 from smithfarm/wip-47321-octopus

octopus: rgw: replace '+' with "%20" in canonical query string for s3 v4 auth.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37336 from smithfarm/wip-47314-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:11:40 +0000 (09:11 -0700)]
Merge pull request #37336 from smithfarm/wip-47314-octopus

octopus: rgw: radosgw-admin: period pull command is not always a raw_storage_op

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37335 from smithfarm/wip-47245-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:10:57 +0000 (09:10 -0700)]
Merge pull request #37335 from smithfarm/wip-47245-octopus

octopus: rgw: Add bucket name to bucket stats error logging

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37208 from smithfarm/wip-46982-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:10:16 +0000 (09:10 -0700)]
Merge pull request #37208 from smithfarm/wip-46982-octopus

octopus: test/rbd-mirror: pool watcher registration error might result in race

4 years agoMerge pull request #37180 from callithea/wip-47410-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:09:43 +0000 (09:09 -0700)]
Merge pull request #37180 from callithea/wip-47410-octopus

octopus: mgr: don't update pending service map epoch on receiving map from mon

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37116 from dillaman/wip-47371-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:05:15 +0000 (09:05 -0700)]
Merge pull request #37116 from dillaman/wip-47371-octopus

octopus: librbd: ensure that thread pool lock is held when processing throttled IOs

4 years agoMerge pull request #36860 from dillaman/wip-46791-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:03:35 +0000 (09:03 -0700)]
Merge pull request #36860 from dillaman/wip-46791-octopus

octopus: librbd: handle DNE from immutable-object-cache

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
4 years agoMerge pull request #36691 from smithfarm/wip-46998-octopus
Yuri Weinstein [Tue, 29 Sep 2020 16:02:58 +0000 (09:02 -0700)]
Merge pull request #36691 from smithfarm/wip-46998-octopus

octopus: rgw: s3: mark bucket encryption as not implemented

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37405 from rhcs-dashboard/wip-47619-octopus
Lenz Grimmer [Tue, 29 Sep 2020 11:53:39 +0000 (13:53 +0200)]
Merge pull request #37405 from rhcs-dashboard/wip-47619-octopus

octopus: mgr/dashboard: fix perf. issue when listing large amounts of buckets

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #37299 from p-se/wip-47547-octopus
Laura Paduano [Mon, 28 Sep 2020 13:26:56 +0000 (15:26 +0200)]
Merge pull request #37299 from p-se/wip-47547-octopus

octopus: mgr/dashboard: Fix many-to-many issue in host-details Grafana dashboard

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
4 years agoos/bluestore: bind attrs to bluestore_cache_meta mempool. 37264/head
Igor Fedotov [Sun, 20 Sep 2020 10:45:28 +0000 (13:45 +0300)]
os/bluestore: bind attrs to bluestore_cache_meta mempool.

This seems to be a better choice over bluestore_cache_data

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f4e09fc103454fa8a41b5ffb822639368a89522d)

 Conflicts:
src/os/bluestore/bluestore_types.h
 Caused by the lack of explicit std:: reference in headers - master has
 got this as a part of crimson effort.

4 years agoos/bluestore: add missing structs to bluestore_cache_other mempool
Igor Fedotov [Sun, 20 Sep 2020 09:22:05 +0000 (12:22 +0300)]
os/bluestore: add missing structs to bluestore_cache_other mempool

Fixes: https://tracker.ceph.com/issues/47551
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 4592c5df554190065fa9a994f7b6dc3fed7ea214)

4 years agoMerge pull request #36991 from s0nea/wip-46407-octopus
Laura Paduano [Mon, 28 Sep 2020 09:01:17 +0000 (11:01 +0200)]
Merge pull request #36991 from s0nea/wip-46407-octopus

octopus: mgr/dashboard: Fix bugs in a unit test and i18n translation

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
4 years agoMerge pull request #36875 from tspmelo/wip-46342-octopus
Laura Paduano [Mon, 28 Sep 2020 08:59:36 +0000 (10:59 +0200)]
Merge pull request #36875 from tspmelo/wip-46342-octopus

octopus: mgr/dashboard: Remove useless tab in monitoring/alerts datatable details

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
4 years agoMerge pull request #36705 from ShyamsundarR/wip-47014-octopus
Yuri Weinstein [Sun, 27 Sep 2020 22:59:43 +0000 (15:59 -0700)]
Merge pull request #36705 from ShyamsundarR/wip-47014-octopus

octopus: mon: store mon updates in ceph context for future MonMap instantiation

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #36985 from k0ste/wip-47282-octopus
Yuri Weinstein [Sun, 27 Sep 2020 15:46:54 +0000 (08:46 -0700)]
Merge pull request #36985 from k0ste/wip-47282-octopus

octopus: prometheus: Properly split the port off IPv6 addresses

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
4 years agodebian: Add missing Python dependency for ceph-mgr 37422/head
Johannes M. Scheuermann [Thu, 24 Sep 2020 10:10:13 +0000 (12:10 +0200)]
debian: Add missing Python dependency for ceph-mgr

The file mgr/volumes/fs/operations/pin_util.py imports distutils.util for using
strtobool and thus the python package is required.

Signed-off-by: Johannes M. Scheuermann <joh.scheuer@gmail.com>
(cherry picked from commit 868e708c9e6538684f9b053d44497bba543ea5eb)

4 years agoMerge pull request #37408 from idryomov/wip-krbd-read-only-override-octopus
Ilya Dryomov [Fri, 25 Sep 2020 15:37:57 +0000 (17:37 +0200)]
Merge pull request #37408 from idryomov/wip-krbd-read-only-override-octopus

octopus: rbd: make common options override krbd-specific options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoceph-volume: remove unneeded call to get_devices() 37412/head
Marc Gariepy [Tue, 22 Sep 2020 13:01:33 +0000 (09:01 -0400)]
ceph-volume: remove unneeded call to get_devices()

there is no need to probe the device to generate the argparse help
message.

also removing the test on the function as it's not there anymore.

Signed-off-by: Marc Gariepy <gariepy.marc@gmail.com>
Fixes: https://tracker.ceph.com/issues/47502
(cherry picked from commit 5c6f66166a7afad87627032cafdc5c4f11f94eac)

4 years agorbd: make common options override krbd-specific options 37408/head
Ilya Dryomov [Fri, 25 Sep 2020 07:56:29 +0000 (09:56 +0200)]
rbd: make common options override krbd-specific options

ceph-csi has added support for passing custom map and unmap options via
mapOptions and unmapOptions storage class parameters.  However, it also
uses --read-only for implementing ROX (ReadOnlyMany) PVs.  If the user
supplies "mapOptions: rw", they will get around the intended read-only
restriction (at least on the block device).

ceph-csi could be patched to use "-o ro", but it actually makes sense
for common options to win over device type-specific equivalents.

Fixes: https://tracker.ceph.com/issues/47625
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a107c47360ecdb8c09768ca9eab2341100245711)

Conflicts:
src/tools/rbd/action/Kernel.cc [ snapshot quiesce support and
  commit 34f539d8af33 ("rbd: delay parsing of default kernel map
  options") not in octopus ]

4 years agomgr/dashboard: fix performance issue when listing large amounts of buckets 37405/head
Alfonso Martínez [Fri, 18 Sep 2020 15:16:34 +0000 (17:16 +0200)]
mgr/dashboard: fix performance issue when listing large amounts of buckets

Fixes: https://tracker.ceph.com/issues/47543
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 924368e1d0aebcb0d8f9747589d9048414d33080)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-details/rgw-bucket-details.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-details/rgw-bucket-details.component.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/api/rgw-bucket.service.ts
  - Adapted changes in these files to octopus code.

4 years agoMerge pull request #36931 from yuriw/wip-yuriw-46529-octopus
Yuri Weinstein [Thu, 24 Sep 2020 23:57:40 +0000 (16:57 -0700)]
Merge pull request #36931 from yuriw/wip-yuriw-46529-octopus

octopus: qa/tests: use bionic only for old clients in rados/thrash-old-clients

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agorpm,deb: drop /etc/sudoers.d/cephadm 37401/head
Nathan Cutler [Tue, 1 Sep 2020 18:38:47 +0000 (20:38 +0200)]
rpm,deb: drop /etc/sudoers.d/cephadm

Current behavior (without this patch) is:

1. cephadm package installs cephadm at /usr/sbin/cephadm
2. cephadm package installs /etc/sudoers.d/cephadm
3. !!! BUT this file refers to a non-existent executable (/usr/bin/cephadm) !!!
4. the PR that introduced this sudoers file (and this discrepancy) was merged in 2019
5. nobody noticed the discrepancy until now

My conclusion: the file /etc/sudoers.d/cephadm is not needed for cephadm to
work.

Fixes: https://tracker.ceph.com/issues/47112
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit c1783d3d8f6fd25817b78ea6f76335347aceb566)

Conflicts:
debian/rules
- octopus does not mention "rbd-nbd_quiesce"

4 years agomgr/dashboard: Landing Page improvements 37390/head
Alfonso Martínez [Thu, 13 Aug 2020 12:29:38 +0000 (14:29 +0200)]
mgr/dashboard: Landing Page improvements

Fixes: https://tracker.ceph.com/issues/42072
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit d66e684b9ec83cca8a58b0a7b8661c568eb0cf6d)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/health-pie/health-pie.component.scss
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/health/health.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/health/health.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/info-card/info-card.component.scss
src/pybind/mgr/dashboard/frontend/src/styles/defaults/_bootstrap-defaults.scss
  this file doesn't exist in octopus, so I moved the code into:
src/pybind/mgr/dashboard/frontend/src/stykes/defaults.scss

4 years agomgr/dashboard: Clean custom header styles
Tiago Melo [Wed, 8 Jul 2020 09:40:30 +0000 (09:40 +0000)]
mgr/dashboard: Clean custom header styles

Fixes: https://tracker.ceph.com/issues/46417
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit cf55a768b26e1eded9cd2235c2966b2529a3cafc)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/auth/login/login.component.scss
src/pybind/mgr/dashboard/frontend/src/styles/ceph-custom/_basics.scss
src/pybind/mgr/dashboard/frontend/src/styles/ceph-custom/_grid.scss
src/pybind/mgr/dashboard/frontend/src/styles.scss

4 years agoos/bluestore: remove preextended WAL support. 37373/head
Igor Fedotov [Tue, 9 Jun 2020 08:44:31 +0000 (11:44 +0300)]
os/bluestore: remove preextended WAL support.

Fixes: https://tracker.ceph.com/issues/45613
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 7fdbf61860b9d9deaf8734cdd57cf5c8d5f93f81)

Conflicts:
src/common/options.cc
- option "bluefs_preextend_wal_files" has a different default value
  ("false") in octopus (but the whole option is being deleted, so it
  doesn't matter)

4 years agomgr/dashboard: Fix npm package's vulnerabilities 36921/head
Tiago Melo [Fri, 28 Aug 2020 13:59:47 +0000 (13:59 +0000)]
mgr/dashboard: Fix npm package's vulnerabilities

Manual update of some npm packages to fix package's vulnerabilities.
This could not have been done by backport since master has a different list
of packages installed.

Fixes: https://tracker.ceph.com/issues/47191
Signed-off-by: Tiago Melo <tmelo@suse.com>
4 years agoqa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down 37349/head
Neha Ojha [Fri, 4 Sep 2020 21:51:50 +0000 (21:51 +0000)]
qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

The test should mark the OSD out to check if only "in" OSDs are considered by
the osdmap trimming logic.

Fixes: https://tracker.ceph.com/issues/47309
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 21c08f0be2e048edd2d3ce7ca803f94a6d32f97c)

4 years agotest/cli/osdmaptool/help.t: add clean-temps 37348/head
Neha Ojha [Wed, 26 Aug 2020 18:40:01 +0000 (18:40 +0000)]
test/cli/osdmaptool/help.t: add clean-temps

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 762966a08ab31234148fe181eb2c107ae5266906)

4 years agodoc: add clean-temps to osdmaptool.rst
Neha Ojha [Wed, 26 Aug 2020 18:37:58 +0000 (18:37 +0000)]
doc: add clean-temps to osdmaptool.rst

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 78224c21d24d68d471eb266a37624d00d1483ed8)

4 years agotools/osdmaptool.cc: add ability to clean_temps
Neha Ojha [Wed, 26 Aug 2020 18:08:03 +0000 (18:08 +0000)]
tools/osdmaptool.cc: add ability to clean_temps

This is particularly useful for debugging purposes when clean_temps()
takes abnormally high amount of time due to flaws in crush rules etc.

Fixes: https://tracker.ceph.com/issues/47159
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ab54d3821a61a4ff1ea9973c5f31ee86868b3009)

4 years agomon: mark pgtemp messages as no_reply more consistently in preprocess_pgtemp 37347/head
Greg Farnum [Wed, 12 Aug 2020 23:44:11 +0000 (23:44 +0000)]
mon: mark pgtemp messages as no_reply more consistently in preprocess_pgtemp

If a message is forwarded, it's conceivable the leader's and peon's evaluation
will disagree about whether the message is useful or not, which could result
in the leader ignoring it and the peon having a dangling forwarded message.
Fix this by marking the op as no_reply whenever ignoring it.

Fixes: https://tracker.ceph.com/issues/46914
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 73a014fc2ca928eb72def31c9e4177063cda421a)

4 years agoqa/workunits/mon: fixed excessively large pool PG count 37346/head
Jason Dillaman [Mon, 14 Sep 2020 12:58:52 +0000 (08:58 -0400)]
qa/workunits/mon: fixed excessively large pool PG count

Fixes: https://tracker.ceph.com/issues/47405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2fa9442dcc7a0448ab7e3588f82f93ca2e55d686)

4 years agoceph-volume: fix journal size argument not work 37344/head
wanghongxu [Thu, 27 Aug 2020 07:11:28 +0000 (15:11 +0800)]
ceph-volume: fix journal size argument not work

Journal size argument not work in batch prepare
filestore with SSD journal device.

Fixes: https://tracker.ceph.com/issues/41374
Signed-off-by: wanghongxu <wanghongxu@t2cloud.net>
(cherry picked from commit a7077dc661478a762a86b88452c98b30717c0b3c)

4 years agoqa: add test for krbd symlinks created by udev 37343/head
Ilya Dryomov [Mon, 7 Sep 2020 14:51:22 +0000 (16:51 +0200)]
qa: add test for krbd symlinks created by udev

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7ccd2c0dcee175e4c5a03985f43e9259a7e4dbd4)

4 years agorbd: include RADOS namespace in krbd symlinks
Ilya Dryomov [Mon, 7 Sep 2020 16:39:22 +0000 (18:39 +0200)]
rbd: include RADOS namespace in krbd symlinks

Fixes: https://tracker.ceph.com/issues/40247
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0b5c11ff30dbb79690e47d5285f197f677e11bf7)

4 years agoqa/workunits/rbd: retrieve peer uuid when attempting to delete duplicate 37342/head
Jason Dillaman [Tue, 18 Aug 2020 21:18:25 +0000 (17:18 -0400)]
qa/workunits/rbd: retrieve peer uuid when attempting to delete duplicate

Previously, the peer uuid variable was empty which resulted in the failure
to remove the duplicate peer.

Fixes: https://tracker.ceph.com/issues/47007
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dec8292a17f4d7543a4e22c293886363c97068b5)

4 years agorbd: re-use helper method for adding mirror peer site-name optional
Jason Dillaman [Tue, 18 Aug 2020 20:29:20 +0000 (16:29 -0400)]
rbd: re-use helper method for adding mirror peer site-name optional

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af3d4cb3c4bb430e47add07b45e670737e6e77f5)