ceph-ci.git
2 years agoosd/scrub: removing PgScrubber's ref to the primary scrub map wip-ronenf-scrub-be2
Ronen Friedman [Fri, 14 Jan 2022 15:01:40 +0000 (15:01 +0000)]
osd/scrub: removing PgScrubber's ref to the primary scrub map

as caching this reference (to an object that is owned by the
scrubber backend) creates a dangling-pointer risk.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: change missing_digest fix-list type
Ronen Friedman [Thu, 13 Jan 2022 09:22:11 +0000 (09:22 +0000)]
osd/scrub: change missing_digest fix-list type

...allowing for efficient creation and, more important,
direct consumption by the function performing the
actual I/O.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: scrub components embedded documentation
Ronen Friedman [Thu, 30 Dec 2021 13:04:58 +0000 (13:04 +0000)]
osd/scrub: scrub components embedded documentation

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: add to PgScrubber a local ref to next-scrub's details
Ronen Friedman [Thu, 30 Dec 2021 13:01:24 +0000 (13:01 +0000)]
osd/scrub: add to PgScrubber a local ref to next-scrub's details

The owner is still the PG.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: minor fixes split from main "scrub backend" commit
Ronen Friedman [Wed, 1 Dec 2021 11:37:04 +0000 (11:37 +0000)]
osd/scrub: minor fixes split from main "scrub backend" commit

Reformatting, minor compiler warnings, etc.
Split from "osd/scrub: extract scrub backend code into separate classes"

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: extract scrub backend code into separate classes
Ronen Friedman [Sun, 11 Apr 2021 18:17:41 +0000 (21:17 +0300)]
osd/scrub: extract scrub backend code into separate classes

A refactoring of the scrub backend code (all Scrub related code
that checks/manipulates objects' (meta)data).

Part of the refactoring was breaking long chunks of code into
separate functions. In order to avoid passing endless lists
of state parameters from/to these new methods, the scrub backend
state is now maintained within its own objects.

a change to note:
Instead of two scrub-map collections - one that holds all
incoming maps, and one that holds pointers to all of those incoming
plus the one we (the Primary) manage - just use one collection
for both incoming and our own.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Split from "osd/scrub: minor fixes split from main "scrub backend" commit"

Split from "osd/scrub: scrub components embedded documentation"

fix

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
osd/scrub: fix the scrubber backend to include all PR #42735

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoMerge pull request #44604 from mgfritch/c-v-skip-rbd-devs
Guillaume Abrioux [Fri, 21 Jan 2022 12:37:25 +0000 (13:37 +0100)]
Merge pull request #44604 from mgfritch/c-v-skip-rbd-devs

ceph-volume: show RBD devices as not available

2 years agoMerge pull request #44687 from sebastian-philipp/gov
Sebastian Wagner [Fri, 21 Jan 2022 08:56:00 +0000 (09:56 +0100)]
Merge pull request #44687 from sebastian-philipp/gov

doc/governance: Adam King

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #44042 from myoungwon/wip-add-blk-paddr
Samuel Just [Thu, 20 Jan 2022 21:27:25 +0000 (13:27 -0800)]
Merge pull request #44042 from myoungwon/wip-add-blk-paddr

seastore: add blk_paddr_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #44685 from rhcs-dashboard/fix-53950-master
Ernesto Puerta [Thu, 20 Jan 2022 20:21:17 +0000 (21:21 +0100)]
Merge pull request #44685 from rhcs-dashboard/fix-53950-master

mgr/dashboard: remove feedback module as always-on

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #44665 from guits/cv-sw-logger
Guillaume Abrioux [Thu, 20 Jan 2022 19:21:30 +0000 (20:21 +0100)]
Merge pull request #44665 from guits/cv-sw-logger

ceph-volume: don't use MultiLogger in find_executable_on_host()

2 years agoMerge pull request #43345 from trociny/wip-osdmaptool-down
Laura Flores [Thu, 20 Jan 2022 18:56:29 +0000 (12:56 -0600)]
Merge pull request #43345 from trociny/wip-osdmaptool-down

tools/osdmaptool: fix possible segfaults when there are down osds

2 years agomgr/dashboard: fix openapi-check
Avan Thakkar [Thu, 20 Jan 2022 16:39:22 +0000 (22:09 +0530)]
mgr/dashboard: fix openapi-check

Fixes: https://tracker.ceph.com/issues/53950
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2 years agoceph-volume: filter RBD devices from the device inventory
Michael Fritch [Tue, 18 Jan 2022 22:15:45 +0000 (15:15 -0700)]
ceph-volume: filter RBD devices from the device inventory

Avoid running `blkid` or deploying OSDs on RBD devices by ensuring they
do not appear in the `ceph-volume inventory`

Fixes: https://tracker.ceph.com/issues/53846
Signed-off-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #44675 from rzarzynski/wip-tests-bl-explicit-rebuild
Radoslaw Zarzynski [Thu, 20 Jan 2022 17:12:20 +0000 (18:12 +0100)]
Merge pull request #44675 from rzarzynski/wip-tests-bl-explicit-rebuild

test/bufferlist: assert the rebuild in rebuild_aligned_size_and_memory() actually happens.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #44677 from zdover23/wip-doc_2022-01-20_weil-crush-sc06pdf-repair
zdover23 [Thu, 20 Jan 2022 15:11:59 +0000 (01:11 +1000)]
Merge pull request #44677 from zdover23/wip-doc_2022-01-20_weil-crush-sc06pdf-repair

doc/rados: update broken pdf link

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agodoc/governance: Adam King
Sebastian Wagner [Thu, 20 Jan 2022 13:02:37 +0000 (14:02 +0100)]
doc/governance: Adam King

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2 years agomgr/dashboard: remove feedback as always-on wip-epuertat-testing-2022-01-20-1324
Ernesto Puerta [Thu, 20 Jan 2022 12:11:58 +0000 (13:11 +0100)]
mgr/dashboard: remove feedback as always-on

It was mistakenly left as always-on

Fixes: https://tracker.ceph.com/issues/53950
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #44557 from kotreshhr/clone-quota-failure
Venky Shankar [Thu, 20 Jan 2022 11:12:35 +0000 (16:42 +0530)]
Merge pull request #44557 from kotreshhr/clone-quota-failure

mgr/volumes: Fix subvoume snapshot clone failure

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoceph-volume: install config_template from collection
Guillaume Abrioux [Wed, 19 Jan 2022 14:39:52 +0000 (15:39 +0100)]
ceph-volume: install config_template from collection

ceph-ansible@master uses config_template from upstream collection.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agoceph-volume: don't use MultiLogger in find_executable_on_host()
Guillaume Abrioux [Wed, 19 Jan 2022 14:04:20 +0000 (15:04 +0100)]
ceph-volume: don't use MultiLogger in find_executable_on_host()

This generates a lot of unnecessary messages on the terminal.

Fixes: https://tracker.ceph.com/issues/53934
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agodoc/rados: update broken pdf link
Zac Dover [Thu, 20 Jan 2022 01:01:42 +0000 (11:01 +1000)]
doc/rados: update broken pdf link

This PR repairs a link to a PDF. The link was broken
when the PDF assets were moved during the restructure
of the ceph.io website in 2021.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agotest/bufferlist: assert the rebuild in rebuild_aligned_size_and_memory() actually...
Radoslaw Zarzynski [Wed, 19 Jan 2022 21:56:05 +0000 (21:56 +0000)]
test/bufferlist: assert the rebuild in rebuild_aligned_size_and_memory() actually happens.

For the investigation of failures like the following one:

```
[ RUN      ] BufferList.rebuild_aligned_size_and_memory
../src/test/bufferlist.cc:1865: Failure
Expected equality of these values:
  bl.get_num_buffers()
    Which is: 2
  1
[  FAILED  ] BufferList.rebuild_aligned_size_and_memory (0 ms)
```

The test case assumes the rebuild before the failed clause **always**
happens while `bufferlist::rebuild_aligned_size_and_memory()` skips it
if buffers are already aligned.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #44615 from ceph/wip-yuriw-crontab-master
Yuri Weinstein [Wed, 19 Jan 2022 16:15:20 +0000 (08:15 -0800)]
Merge pull request #44615 from ceph/wip-yuriw-crontab-master

qa/tests: run upgrades on quincy

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #44617 from cbodley/wip-rgw-swift-bulk-upload-no-such-bucket
Casey Bodley [Wed, 19 Jan 2022 15:43:44 +0000 (10:43 -0500)]
Merge pull request #44617 from cbodley/wip-rgw-swift-bulk-upload-no-such-bucket

rgw/swift: don't crash on nonexistent bucket in BulkUpload

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #44601 from trociny/wip-53888
Ilya Dryomov [Wed, 19 Jan 2022 15:36:22 +0000 (16:36 +0100)]
Merge pull request #44601 from trociny/wip-53888

cls/journal: skip disconnected clients when calculating min_commit_position

Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #44381 from ly798/add-ns-to-snap-dump
Ilya Dryomov [Wed, 19 Jan 2022 15:31:45 +0000 (16:31 +0100)]
Merge pull request #44381 from ly798/add-ns-to-snap-dump

cls/rbd: add namespace to snapshot dump results

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #44553 from kamoltat/wip-ksirivad-progress-mgr-enforce-key-error...
Kamoltat Sirivadhna [Wed, 19 Jan 2022 15:24:07 +0000 (10:24 -0500)]
Merge pull request #44553 from kamoltat/wip-ksirivad-progress-mgr-enforce-key-error-exception

pybind/mgr/progress: enforced try and except on event dictionary
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoMerge pull request #44024 from rhcs-dashboard/fix-53334-master
Ernesto Puerta [Wed, 19 Jan 2022 13:03:42 +0000 (14:03 +0100)]
Merge pull request #44024 from rhcs-dashboard/fix-53334-master

mgr/dashboard: Improve notifications for osd nearfull, full

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #44238 from rhcs-dashboard/fix-51575-fix
Ernesto Puerta [Wed, 19 Jan 2022 12:35:03 +0000 (13:35 +0100)]
Merge pull request #44238 from rhcs-dashboard/fix-51575-fix

mgr/dashboard: Notification banners at the top of the UI have fixed height

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2 years agoMerge pull request #44332 from rhcs-dashboard/grafana-warnings
Ernesto Puerta [Wed, 19 Jan 2022 12:18:31 +0000 (13:18 +0100)]
Merge pull request #44332 from rhcs-dashboard/grafana-warnings

monitoring/grafana: Replace missing legendFormat warning with error.

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #44600 from phlogistonjohn/jjm-mirror-service-module
Sebastian Wagner [Wed, 19 Jan 2022 11:16:29 +0000 (12:16 +0100)]
Merge pull request #44600 from phlogistonjohn/jjm-mirror-service-module

mgr/cephadm: auto-enable mirroring module when deploying service

Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #44647 from melissa-kun-li/remove-duplicate-deployment-doc
Sebastian Wagner [Wed, 19 Jan 2022 11:14:10 +0000 (12:14 +0100)]
Merge pull request #44647 from melissa-kun-li/remove-duplicate-deployment-doc

doc/cephadm: remove duplicate deployment scenario section

Reviewed-by: Adam King <adking@redhat.com>
2 years agomgr/dashboard: Improve notifications for osd nearfull, full
Aashish Sharma [Fri, 19 Nov 2021 09:02:49 +0000 (14:32 +0530)]
mgr/dashboard: Improve notifications for osd nearfull, full

This PR adds some visual hints for osds that are near full or full

Fixes: https://tracker.ceph.com/issues/53334
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #44430 from 5cs/fix-check-pg-num-underflow
Yuri Weinstein [Tue, 18 Jan 2022 22:39:11 +0000 (14:39 -0800)]
Merge pull request #44430 from 5cs/fix-check-pg-num-underflow

mon/OSDMonitor: fix integer underflow of check_pg_num

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agodoc/cephadm: remove duplicate deployment scenario section
Melissa Li [Tue, 18 Jan 2022 21:53:04 +0000 (16:53 -0500)]
doc/cephadm: remove duplicate deployment scenario section

Signed-off-by: Melissa Li <melissali@redhat.com>
2 years agoMerge pull request #43732 from pdvian/wip-limit-slowop
Yuri Weinstein [Tue, 18 Jan 2022 20:33:07 +0000 (12:33 -0800)]
Merge pull request #43732 from pdvian/wip-limit-slowop

osd/OSD: Log aggregated slow ops detail to cluster logs

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoMerge pull request #44578 from rhcs-dashboard/fix-53843-master
Ernesto Puerta [Tue, 18 Jan 2022 19:23:07 +0000 (20:23 +0100)]
Merge pull request #44578 from rhcs-dashboard/fix-53843-master

qa/dashboard: ensure node 16 is installed

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2 years agoMerge pull request #44607 from rhcs-dashboard/fix-autopep8-qa-tasks
Ernesto Puerta [Tue, 18 Jan 2022 19:11:07 +0000 (20:11 +0100)]
Merge pull request #44607 from rhcs-dashboard/fix-autopep8-qa-tasks

mgr/dashboard: include autopep8 for dashboard qa tasks

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #44575 from rhcs-dashboard/dashboard-cephadm-more-stability
Ernesto Puerta [Tue, 18 Jan 2022 19:02:25 +0000 (20:02 +0100)]
Merge pull request #44575 from rhcs-dashboard/dashboard-cephadm-more-stability

mgr/dashboard: Refactoring dashboard cephadm checks

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #42603 from cypherean/feedback_frontend
Ernesto Puerta [Tue, 18 Jan 2022 18:47:13 +0000 (19:47 +0100)]
Merge pull request #42603 from cypherean/feedback_frontend

mgr/dashboard: report ceph tracker bug/feature through GUI

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2 years agomgr/cephadm: add a test for enabling cephfs mirroring module
John Mulligan [Tue, 18 Jan 2022 18:31:03 +0000 (13:31 -0500)]
mgr/cephadm: add a test for enabling cephfs mirroring module

Add a test that checks that when cephfs mirror service is enabled
the mirroring mgr module gets enabled.

Actually-written-by: Sebastian Wagner <sewagner@redhat.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agomgr/cephadm: auto-enable mirroring module when deploying service
John Mulligan [Thu, 13 Jan 2022 19:01:51 +0000 (14:01 -0500)]
mgr/cephadm: auto-enable mirroring module when deploying service

Automatically enable the mgr's mirroring module when creating
cephfs-mirror services. This will trigger a mgr respawn.

Fixes: https://tracker.ceph.com/issues/50593
Based roughly on 50dc1d0decb2fbf7b9129bddba940969410be5cd

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #33934 from cbodley/wip-40177
Casey Bodley [Tue, 18 Jan 2022 18:22:18 +0000 (13:22 -0500)]
Merge pull request #33934 from cbodley/wip-40177

rgw: delete full sync index before switching to incremental sync

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #40011 from cbodley/wip-49723
Casey Bodley [Tue, 18 Jan 2022 18:21:06 +0000 (13:21 -0500)]
Merge pull request #40011 from cbodley/wip-49723

rgw: allow rgw_data_notify_interval_msec=0 to disable notifications

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #44581 from cbodley/wip-53177
Casey Bodley [Tue, 18 Jan 2022 18:20:15 +0000 (13:20 -0500)]
Merge pull request #44581 from cbodley/wip-53177

rgw/dbstore: hide dbstore_log.h from rgw_main.cc

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #44413 from cybozu/rgw-fix-error-code-of-remove-bucket-api
Casey Bodley [Tue, 18 Jan 2022 18:19:53 +0000 (13:19 -0500)]
Merge pull request #44413 from cybozu/rgw-fix-error-code-of-remove-bucket-api

rgw: remove bucket API returns NoSuchKey than NoSuchBucket

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #44078 from cbodley/wip-rgw-multisite-metadata-retry-error
Casey Bodley [Tue, 18 Jan 2022 18:18:46 +0000 (13:18 -0500)]
Merge pull request #44078 from cbodley/wip-rgw-multisite-metadata-retry-error

rgw/multisite: metadata sync only retries on errors

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #44602 from neha-ojha/wip-qct-remove
Neha Ojha [Tue, 18 Jan 2022 17:17:59 +0000 (09:17 -0800)]
Merge pull request #44602 from neha-ojha/wip-qct-remove

doc/foundation.rst: qct is no longer a member

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
2 years agoseastore/seastore_types: fix wrong masking
myoungwon oh [Fri, 3 Dec 2021 08:31:30 +0000 (17:31 +0900)]
seastore/seastore_types: fix wrong masking

Fix bit operation due to device_id_t is 8 bit for now

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoseastore/random_block_manager: use paddr_t instead of uint64
myoungwon oh [Wed, 24 Nov 2021 10:19:45 +0000 (19:19 +0900)]
seastore/random_block_manager: use paddr_t instead of uint64

Reagarding read() and write(), use paddr_t as input parameter
instead of uint64

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoseatore/random_block_manager: remove blocks_per_segment
myoungwon oh [Mon, 22 Nov 2021 03:30:58 +0000 (12:30 +0900)]
seatore/random_block_manager: remove blocks_per_segment

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoseastore: changes due to adding blk_paddr_t
myoungwon oh [Fri, 19 Nov 2021 07:56:16 +0000 (16:56 +0900)]
seastore: changes due to adding blk_paddr_t

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoseastore: add blk_paddr_t
myoungwon oh [Tue, 16 Nov 2021 04:23:15 +0000 (13:23 +0900)]
seastore: add blk_paddr_t

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agomgr/dashboard: Refactoring dashboard cephadm checks
Nizamudeen A [Thu, 13 Jan 2022 12:58:56 +0000 (18:28 +0530)]
mgr/dashboard: Refactoring dashboard cephadm checks

I isolated all the tests suites into there respective files
so that in future it is easier to add more tests to it.

I also given priority to the host actions.

Create OSD checks are now written in a way that OSDs
are created only on the intended hosts. This will make
the host draining process easier and less time consuming.

Also tried to address the flaky force maintenance checks.

Removed some duplicated codes

Service creation part improved to reduce the time taken
for its completion

Fixes: https://tracker.ceph.com/issues/53905
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #44505 from guits/fix-53812
Sebastian Wagner [Tue, 18 Jan 2022 15:31:49 +0000 (16:31 +0100)]
Merge pull request #44505 from guits/fix-53812

ceph-volume: fix regression introcuded via #43536

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Teoman Onay <tonay@redhat.com>
2 years agoMerge pull request #44489 from adk3798/agent-down-alerts
Sebastian Wagner [Tue, 18 Jan 2022 15:23:40 +0000 (16:23 +0100)]
Merge pull request #44489 from adk3798/agent-down-alerts

mgr/cephadm: still check agent deps if it is marked down

2 years agomonitoring/grafana: replace filestore osd count
Pere Diaz Bou [Tue, 18 Jan 2022 13:14:41 +0000 (14:14 +0100)]
monitoring/grafana: replace filestore osd count

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agomonitoring/grafana: use Path class instead of split
Pere Diaz Bou [Mon, 3 Jan 2022 11:52:09 +0000 (12:52 +0100)]
monitoring/grafana: use Path class instead of split

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agomonitoring/grafana: remove explicit str casting
Pere Diaz Bou [Mon, 3 Jan 2022 08:32:51 +0000 (09:32 +0100)]
monitoring/grafana: remove explicit str casting

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agomonitoring/grafana: add generated json files
Pere Diaz Bou [Fri, 17 Dec 2021 07:58:58 +0000 (08:58 +0100)]
monitoring/grafana: add generated json files

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agomonitoring/grafana: ValueError instead of RuntimeError
Pere Diaz Bou [Thu, 16 Dec 2021 14:31:28 +0000 (15:31 +0100)]
monitoring/grafana: ValueError instead of RuntimeError

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agomonitoring/grafana: Replace missing legendFormat warning with error
Pere Diaz Bou [Thu, 16 Dec 2021 12:34:35 +0000 (13:34 +0100)]
monitoring/grafana: Replace missing legendFormat warning with error

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoqa/cephadm: install hwe kernel only for focal
Guillaume Abrioux [Fri, 14 Jan 2022 17:20:10 +0000 (18:20 +0100)]
qa/cephadm: install hwe kernel only for focal

Let's install hwe kernel only on Ubuntu focal, otherwise we only shift the
problem on Ubuntu bionic given that the hwe kernel for bionic is 5.4.

Fixes: https://tracker.ceph.com/issues/53863
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agoseastore: rename segment_off_t to seastore_off_t
myoungwon oh [Fri, 3 Dec 2021 07:18:10 +0000 (16:18 +0900)]
seastore: rename segment_off_t to seastore_off_t

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoMerge pull request #44591 from athanatos/sjust/wip-seastore-flush
Samuel Just [Tue, 18 Jan 2022 03:36:10 +0000 (19:36 -0800)]
Merge pull request #44591 from athanatos/sjust/wip-seastore-flush

crimson/os/seastore: avoid empty Transactions by adding explicit flush() call

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #44556 from cyx1231st/wip-crimson-improve-log-journal
Samuel Just [Mon, 17 Jan 2022 21:19:02 +0000 (13:19 -0800)]
Merge pull request #44556 from cyx1231st/wip-crimson-improve-log-journal

crimson/os/seastore: consolidate seastore_journal logs with cleanup and validations

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agorgw: clean up index after full metadata sync
Casey Bodley [Thu, 12 Mar 2020 20:51:26 +0000 (16:51 -0400)]
rgw: clean up index after full metadata sync

Fixes: https://tracker.ceph.com/issues/40177
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: clean up index after full data sync
Casey Bodley [Thu, 12 Mar 2020 20:43:05 +0000 (16:43 -0400)]
rgw: clean up index after full data sync

Fixes: https://tracker.ceph.com/issues/40177
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agocrimson/os/seastore: implement FuturizedStore::flush
Samuel Just [Fri, 14 Jan 2022 06:53:17 +0000 (06:53 +0000)]
crimson/os/seastore: implement FuturizedStore::flush

Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agorgw/swift: don't crash on nonexistent bucket in BulkUpload
Casey Bodley [Mon, 17 Jan 2022 19:45:28 +0000 (14:45 -0500)]
rgw/swift: don't crash on nonexistent bucket in BulkUpload

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agomgr/dashboard: Notification banners at the top of the UI have fixed height
Waad AlKhoury [Mon, 17 Jan 2022 19:45:58 +0000 (20:45 +0100)]
mgr/dashboard: Notification banners at the top of the UI have fixed height

Fixes: https://tracker.ceph.com/issues/51575
Signed-off-by: Waad AlKhoury <walkhour@redhat.com>
2 years agoqa/rgw: run multisite tests with some async notifications disabled
Casey Bodley [Wed, 10 Mar 2021 21:24:52 +0000 (16:24 -0500)]
qa/rgw: run multisite tests with some async notifications disabled

disable the sending of async datalog notifications on one zone per
cluster. this helps to verify that tests don't rely on notifications to
succeed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: allow rgw_data_notify_interval_msec=0 to disable notifications
Casey Bodley [Wed, 10 Mar 2021 21:12:13 +0000 (16:12 -0500)]
rgw: allow rgw_data_notify_interval_msec=0 to disable notifications

the data changes log for multisite will occasionally broadcast recent
changes to other zones, which they can use to prioritize sync of some
of the most recent changes. they'll eventually see all changes as they
replay the data changes log, though, so notifications aren't required
for successful sync. the ability to turn them off is useful for testing

Fixes: https://tracker.ceph.com/issues/49723
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agocls/journal: skip disconnected clients when finding min_commit_position
Mykola Golub [Fri, 14 Jan 2022 18:21:29 +0000 (18:21 +0000)]
cls/journal: skip disconnected clients when finding min_commit_position

When a new journal client is registered, all already registered
clients are checked, and a client with min position is selected
as a position for the new client. Thus we may expect that
starting from the registered position all journal entries will be
available (not trimmed) for the new client.

But when looking for a min commit position, the client_register
function did not take into account that a registered client might
be in disconnected state, and in that case the journal entries
might be trimmed for this client.

Fixes: https://tracker.ceph.com/issues/53888
Signed-off-by: Mykola Golub <mgolub@suse.com>
2 years agoqa/tests: run upgrades on quincy
Yuri Weinstein [Mon, 17 Jan 2022 18:15:06 +0000 (10:15 -0800)]
qa/tests: run upgrades on quincy

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2 years agoqa/nvme_loop: fix an issue on ubuntu 18.04
Guillaume Abrioux [Thu, 13 Jan 2022 21:46:03 +0000 (22:46 +0100)]
qa/nvme_loop: fix an issue on ubuntu 18.04

The following command:

```
echo /dev/sda | tee /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/device_path
```

makes nvme_loop fail because fascinatingly, it adds an unexpected newline.

See:
```
/dev/sda
/dev/sda

1
tee: /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/enable: No such file or directory
/dev/sda
1
```

Other distros don't have the same behavior:

```
CentOS 8
/dev/sda
/dev/sda
1

Ubuntu 20.04
/dev/sda
/dev/sda
1
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agoceph-volume: fix regression introcuded via #43536
Guillaume Abrioux [Mon, 10 Jan 2022 09:21:53 +0000 (10:21 +0100)]
ceph-volume: fix regression introcuded via #43536

The recent changes from PR #43536 introduced a regeression preventing from
running ceph-volume in a containerized context on Ubuntu 18.04.

Given that the path for the binary `lvs` differs between CentOS 8 and Ubuntu 18.04.
(`/usr/sbin/lvs` and `/sbin/lvs` respictively). It means that ceph-volume running
in the container on CentOS 8 sees the `lvs` binary at `/usr/sbin/lvs` and try to
run it with `nsenter` on the host which is running Ubuntu 18.04.

Fixes: https://tracker.ceph.com/issues/53812
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 95e88cda3df76b59b548ae808df0ef7f19db1f63)
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agoMerge pull request #44566 from falcon78921/minor-messaging-nit
Sebastian Wagner [Mon, 17 Jan 2022 15:47:53 +0000 (16:47 +0100)]
Merge pull request #44566 from falcon78921/minor-messaging-nit

mgr/cephadm: fix minor grammar nit in Dry-Runs message

2 years agomgr/dashboard: report ceph tracker bug/feature through GUI
Avan Thakkar [Mon, 10 Jan 2022 09:22:26 +0000 (14:52 +0530)]
mgr/dashboard: report ceph tracker bug/feature through GUI

Fixes: https://tracker.ceph.com/issues/44851
Signed-off-by: Shreya Sharma <shreyasharma.ss305@gmail.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2 years agoMerge pull request #44510 from rzarzynski/wip-cephadm-docfix
Sebastian Wagner [Mon, 17 Jan 2022 09:21:45 +0000 (10:21 +0100)]
Merge pull request #44510 from rzarzynski/wip-cephadm-docfix

doc/cephadm: improve the development doc a bit

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoMerge pull request #44485 from adk3798/agent-permissions
Sebastian Wagner [Mon, 17 Jan 2022 08:40:13 +0000 (09:40 +0100)]
Merge pull request #44485 from adk3798/agent-permissions

cephadm: fix permissions on agent files

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoMerge pull request #44506 from sebastian-philipp/orch-suite-add-scsi
Sebastian Wagner [Mon, 17 Jan 2022 08:39:50 +0000 (09:39 +0100)]
Merge pull request #44506 from sebastian-philipp/orch-suite-add-scsi

qa/suites/orch/cephadm: Also run the rbd/iscsi suite

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Melissa Li <mingkli@redhat.com>
2 years agoqa: Add tests snapshot clone failure with quota
Kotresh HR [Wed, 12 Jan 2022 09:37:13 +0000 (15:07 +0530)]
qa: Add tests snapshot clone failure with quota

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agomgr/volumes: Fix subvoume snapshot clone failure
Kotresh HR [Wed, 12 Jan 2022 09:31:53 +0000 (15:01 +0530)]
mgr/volumes: Fix subvoume snapshot clone failure

Problem:
The subvolume snapshot clone fails if the quota on the source
has exceeded. Since the quota is not strictly enforced at the
byte range, this is a possibility.

Cause:
The quota on the clone is set prior to copying the data
from the source. Hence the quota mostly get enforced before
copying the entire data from the source resulting in the
clone failure.

Solution:
Enforce quota on the clone after the data is copied.

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agomgr/dashboard: include autopep8 for dashboard qa tasks
Avan Thakkar [Sun, 16 Jan 2022 20:07:51 +0000 (01:37 +0530)]
mgr/dashboard: include autopep8 for dashboard qa tasks

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2 years agoMerge pull request #44603 from cbodley/wip-cmake-parquet
Casey Bodley [Fri, 14 Jan 2022 22:48:07 +0000 (17:48 -0500)]
Merge pull request #44603 from cbodley/wip-cmake-parquet

rgw: disable parquet by default

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agobuild: revert arrow package dependency wip-cmake-parquet
Casey Bodley [Fri, 14 Jan 2022 19:54:09 +0000 (14:54 -0500)]
build: revert arrow package dependency

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agocmake: disable parquet by default
Casey Bodley [Fri, 14 Jan 2022 19:50:47 +0000 (14:50 -0500)]
cmake: disable parquet by default

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agodoc/foundation.rst: qct is no longer a member
Neha Ojha [Fri, 14 Jan 2022 19:17:58 +0000 (19:17 +0000)]
doc/foundation.rst: qct is no longer a member

Signed-off-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #44523 from ljflores/wip-telemetry-dashboard
Ernesto Puerta [Fri, 14 Jan 2022 19:11:15 +0000 (20:11 +0100)]
Merge pull request #44523 from ljflores/wip-telemetry-dashboard

mgr/dashboard/telemetry: reduce telemetry dashboard preview size

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2 years agoMerge pull request #44550 from jdurgin/wip-pool-get-quota
Yuri Weinstein [Fri, 14 Jan 2022 18:46:49 +0000 (10:46 -0800)]
Merge pull request #44550 from jdurgin/wip-pool-get-quota

mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #42735 from amathuria/wip-amathuria-scrub-stats
Yuri Weinstein [Fri, 14 Jan 2022 18:46:28 +0000 (10:46 -0800)]
Merge pull request #42735 from amathuria/wip-amathuria-scrub-stats

osd/scrub: Add stats to PG dump for number of objects scrubbed

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoMerge pull request #43667 from ifed01/wip-ifed-fix-ram-gridy-fsck
Neha Ojha [Fri, 14 Jan 2022 18:27:31 +0000 (10:27 -0800)]
Merge pull request #43667 from ifed01/wip-ifed-fix-ram-gridy-fsck

os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agoMerge pull request #44440 from soumyakoduri/wip-skoduri-dbstore-fixes
Soumya Koduri [Fri, 14 Jan 2022 18:08:22 +0000 (23:38 +0530)]
Merge pull request #44440 from soumyakoduri/wip-skoduri-dbstore-fixes

rgw/dbstore: Misc fixes

2 years agoMerge pull request #44552 from jdurgin/wip-releases-doc
Neha Ojha [Fri, 14 Jan 2022 17:42:08 +0000 (09:42 -0800)]
Merge pull request #44552 from jdurgin/wip-releases-doc

doc/releases: remove outdated info and versions; mark nautilus eol

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agorgw/dbstore: hide dbstore_log.h from rgw_main.cc
Casey Bodley [Thu, 13 Jan 2022 20:56:11 +0000 (15:56 -0500)]
rgw/dbstore: hide dbstore_log.h from rgw_main.cc

dbstore_log.h sets global dout_subsys/dout_prefix macros, and was
leaking into rgw_main.cc through the common/dbstore.h. this caused all
of rgw_main's log output to start with the wrong prefix "rgw dbstore: "

Fixes: https://tracker.ceph.com/issues/53177
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoqa/dashboard: ensure node 16 is installed
Ernesto Puerta [Thu, 13 Jan 2022 16:21:12 +0000 (17:21 +0100)]
qa/dashboard: ensure node 16 is installed

For Ubuntu: https://github.com/nodesource/distributions#manual-installation

Fixes: https://tracker.ceph.com/issues/53843
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #44370 from benhanokh/NCB_expand_device_fix
Yuri Weinstein [Fri, 14 Jan 2022 17:06:41 +0000 (09:06 -0800)]
Merge pull request #44370 from benhanokh/NCB_expand_device_fix

NCB code doesn't update allocation file when we expand-device

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agoMerge pull request #44251 from yaarith/telemetry-opt-in
Yuri Weinstein [Fri, 14 Jan 2022 17:06:11 +0000 (09:06 -0800)]
Merge pull request #44251 from yaarith/telemetry-opt-in

mgr/telemetry: introduce new design for varying report data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>