]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agodoc: add setting/unsetting `mon_mds_skip_sanity` config for manual ceph-mds upgrade 46236/head
Venky Shankar [Thu, 12 May 2022 05:23:26 +0000 (10:53 +0530)]
doc: add setting/unsetting `mon_mds_skip_sanity` config for manual ceph-mds upgrade

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46136 from athanatos/sjust/wip-crimson-operation-refactor-for...
Samuel Just [Thu, 12 May 2022 02:06:53 +0000 (19:06 -0700)]
Merge pull request #46136 from athanatos/sjust/wip-crimson-operation-refactor-for-review

crimson: refactor osd operation pipeline

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #45390 from xxhdx1985126/wip-backref2
Samuel Just [Wed, 11 May 2022 20:51:13 +0000 (13:51 -0700)]
Merge pull request #45390 from xxhdx1985126/wip-backref2

crimson/os/seastore: add back-reference

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #46126 from soumyakoduri/wip-skoduri-dbstore-vstart
Soumya Koduri [Wed, 11 May 2022 18:01:29 +0000 (23:31 +0530)]
Merge pull request #46126 from soumyakoduri/wip-skoduri-dbstore-vstart

rgw/dbstore: Support user creation via `radosgw-admin`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45064 from s-t-e-v-e-n-k/no-direct-use-of-nose
Casey Bodley [Wed, 11 May 2022 16:28:48 +0000 (12:28 -0400)]
Merge pull request #45064 from s-t-e-v-e-n-k/no-direct-use-of-nose

test: No direct use of nose

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45762 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks
Yuri Weinstein [Wed, 11 May 2022 14:34:42 +0000 (07:34 -0700)]
Merge pull request #45762 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks

cls_lock: check expired lock before unlock

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge pull request #45529 from NitzanMordhai/wip-nitzan-pglog-dups-not-trimmed
Yuri Weinstein [Wed, 11 May 2022 14:34:12 +0000 (07:34 -0700)]
Merge pull request #45529 from NitzanMordhai/wip-nitzan-pglog-dups-not-trimmed

osd/PGLog.cc: Trim duplicates by number of entries

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #44839 from NitzanMordhai/wip-nitzan-mon-ok-to-stop
Yuri Weinstein [Wed, 11 May 2022 14:32:49 +0000 (07:32 -0700)]
Merge pull request #44839 from NitzanMordhai/wip-nitzan-mon-ok-to-stop

mon/Monitor: Validate monitor ids during 'mon ok-to-stop' cmd

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #46164 from rishabh-d-dave/vstart-minor-fixes
Venky Shankar [Wed, 11 May 2022 05:56:09 +0000 (11:26 +0530)]
Merge pull request #46164 from rishabh-d-dave/vstart-minor-fixes

qa/vstart_runner: minor bug fix and minor adjustments

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46217 from Matan-B/wip-matanb-c-shard_services
Samuel Just [Wed, 11 May 2022 04:50:27 +0000 (21:50 -0700)]
Merge pull request #46217 from Matan-B/wip-matanb-c-shard_services

crimson/osd: Moving shard_services to PGBackend

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46208 from rzarzynski/wip-crimson-main-using-cleanups
Kefu Chai [Wed, 11 May 2022 02:41:06 +0000 (10:41 +0800)]
Merge pull request #46208 from rzarzynski/wip-crimson-main-using-cleanups

crimson/osd: drop "using ..." from main.cc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #46110 from dparmar18/fix_onecmd
Venky Shankar [Tue, 10 May 2022 17:28:41 +0000 (22:58 +0530)]
Merge pull request #46110 from dparmar18/fix_onecmd

cephfs-shell: make onecmd() print proper error msg

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46047 from dparmar18/qa_testcephfs_shell_pep8_fixes
Venky Shankar [Tue, 10 May 2022 17:27:16 +0000 (22:57 +0530)]
Merge pull request #46047 from dparmar18/qa_testcephfs_shell_pep8_fixes

qa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #45808 from rhcs-dashboard/add-cpu/mem-usage
Ernesto Puerta [Tue, 10 May 2022 16:06:21 +0000 (18:06 +0200)]
Merge pull request #45808 from rhcs-dashboard/add-cpu/mem-usage

mgr/dashboard: introduce memory and cpu usage for daemons

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: ceph-jenkins <NOT@FOUND>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agoMerge pull request #46190 from rhcs-dashboard/smart-data-fix
Ernesto Puerta [Tue, 10 May 2022 15:55:58 +0000 (17:55 +0200)]
Merge pull request #46190 from rhcs-dashboard/smart-data-fix

mgr/dashboard: fix smart data error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #45894 from rishabh-d-dave/stop-sh-check-mdss
Venky Shankar [Tue, 10 May 2022 14:58:01 +0000 (20:28 +0530)]
Merge pull request #45894 from rishabh-d-dave/stop-sh-check-mdss

src/stop.sh: check if ceph-mds daemon(s) exist beforehand

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46143 from tchaikov/wip-dashboard-angular
Kefu Chai [Tue, 10 May 2022 14:19:16 +0000 (22:19 +0800)]
Merge pull request #46143 from tchaikov/wip-dashboard-angular

cmake: s/--prod/--configuration=production/

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agocrimson/osd: Moving shard_services to PGBackend 46217/head
Matan Breizman [Tue, 10 May 2022 11:11:48 +0000 (11:11 +0000)]
crimson/osd: Moving shard_services to PGBackend

ec and replicated backends are derived from PGBackend,
shard_services should be a member of the base class.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
3 years agoMerge pull request #46093 from pdvian/wip-admin-curl
Casey Bodley [Tue, 10 May 2022 12:24:41 +0000 (08:24 -0400)]
Merge pull request #46093 from pdvian/wip-admin-curl

examples: Add rgw script to access admin APIs using curl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #46196 from rhcs-dashboard/smart-data-scsi
Ernesto Puerta [Tue, 10 May 2022 12:17:12 +0000 (14:17 +0200)]
Merge pull request #46196 from rhcs-dashboard/smart-data-scsi

mgr/dashboard: smart data for devices with scsi protocol

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
3 years agoMerge pull request #46201 from rhcs-dashboard/daemon_count_hosts
Ernesto Puerta [Tue, 10 May 2022 11:52:18 +0000 (13:52 +0200)]
Merge pull request #46201 from rhcs-dashboard/daemon_count_hosts

mgr/dashboard: show daemon count instead of services in host page

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agoMerge pull request #46174 from rhcs-dashboard/physical-disks-selection
Ernesto Puerta [Tue, 10 May 2022 11:51:27 +0000 (13:51 +0200)]
Merge pull request #46174 from rhcs-dashboard/physical-disks-selection

mgr/dashboard: devices with same UID causes multiselection

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #46159 from rhcs-dashboard/disable-external-grafana-snapshots
Ernesto Puerta [Tue, 10 May 2022 11:48:19 +0000 (13:48 +0200)]
Merge pull request #46159 from rhcs-dashboard/disable-external-grafana-snapshots

mgr/cephadm: Disable external snapshots in grafana

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agorgw/dbstore: Support user creation via `radosgw-admin` 46126/head
Soumya Koduri [Tue, 3 May 2022 14:30:41 +0000 (20:00 +0530)]
rgw/dbstore: Support user creation via `radosgw-admin`

With the changes in https://github.com/ceph/ceph/pull/45987 ,
'radosgw-admin' command can be used to execute few admin operations on other stores.

This fix include changes to support  user creation/remove via `radosgw-admin`
command in dbstore.

Also fixed an issue with updating objv_tracker in op_state.user

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agoMerge pull request #46169 from rishabh-d-dave/vr-run-mnt-cmd
Venky Shankar [Tue, 10 May 2022 09:38:01 +0000 (15:08 +0530)]
Merge pull request #46169 from rishabh-d-dave/vr-run-mnt-cmd

qa/vstart_runner: fix bug in LocalFuseMount._run_mount_cmd()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agomgr/dashboard: smart data for devices with scsi protocol 46196/head
Nizamudeen A [Sun, 8 May 2022 14:27:34 +0000 (19:57 +0530)]
mgr/dashboard: smart data for devices with scsi protocol

In the dashboard, we've been showing smart data for hdd devices with ata
protocol only. Otherwise we show a No Smart Data found error which is
clearly misleading since Smart Data is returned even in the api call.

So this PR is trying to show the smart data for hdd devices
that uses scsi protocol too.

Fixes: https://tracker.ceph.com/issues/55574
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agomgr/dashboard: show daemon count instead of services in host page 46201/head
Nizamudeen A [Mon, 9 May 2022 08:43:19 +0000 (14:13 +0530)]
mgr/dashboard: show daemon count instead of services in host page

Fixes: https://tracker.ceph.com/issues/47218
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoqa/vstart_runner: fix args to os.path.join() 46164/head
Rishabh Dave [Thu, 5 May 2022 14:12:55 +0000 (19:42 +0530)]
qa/vstart_runner: fix args to os.path.join()

Result of os.path.join() before "./bin/ceph-mds" and after
"./bin/./ceph-mds".

Before -

2022-05-05 19:36:11,100.100 DEBUG:__main__:> ./bin/./ceph-mds -i a

After -

2022-05-05 19:38:48,179.179 DEBUG:__main__:> ./bin/ceph-mds -i a

Signed-off-by: Rishabh Dave <ridave@redhat.com>
3 years agoqa/vstart_runner: minor bug fix and minor adjustments
Rishabh Dave [Thu, 5 May 2022 13:17:08 +0000 (18:47 +0530)]
qa/vstart_runner: minor bug fix and minor adjustments

The message regarding deletion of helper tools is printed for every
command. This message should be printed only when applicable.

Besides -

* Move XXX comments to _do_run() since it increases visibility of
  these messages.
* Move omission of arguments stuff to new method clear up the clutter.
* And remove shell as a parameter from _perform_checks_and_adjustments
  since it's redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
3 years agoMerge pull request #37667 from rishabh-d-dave/vr-inherit-Remote
Venky Shankar [Tue, 10 May 2022 05:00:22 +0000 (10:30 +0530)]
Merge pull request #37667 from rishabh-d-dave/vr-inherit-Remote

vstart_runner: let LocalRemote inherit Remote

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #46194 from athanatos/sjust/wip-55565
Samuel Just [Tue, 10 May 2022 00:31:54 +0000 (17:31 -0700)]
Merge pull request #46194 from athanatos/sjust/wip-55565

crimson: fix op tracking related bugs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common/operation: trim AggregateBlockingEvent::events entries 46194/head
Samuel Just [Sat, 7 May 2022 07:02:58 +0000 (07:02 +0000)]
crimson/common/operation: trim AggregateBlockingEvent::events entries

PglogBasedRecovery and BackfillRecovery reuse the same Operation
until their respective operations are complete.  Each recovery
operation adds an entry to AggregateBlockingEvent::events.  This
way, we only retain entries that are currently blocking.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46127 from Matan-B/wip-matanb-cyan-clone
Samuel Just [Mon, 9 May 2022 19:53:30 +0000 (12:53 -0700)]
Merge pull request #46127 from Matan-B/wip-matanb-cyan-clone

crimson/os: Add OP_CLONE to cyanstore

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46205 from rzarzynski/wip-crimson-cls_rgw-objapi
Samuel Just [Mon, 9 May 2022 19:47:17 +0000 (12:47 -0700)]
Merge pull request #46205 from rzarzynski/wip-crimson-cls_rgw-objapi

crimson/osd, rgw: implement the new Objclass API methods for cls_rgw

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44418 from AmnonHanuhov/wip-fix_futurized_store
Samuel Just [Mon, 9 May 2022 19:46:25 +0000 (12:46 -0700)]
Merge pull request #44418 from AmnonHanuhov/wip-fix_futurized_store

crimson/os: Don't limit the amount of returned keys per omap get call

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd: drop "using std::string" from main.cc 46208/head
Radosław Zarzyński [Mon, 9 May 2022 16:09:37 +0000 (18:09 +0200)]
crimson/osd: drop "using std::string" from main.cc

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: drop "using config_t = ..." from main.cc
Radosław Zarzyński [Mon, 9 May 2022 16:06:45 +0000 (18:06 +0200)]
crimson/osd: drop "using config_t = ..." from main.cc

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd, rgw: implement the new Objclass API methods for cls_rgw 46205/head
Radosław Zarzyński [Mon, 9 May 2022 12:43:34 +0000 (14:43 +0200)]
crimson/osd, rgw: implement the new Objclass API methods for cls_rgw

They were introduced by 3877c1e37f2fa4e1574b57f05132288f210835a7.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agoMerge pull request #45732 from mchangir/mgr-snap_schedule-validate-fs-for-all-commands
Venky Shankar [Mon, 9 May 2022 14:00:45 +0000 (19:30 +0530)]
Merge pull request #45732 from mchangir/mgr-snap_schedule-validate-fs-for-all-commands

mgr/snap_schedule: validate fs name argument

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agocephfs-shell: make onecmd() print proper error msg 46110/head
dparmar18 [Mon, 2 May 2022 11:03:30 +0000 (16:33 +0530)]
cephfs-shell: make onecmd() print proper error msg

Rationale: Whenever a python exception occurred in cephfs-shell,
           it would often only be the exception message but doesn't
           say anything about the type of exception. For example if
           `ZeroDivisionError: division by zero` occurred, the onecmd()
           would print `division by zero` but will omit the type of
           exception. In this case it's easy to understand but let's
           say an `KeyError` exception occurred for a key `9999` which
           is not existent in the dictionary, onecmd() would print
           just `9999` in this scenario and it would be very difficult
           to interpret what type of error it is.

Fixes: https://tracker.ceph.com/issues/55536
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
3 years agoMerge branch 'master' into qa_testcephfs_shell_pep8_fixes 46047/head
Dhairya Parmar [Mon, 9 May 2022 10:16:04 +0000 (15:46 +0530)]
Merge branch 'master' into qa_testcephfs_shell_pep8_fixes

3 years agoqa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations
dparmar18 [Wed, 27 Apr 2022 08:31:37 +0000 (14:01 +0530)]
qa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations
NOTE: Although most of the issues are fixed but a few function
      and variable names are unchanged in order to prevent
      ambiguity and preserve their meaning.

      They are:
      - functions: setUp(), test_ls_H_prints_human_readable_file_size(),
      - variables: ls_H_output, ls_H_file_size

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
3 years agomgr/dashboard: fix smart data error 46190/head
Nizamudeen A [Fri, 6 May 2022 15:19:18 +0000 (20:49 +0530)]
mgr/dashboard: fix smart data error

the error in the log was this

```
"/usr/share/ceph/mgr/dashboard/services/ceph_service.py", line 253, in _get_smart_data_by_device
May 06 07:38:39 occldlr750-1.occl208.lab conmon[2142938]:     svc_type, svc_id = daemon.split('.')
May 06 07:38:39 occldlr750-1.occl208.lab conmon[2142938]: ValueError: too many values to unpack (expected 2)
```

on the cluster, the output of `ceph device ls-by-host` looks like this

```
ceph: root@occldlr750-1 /]# ceph device ls-by-host occldlr750-1.occl208.lab
DEVICE                        DEV   DAEMONS                       EXPECTED FAILURE
DELLBOSS_VD_cbd004c975390010  sda   mon.occldlr750-1.occl208.lab
WDC_WUH721818AL5204_3FGZR3JT  sdda  osd.20
WDC_WUH721818AL5204_3FH4315T  sdbf  osd.94
WDC_WUH721818AL5204_3FHP58TT  sdec  osd.30
WDC_WUH721818AL5204_3FHSK8HT  sdu   osd.78
WDC_WUH721818AL5204_3FHVTS9T  sdfi  osd.47
WDC_WUH721818AL5204_3FHWJE8T  sdv   osd.23
WDC_WUH721818AL5204_3FHXHETT  sdcl  osd.11
WDC_WUH721818AL5204_3FHXKP1T  sdcj  osd.10
```

the first device is mon and its name is mon.occldlr750-1.occl208.lab.

In our dashboard code, when fetching the smart data we have a line like
this

`svc_type, svc_id = daemon.split('.')`

so for the mon the output of `daemon.split('.') will be ['mon', 'occldlr750-1', 'occl208', 'lab']. The svc_id gets split into three because of the split. I am changing that and giving the criteria as splitting only on the first occurence of the dot and the considering everything that comes after the dot as the svc_id of the device.

Fixes: https://tracker.ceph.com/issues/55571
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #46064 from liu-chunmei/crimson-pool-quota
Liu-Chunmei [Mon, 9 May 2022 02:29:39 +0000 (19:29 -0700)]
Merge pull request #46064 from liu-chunmei/crimson-pool-quota

Crimson pool quota

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agomgr/dashboard: devices with same UID causes multiselection 46174/head
Nizamudeen A [Thu, 5 May 2022 17:43:38 +0000 (23:13 +0530)]
mgr/dashboard: devices with same UID causes multiselection

In the Physical Disks page, the uids for multiple devices are coming in
as same and that causes the selection to go berserk and select multiple
rows with same UID. The uid is generated in the frontend service call
itself. I just added some more parameters to it inorder to make it more
unique.

The second issue is the number of selected number getting multiplied
exponentially. Its because each time the table is updated or refreshed,
we push the row with the number of selected items we had before and that
causes the number of selection to multiply.

Fixes: https://tracker.ceph.com/issues/55523
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agocrimson/os: Add OP_CLONE to cyanstore 46127/head
Matan Breizman [Tue, 3 May 2022 14:15:57 +0000 (14:15 +0000)]
crimson/os: Add OP_CLONE to cyanstore

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
3 years agocrimson/osd: check full 46064/head
chunmei-liu [Sat, 7 May 2022 23:42:11 +0000 (16:42 -0700)]
crimson/osd: check full

need check if pool exceeds its quota

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agocrimson/errorator: change errorator template to int
chunmei-liu [Wed, 27 Apr 2022 08:52:06 +0000 (01:52 -0700)]
crimson/errorator: change errorator template to int

need define EDQUOT which is not included in std::errc.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agocrimson: mark OSD::get_or_create_pg and wait_for_pg private 46136/head
Samuel Just [Wed, 4 May 2022 23:36:01 +0000 (23:36 +0000)]
crimson: mark OSD::get_or_create_pg and wait_for_pg private

All users now use start_pg_operation.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: refactor RepRequest to use start_pg_operation
Samuel Just [Wed, 4 May 2022 23:14:44 +0000 (23:14 +0000)]
crimson: refactor RepRequest to use start_pg_operation

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: refactor RecoverySubRequest to use start_pg_operation
Samuel Just [Wed, 4 May 2022 23:00:35 +0000 (23:00 +0000)]
crimson: refactor RecoverySubRequest to use start_pg_operation

A side effect of this PR is that RecoverySubRequest will now use the
peering pipeline for ordering.  Not using before was, I think, actually
a bug which could result in pull operations ordering ahead of the
activation message when pulling objects from non-acting osds.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd/pg_recovery: remove new_started, fix parallel_for_each call
Samuel Just [Sat, 7 May 2022 06:55:31 +0000 (23:55 -0700)]
crimson/osd/pg_recovery: remove new_started, fix parallel_for_each call

Presumably typo from b7e2b11595c6e2cd2c638135ef6f044c40efaa87.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/common/operation: use std::list for AggregateBlockingEvent::events
Samuel Just [Sat, 7 May 2022 05:42:47 +0000 (22:42 -0700)]
crimson/common/operation: use std::list for AggregateBlockingEvent::events

std::vector's references aren't stable after emplace_back.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/segment_cleaner: set gc size back to 1MB per cycle 45390/head
Xuehan Xu [Fri, 1 Apr 2022 08:14:53 +0000 (16:14 +0800)]
crimson/os/seastore/segment_cleaner: set gc size back to 1MB per cycle

This should reduce the conflicts between gc transactions and io transactions

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/btree: allow backref extents to be loaded from disk without initi...
Xuehan Xu [Tue, 29 Mar 2022 06:48:48 +0000 (14:48 +0800)]
crimson/os/seastore/btree: allow backref extents to be loaded from disk without initializing

For the purpose of higher space reclaim perf, we load backref extents by
parallelly invoking Cache::get_extent_by_type, which means loaded backref
extents won't be initialized

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/btree: fix log error
Xuehan Xu [Tue, 29 Mar 2022 05:16:30 +0000 (13:16 +0800)]
crimson/os/seastore/btree: fix log error

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: record replay_from info for dirty extents and alloc infos in...
Xuehan Xu [Thu, 10 Mar 2022 09:46:37 +0000 (17:46 +0800)]
crimson/os/seastore: record replay_from info for dirty extents and alloc infos in segment header/tail

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: better debug outputs
Xuehan Xu [Tue, 29 Mar 2022 09:40:36 +0000 (17:40 +0800)]
crimson/os/seastore: better debug outputs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/transaction_manager: prepare extents for lba_manager/backref_mana...
Xuehan Xu [Sun, 27 Mar 2022 08:44:16 +0000 (16:44 +0800)]
crimson/os/seastore/transaction_manager: prepare extents for lba_manager/backref_manager's complete_transaction

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/btree: clean up: add print_detail method for btree leaf nodes
Xuehan Xu [Sun, 27 Mar 2022 07:44:00 +0000 (15:44 +0800)]
crimson/os/seastore/btree: clean up: add print_detail method for btree leaf nodes

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/segment_cleaner: trim journal and reclaim space with the help...
Xuehan Xu [Tue, 15 Mar 2022 12:44:30 +0000 (20:44 +0800)]
crimson/os/seastore/segment_cleaner: trim journal and reclaim space with the help of backref

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/transaction_manager: scan backref tree on mount
Xuehan Xu [Tue, 15 Mar 2022 03:40:52 +0000 (11:40 +0800)]
crimson/os/seastore/transaction_manager: scan backref tree on mount

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/cache: record backref extents placement update in Cache::backref_...
Xuehan Xu [Tue, 15 Mar 2022 03:37:28 +0000 (11:37 +0800)]
crimson/os/seastore/cache: record backref extents placement update in Cache::backref_extents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: add depth parameter to FixedKVBtree::mapped_space_visitor_t
Xuehan Xu [Tue, 15 Mar 2022 02:28:01 +0000 (10:28 +0800)]
crimson/os/seastore: add depth parameter to FixedKVBtree::mapped_space_visitor_t

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/cache: replay alloc info deltas
Xuehan Xu [Tue, 15 Mar 2022 01:18:18 +0000 (09:18 +0800)]
crimson/os/seastore/cache: replay alloc info deltas

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/segment_cleaener: flush backrefs when trimming journal
Xuehan Xu [Mon, 14 Mar 2022 08:55:16 +0000 (16:55 +0800)]
crimson/os/seastore/segment_cleaener: flush backrefs when trimming journal

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/cache: add extent alloc info to journal records
Xuehan Xu [Sun, 13 Mar 2022 04:15:38 +0000 (12:15 +0800)]
crimson/os/seastore/cache: add extent alloc info to journal records

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: rename rbm_alloc_delta_t to alloc_delta_t
Xuehan Xu [Sun, 13 Mar 2022 03:05:57 +0000 (11:05 +0800)]
crimson/os/seastore: rename rbm_alloc_delta_t to alloc_delta_t

Devices other than rbm need this delta

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: add backref manager
Xuehan Xu [Fri, 18 Feb 2022 09:36:10 +0000 (17:36 +0800)]
crimson/os/seastore: add backref manager

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: lazy backref tree updates for extent (re)placement
Xuehan Xu [Thu, 10 Mar 2022 02:55:38 +0000 (10:55 +0800)]
crimson/os/seastore: lazy backref tree updates for extent (re)placement

Buffer all backref updates(allocs/releases) in Cache.
Also, organize them into an intrusive set, so later backref query can be done efficiently
Backref query should always be first applied to this set and then to the backref manager

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/osd/pg_recovery: avoid using create_part_trigger() directly
Samuel Just [Sat, 7 May 2022 01:05:15 +0000 (18:05 -0700)]
crimson/osd/pg_recovery: avoid using create_part_trigger() directly

AggregateBlockingEvent::TriggerI::maybe_record_blocking() should handle
invoking create_part_trigger().

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: retain reference to WaitForObjectRecovery until blocker resolves
Samuel Just [Fri, 6 May 2022 05:05:31 +0000 (05:05 +0000)]
crimson: retain reference to WaitForObjectRecovery until blocker resolves

PGRecovery::on_global_recover destroys the map entry without waiting for
the future returned from

    seastar::future<> wait_for_recovered(BlockingEvent::TriggerI& trigger) {

This commit changes WaitForObjectRecovery to be refcounted and retains a
reference until the future resolves.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46170 from elacunza/patch-1
Anthony D'Atri [Fri, 6 May 2022 15:42:41 +0000 (08:42 -0700)]
Merge pull request #46170 from elacunza/patch-1

doc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule example

3 years agoexamples: Add rgw script to access admin APIs using curl 46093/head
Prashant D [Fri, 29 Apr 2022 15:28:55 +0000 (11:28 -0400)]
examples: Add rgw script to access admin APIs using curl

The rgw_admin_curl.sh script will allow end-user/developers to
access RGW admin APIs through curl command.

Signed-off-by: Prashant D <pdhange@redhat.com>
3 years agoMerge pull request #46108 from rhcs-dashboard/fix-ingress-ssl-cert-validation
Ernesto Puerta [Fri, 6 May 2022 11:52:13 +0000 (13:52 +0200)]
Merge pull request #46108 from rhcs-dashboard/fix-ingress-ssl-cert-validation

mgr/dashboard: fix ssl cert validation for ingress service creation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agorgw/dbstore: Add options to configure dbstore db file path & name_prefix
Soumya Koduri [Tue, 19 Apr 2022 18:04:10 +0000 (23:34 +0530)]
rgw/dbstore: Add options to configure dbstore db file path & name_prefix

DBStore will use the below options when creating .db file for rgw server
to use.

'dbstore_db_dir':
    - path for the directory to store db backend store files
    - default: /var/run/ceph
'dbstore_db_name_prefix':
    - prefix to be used for the files created by db backend store
    - defauult: dbstore

For eg., by default the full path of the file shall be
eg., /var/run/ceph/dbstore-default_ns.db

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agomgr/cephadm: Disable external snapshots in grafana 46159/head
Aashish Sharma [Thu, 5 May 2022 08:22:33 +0000 (13:52 +0530)]
mgr/cephadm: Disable external snapshots in grafana

This PR intends to disable the external snapshots in grafana to reduce security threats

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
3 years agodoc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule... 46170/head 46189/head
Eneko Lacunza [Thu, 5 May 2022 15:41:51 +0000 (17:41 +0200)]
doc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule example

Rule example is missing min_size and max_size, both are required.

Signed-off-by: Eneko Lacunza <elacunza@binovo.es>
3 years agocrimson: remove unused PGRecovery method declarations
Samuel Just [Fri, 6 May 2022 07:11:52 +0000 (07:11 +0000)]
crimson: remove unused PGRecovery method declarations

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd: replace OpSequencer with simpler intrusive_list based implementation
Samuel Just [Fri, 29 Apr 2022 01:16:32 +0000 (01:16 +0000)]
crimson/osd: replace OpSequencer with simpler intrusive_list based implementation

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #45935 from dparmar18/cephfs-shell-doc-update
Venky Shankar [Fri, 6 May 2022 04:22:03 +0000 (09:52 +0530)]
Merge pull request #45935 from dparmar18/cephfs-shell-doc-update

doc/man/cephfs-shell.rst: add use of exclamation mark with cmds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agocrimson/osd/osd_operations/client_request: refactor for OSD::start_pg_operation
Samuel Just [Fri, 29 Apr 2022 22:37:57 +0000 (22:37 +0000)]
crimson/osd/osd_operations/client_request: refactor for OSD::start_pg_operation

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd/osd_operations/peering_event: refactor to use OSD::start_pg_operation
Samuel Just [Thu, 27 Jan 2022 22:55:11 +0000 (22:55 +0000)]
crimson/osd/osd_operations/peering_event: refactor to use OSD::start_pg_operation

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd/osd: introduce start_pg_operation
Samuel Just [Fri, 11 Feb 2022 04:22:56 +0000 (04:22 +0000)]
crimson/osd/osd: introduce start_pg_operation

We're going to unify responsibility for routing requests to requests
to start_pg_operation from the different osd operation types.  Later,
this will move into a new type responsible for dealing with the pg->core
mapping.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/osd: consolidate connection pipeline definitions
Samuel Just [Fri, 11 Feb 2022 02:33:34 +0000 (18:33 -0800)]
crimson/osd: consolidate connection pipeline definitions

Really, all pg routable events (peering events, client operations, etc)
should have the same sequence of pre-pg blockers.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/common/operation: return intrusive_ptr<T> rather than T::IRef
Samuel Just [Thu, 3 Mar 2022 01:10:36 +0000 (17:10 -0800)]
crimson/common/operation: return intrusive_ptr<T> rather than T::IRef

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: rename WaitForActiveBlocker::on_active
Samuel Just [Mon, 25 Apr 2022 23:32:11 +0000 (16:32 -0700)]
crimson: rename WaitForActiveBlocker::on_active

We need to unblock this in cases other than on_active.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46097 from tchaikov/wip-crimson-cleanups
Kefu Chai [Fri, 6 May 2022 00:02:08 +0000 (08:02 +0800)]
Merge pull request #46097 from tchaikov/wip-crimson-cleanups

crimson/osd/main: s/values/config_values/

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #45788 from rzarzynski/wip-crimson-op-tracking-2
Samuel Just [Thu, 5 May 2022 19:56:42 +0000 (12:56 -0700)]
Merge pull request #45788 from rzarzynski/wip-crimson-op-tracking-2

crimson: bring the core of op tracking

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #46006 from phlogistonjohn/jjm-nfs-cleanups-pt4
Adam King [Thu, 5 May 2022 18:59:39 +0000 (14:59 -0400)]
Merge pull request #46006 from phlogistonjohn/jjm-nfs-cleanups-pt4

mgr/nfs: minor organizational cleanups

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #46035 from rkachach/fix_issue_52906
Adam King [Thu, 5 May 2022 18:52:07 +0000 (14:52 -0400)]
Merge pull request #46035 from rkachach/fix_issue_52906

mgr/cephadm: adding logic to close ports when removing a daemon

Reviewed-by: Adam King <adking@redhat.com>
3 years agoqa/vstart_runner: fix bug in LocalFuseMount._run_mount_cmd() 46169/head
Rishabh Dave [Sat, 30 Apr 2022 07:26:49 +0000 (12:56 +0530)]
qa/vstart_runner: fix bug in LocalFuseMount._run_mount_cmd()

This methods fails to collect return value from
FuseMount._run_mount_cmd() and return it. This leads to a bug for tests
that expect mount command to fail when executed with vstart_runner.py.

Fixes: https://tracker.ceph.com/issues/55553
Signed-off-by: Rishabh Dave <ridave@redhat.com>
3 years agocrimson/osd: drop legacy base classes for pipeline stages 45788/head
Radosław Zarzyński [Thu, 28 Apr 2022 15:02:09 +0000 (17:02 +0200)]
crimson/osd: drop legacy base classes for pipeline stages

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: drop the old blocking infrastructure
Radosław Zarzyński [Thu, 28 Apr 2022 10:02:28 +0000 (12:02 +0200)]
crimson/osd: drop the old blocking infrastructure

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: strip blokcing_future from OperationT
Radosław Zarzyński [Wed, 27 Apr 2022 14:51:35 +0000 (16:51 +0200)]
crimson/osd: strip blokcing_future from OperationT

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: migrate OperationThrottler to new tracking infra.
Radosław Zarzyński [Wed, 27 Apr 2022 13:51:33 +0000 (15:51 +0200)]
crimson/osd: migrate OperationThrottler to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: crimson/osd: drop blocking_future from PGActivationBlocker
Radosław Zarzyński [Wed, 27 Apr 2022 12:17:33 +0000 (14:17 +0200)]
crimson/osd: crimson/osd: drop blocking_future from PGActivationBlocker

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: crimson/osd: drop blocking_future from PGMap
Radosław Zarzyński [Wed, 27 Apr 2022 11:29:28 +0000 (13:29 +0200)]
crimson/osd: crimson/osd: drop blocking_future from PGMap

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson/osd: drop blocking_future from OSDMapGate
Radosław Zarzyński [Wed, 27 Apr 2022 11:04:37 +0000 (13:04 +0200)]
crimson/osd: drop blocking_future from OSDMapGate

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agocrimson: bring the WaitBarrier tracking
Radosław Zarzyński [Wed, 27 Apr 2022 00:12:21 +0000 (02:12 +0200)]
crimson: bring the WaitBarrier tracking

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>