]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
22 months agoqa/suites/krbd: disable POOL_APP_NOT_ENABLED health check 53311/head
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/krbd: disable POOL_APP_NOT_ENABLED health check

... same as for rbd suite.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
22 months agoqa/suites/rbd: drop POOL_APP_NOT_ENABLED from ignorelists
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/rbd: drop POOL_APP_NOT_ENABLED from ignorelists

With "mon warn on pool no app = false" in the config, it's obviously
redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
22 months agoqa/suites/rbd: disable POOL_APP_NOT_ENABLED health check
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/rbd: disable POOL_APP_NOT_ENABLED health check

Commit 990806e635a1 ("mon, qa: issue pool application warning even
if pool is empty") made it impossible to create a pool without raising
a (bogus) health alert.  See [1] for details.

[1] https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/ZTDYC5HN677RR26EB4P6PORN6L2IFH4R/

Fixes: https://tracker.ceph.com/issues/62711
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
22 months agoMerge pull request #53448 from cbodley/wip-62378
Casey Bodley [Thu, 21 Sep 2023 21:15:37 +0000 (22:15 +0100)]
Merge pull request #53448 from cbodley/wip-62378

rgw/crypt: don't deref null manifest_bl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
22 months agoMerge pull request #53045 from smanjara/wip-shilpa-revert-51772
Casey Bodley [Thu, 21 Sep 2023 19:59:59 +0000 (20:59 +0100)]
Merge pull request #53045 from smanjara/wip-shilpa-revert-51772

rgw/multisite: fixes assertion failure during realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #53493 from zdover23/wip-doc-2023-09-18-architecture-7-of-x
zdover23 [Thu, 21 Sep 2023 17:41:21 +0000 (03:41 +1000)]
Merge pull request #53493 from zdover23/wip-doc-2023-09-18-architecture-7-of-x

doc/architecture: "Edit HA Auth" (one of several)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agoMerge pull request #53505 from cbodley/wip-62771
Casey Bodley [Thu, 21 Sep 2023 17:30:07 +0000 (18:30 +0100)]
Merge pull request #53505 from cbodley/wip-62771

rgw/sal: get_placement_target_names() returns void

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
22 months agoMerge pull request #52392 from yuvalif/wip-yuval-trace-name
Yuval Lifshitz [Thu, 21 Sep 2023 17:12:01 +0000 (20:12 +0300)]
Merge pull request #52392 from yuvalif/wip-yuval-trace-name

rgw: rename request traces and change for tags

reviewed-by: cbodley

22 months agoMerge PR #50503 into main
Patrick Donnelly [Thu, 21 Sep 2023 15:51:31 +0000 (11:51 -0400)]
Merge PR #50503 into main

* refs/pull/50503/head:
mon: do not change pending if strategy is unchanged
mon/MonmapMonitor: do not propose on error in prepare_update
mon/MonmapMonitor: wait for commit before reply
mon: use wait_for_commit to reply
mon: add context list for commit wait
mon: remove unused method
test/mon: add commit benchmark script
mon/MonClient: provide config to target specific rank

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
22 months agodoc/architecture: "Edit HA Auth" (one of several) 53493/head
Zac Dover [Sun, 17 Sep 2023 20:41:28 +0000 (06:41 +1000)]
doc/architecture: "Edit HA Auth" (one of several)

Edit "High Availability Authentication" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #52863 from batrick/i62326
Adam King [Thu, 21 Sep 2023 12:14:54 +0000 (08:14 -0400)]
Merge pull request #52863 from batrick/i62326

pybind/mgr/cephadm/upgrade: stop disabling FSMap sanity checks

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53501 from zhscn/wip-lba-backref-node-size
Yingxin [Thu, 21 Sep 2023 09:20:28 +0000 (17:20 +0800)]
Merge pull request #53501 from zhscn/wip-lba-backref-node-size

crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
22 months agoMerge pull request #52150 from paulreece42/wip-grafana-quorum-fix
Nizamudeen A [Thu, 21 Sep 2023 07:06:21 +0000 (12:36 +0530)]
Merge pull request #52150 from paulreece42/wip-grafana-quorum-fix

monitoring: grafana mons out of quorum should be count - sum

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoMerge PR #53028 into main
Venky Shankar [Thu, 21 Sep 2023 00:41:54 +0000 (06:11 +0530)]
Merge PR #53028 into main

* refs/pull/53028/head:
Update MDSDaemon.cc
Update MDSRank.cc - Logoutput: Fix personal pronoun "I" to uppercase

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #53176 into main
Venky Shankar [Thu, 21 Sep 2023 00:33:42 +0000 (06:03 +0530)]
Merge PR #53176 into main

* refs/pull/53176/head:
doc: add note for removing (automatic) partitioning policy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
22 months agoMerge pull request #53467 from phlogistonjohn/jjm-cephadm-move-get_data_dir
Adam King [Wed, 20 Sep 2023 18:20:09 +0000 (14:20 -0400)]
Merge pull request #53467 from phlogistonjohn/jjm-cephadm-move-get_data_dir

cephadm: move get data dir function to daemonidentity method

Reviewed-by: Adam King <adking@redhat.com>
22 months agoMerge pull request #53415 from rkachach/fix_issue_62814
Adam King [Wed, 20 Sep 2023 18:19:02 +0000 (14:19 -0400)]
Merge pull request #53415 from rkachach/fix_issue_62814

cephadm: fix cephadm binary mount when --shared_ceph_folder is used

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoMerge pull request #53298 from phlogistonjohn/jjm-logging-2
Adam King [Wed, 20 Sep 2023 18:17:03 +0000 (14:17 -0400)]
Merge pull request #53298 from phlogistonjohn/jjm-logging-2

cephadm: enhance logging behavior

Reviewed-by: Adam King <adking@redhat.com>
22 months agoMerge pull request #52251 from rkachach/fix_issue_61856
Adam King [Wed, 20 Sep 2023 18:14:23 +0000 (14:14 -0400)]
Merge pull request #52251 from rkachach/fix_issue_61856

mgr/cephadm: Adding sort-by support for ceph orch ps

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #52982 from batrick/backport-cp-simplify
Ilya Dryomov [Wed, 20 Sep 2023 17:58:40 +0000 (19:58 +0200)]
Merge pull request #52982 from batrick/backport-cp-simplify

script/ceph-backport: perform cherry-pick in single command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge PR #53145 into main
Patrick Donnelly [Wed, 20 Sep 2023 12:57:07 +0000 (08:57 -0400)]
Merge PR #53145 into main

* refs/pull/53145/head:
mds: log message when exiting due to asok command

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
22 months agoMerge PR #53149 into main
Patrick Donnelly [Wed, 20 Sep 2023 12:31:17 +0000 (08:31 -0400)]
Merge PR #53149 into main

* refs/pull/53149/head:
qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
22 months agodoc: add note for removing (automatic) partitioning policy 53176/head
Venky Shankar [Mon, 28 Aug 2023 10:42:57 +0000 (16:12 +0530)]
doc: add note for removing (automatic) partitioning policy

Signed-off-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53309 from guits/bz2203397
Guillaume Abrioux [Wed, 20 Sep 2023 07:32:49 +0000 (09:32 +0200)]
Merge pull request #53309 from guits/bz2203397

ceph-volume: fix mpath device support

22 months agocrimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent 53501/head
Zhang Song [Tue, 19 Sep 2023 06:08:51 +0000 (14:08 +0800)]
crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
22 months agodoc/cephadm: document new cephadm logging destination settings 53298/head
John Mulligan [Wed, 6 Sep 2023 20:56:40 +0000 (16:56 -0400)]
doc/cephadm: document new cephadm logging destination settings

Add docs for setting the binary's log destination at cephadm bootstrap
or on a running cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agodoc/cephadm: clarify what cephadm component writes to the cluster log channel
John Mulligan [Wed, 6 Sep 2023 20:18:37 +0000 (16:18 -0400)]
doc/cephadm: clarify what cephadm component writes to the cluster log channel

Clarify that the cephadm orchestrator module, a part of the ceph mgr,
logs to the cluster log channel. This prepares for adding a specific
section to cover logging for the cephadm "binary".

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: remember log destination used during bootstrap
John Mulligan [Wed, 6 Sep 2023 18:15:41 +0000 (14:15 -0400)]
cephadm: remember log destination used during bootstrap

Store the log destination(s) specified on the CLI for cephadm bootstrap
as the manager configuration, unless the configuration key is explicitly
set by the input config.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agomgr/cephadm: add a module option for controlling cephadm log dest
John Mulligan [Wed, 6 Sep 2023 17:39:06 +0000 (13:39 -0400)]
mgr/cephadm: add a module option for controlling cephadm log dest

Now that cephadm has multiple possible persistent logging destinations
we need a way to choose which one to use when the command is started by
the mgr. Add the option 'cephadm_log_destination' which can take one
of 'file', 'syslog', or 'file,syslog'. If left unset (empty string)
then the behavior is equivalent to 'file' and that is the same as
previous cephadm versions.

Fixes: https://tracker.ceph.com/issues/62233
Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: add cli option to enable logging to syslog
John Mulligan [Tue, 22 Aug 2023 19:11:35 +0000 (15:11 -0400)]
cephadm: add cli option to enable logging to syslog

Add the --log-dest option to cephadm. The --log-dest option can be
specified 0, 1 or more times. If unspecified, cephadm will log to
the default location, the log file. If specified one ore more times,
each instance will enable the named logging destination.
Example:

```
cephadm boostrap

cephadm --log-dest=syslog bootstrap

cephadm --log-dest=file bootstrap

cephadm --log-dest=syslog --log-dest=file bootstrap
```

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: add support for logging to syslog/journal
John Mulligan [Tue, 22 Aug 2023 19:11:16 +0000 (15:11 -0400)]
cephadm: add support for logging to syslog/journal

Add support to logging.py for persistent logging to syslog and thus to
journald. This is accomplished by switching logging handlers depending
on the log_dest attribute of the context. Setting this value is left
for a future patch.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: move colored output support into logging.py
John Mulligan [Tue, 22 Aug 2023 16:42:14 +0000 (12:42 -0400)]
cephadm: move colored output support into logging.py

Rewrite cephadm's colored output support such that it abstracts away
the colorization into extra logging metadata. The new code will not
unconditionally put control characters into the log files. It will
only print the control chars if the stderr is a tty.
In theory this is probably more future proof as well, but it's only
got two callers so it is hard to say how useful it'll be.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agorgw/crypt: don't deref null manifest_bl 53448/head
Casey Bodley [Wed, 13 Sep 2023 20:30:03 +0000 (16:30 -0400)]
rgw/crypt: don't deref null manifest_bl

with dbstore, the manifest_bl pointer was null; check for null before
dereferencing for read_manifest_parts()

Fixes: https://tracker.ceph.com/issues/62378
Signed-off-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #51921 from kamoltat/wip-ksirivad-fix-54136
Kamoltat (Junior) Sirivadhna [Tue, 19 Sep 2023 18:12:35 +0000 (14:12 -0400)]
Merge pull request #51921 from kamoltat/wip-ksirivad-fix-54136

pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
22 months agoMerge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool
Ilya Dryomov [Tue, 19 Sep 2023 14:35:18 +0000 (16:35 +0200)]
Merge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool

doc/man: s/kvstore-tool/monstore-tool/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
22 months agodoc/man: s/kvstore-tool/monstore-tool/ 53532/head
Zac Dover [Tue, 19 Sep 2023 13:12:34 +0000 (23:12 +1000)]
doc/man: s/kvstore-tool/monstore-tool/

s/kvstore-tool/monstore-tool/ in accordance with Ilya's remark here:
https://github.com/ceph/ceph/pull/53450#discussion_r1329804085

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #53411 from rhcs-dashboard/align-charts
Nizamudeen A [Tue, 19 Sep 2023 12:38:29 +0000 (18:08 +0530)]
Merge pull request #53411 from rhcs-dashboard/align-charts

mgr/dashboard: align charts of landing page

Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoceph-volume: fix mpath device support 53309/head
Guillaume Abrioux [Wed, 6 Sep 2023 09:30:41 +0000 (09:30 +0000)]
ceph-volume: fix mpath device support

commit [1] broke mpath devices support in `disk.is_device()`

[1] https://github.com/ceph/ceph/commit/4fc6bc394dffaf3ad375ff29cbb0a3eb9e4dbefc

Fixes: https://tracker.ceph.com/issues/62722
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
22 months agoMerge pull request #53230 from myoungwon/fix-cbj-overflow-bug
Yingxin [Tue, 19 Sep 2023 08:03:34 +0000 (16:03 +0800)]
Merge pull request #53230 from myoungwon/fix-cbj-overflow-bug

crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
22 months agoMerge PR #52631 into main
Milind Changire [Tue, 19 Sep 2023 07:51:09 +0000 (13:21 +0530)]
Merge PR #52631 into main

* refs/pull/52631/head:
mds: add debug logs to monitor ceph.dir.subvolume management
mds: dump subvolume flag for inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #52892 into main
Milind Changire [Tue, 19 Sep 2023 07:23:29 +0000 (12:53 +0530)]
Merge PR #52892 into main

* refs/pull/52892/head:
qa: add test to validate periodic checks by async threads
mgr/volumes: periodically check for async work

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
22 months agoMerge PR #52755 into main
Milind Changire [Tue, 19 Sep 2023 07:22:48 +0000 (12:52 +0530)]
Merge PR #52755 into main

* refs/pull/52755/head:
mds: adjust pre_segments_size for MDLog when trimming segments for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
22 months agoMerge PR #52692 into main
Milind Changire [Tue, 19 Sep 2023 07:21:52 +0000 (12:51 +0530)]
Merge PR #52692 into main

* refs/pull/52692/head:
qa/tasks/cephfs: reset the client_inject_fixed_oldest_tid after test

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53518 from likid0/patch-2
zdover23 [Mon, 18 Sep 2023 23:04:02 +0000 (09:04 +1000)]
Merge pull request #53518 from likid0/patch-2

doc/dev: Fix typos in cephfs-mirroring.rst and  deduplication.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agodoc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst 53518/head
Daniel Parkes [Mon, 18 Sep 2023 21:03:28 +0000 (23:03 +0200)]
doc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst

Typo Error in Doc cephfs-mirroring.rst , replace RAODS with RADOS
Typo Error in Doc deduplication.rst , replace RAODS with RADOS

Signed-off-by: Daniel Parkes <dparkes@redhat.com>
22 months agoMerge pull request #53502 from dang/wip-dang-cls-test
Casey Bodley [Mon, 18 Sep 2023 22:01:31 +0000 (23:01 +0100)]
Merge pull request #53502 from dang/wip-dang-cls-test

RGW - Fix cls test build on new gcc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #53508 from dang/wip-dang-posix-cache
Daniel Gryniewicz [Mon, 18 Sep 2023 18:36:58 +0000 (14:36 -0400)]
Merge pull request #53508 from dang/wip-dang-posix-cache

RGW - Add wait backoff to posix bucket cache test

Reviewed-by Matt Benjamin <mbenjamin@redhat.com>

22 months agoRGW - Add wait backoff to posix bucket cache test 53508/head
Daniel Gryniewicz [Mon, 18 Sep 2023 15:47:54 +0000 (11:47 -0400)]
RGW - Add wait backoff to posix bucket cache test

The CI appears to be really slow, and even a second of wait for inotify
sometimes fails.  Add an exponential backoff wait of up to ~25 seconds
to hopefully make the test pass reliably.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
22 months agorgw/sal: get_placement_target_names() returns void 53505/head
Casey Bodley [Mon, 18 Sep 2023 15:15:02 +0000 (11:15 -0400)]
rgw/sal: get_placement_target_names() returns void

the function returned an integer error code, but two callers were
incorrectly testing the return value as a boolean

the function just returns placement ids that are in-memory, so none of
the drivers have a failure case; change the return value to void

Fixes: https://tracker.ceph.com/issues/62771
Signed-off-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #53478 from EdwardVitor/cuiming_chinamobile
Casey Bodley [Mon, 18 Sep 2023 13:46:55 +0000 (14:46 +0100)]
Merge pull request #53478 from EdwardVitor/cuiming_chinamobile

auth:rectify a cmake compilation warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoRGW - Fix cls test build on new gcc 53502/head
Daniel Gryniewicz [Mon, 18 Sep 2023 12:50:53 +0000 (08:50 -0400)]
RGW - Fix cls test build on new gcc

The new encoder types broke building the cls test on newer gcc (13+) due
to undefined encoder/decoder.  Add the file that defines those to the
test.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
22 months agoMerge PR #53175 into main
Patrick Donnelly [Mon, 18 Sep 2023 12:37:16 +0000 (08:37 -0400)]
Merge PR #53175 into main

* refs/pull/53175/head:
qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoauth:rectify a cmake compilation warning 53478/head
cuiming_yewu [Fri, 15 Sep 2023 06:14:39 +0000 (14:14 +0800)]
auth:rectify a cmake compilation warning

rectify src/auth/cephx/CephxProtocol.cc 1 warning
with the variable 'ch' Used before initialized
auth/cephx/CephxProtocol.cc:595:57: warning: '*((void*)& ch +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
     msg.server_challenge_plus_one = ch.server_challenge + 1;
                                     ~~~~~~~~~~~~~~~~~~~~^~~

Signed-off-by: cuiming <cuiming_yewu@cmss.chinamobile.com>
22 months agoMerge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x
Anthony D'Atri [Sun, 17 Sep 2023 14:40:26 +0000 (10:40 -0400)]
Merge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x

doc/architecture: "Edit HA Auth" (one of several)

22 months agodoc/architecture: "Edit HA Auth" (one of several) 53490/head
Zac Dover [Sun, 17 Sep 2023 08:56:40 +0000 (18:56 +1000)]
doc/architecture: "Edit HA Auth" (one of several)

Edit "High Availability Authentication" in doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #53487 from zdover23/wip-doc-2023-09-16-architecture-5-of-x
Anthony D'Atri [Sat, 16 Sep 2023 16:46:58 +0000 (12:46 -0400)]
Merge pull request #53487 from zdover23/wip-doc-2023-09-16-architecture-5-of-x

doc/architecture: Edit "HA Auth"

22 months agodoc/architecture: Edit "HA Auth" 53487/head
Zac Dover [Sat, 16 Sep 2023 12:27:29 +0000 (22:27 +1000)]
doc/architecture: Edit "HA Auth"

Edit "High Availability Authentication" in doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge PR #52638 into main
Patrick Donnelly [Fri, 15 Sep 2023 18:38:02 +0000 (14:38 -0400)]
Merge PR #52638 into main

* refs/pull/52638/head:
mds: flush monc log before abort
qa: check for expected cluster log message
qa: ignore expected cluster warning from damage tests

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
22 months agoMerge PR #52199 into main
Patrick Donnelly [Fri, 15 Sep 2023 16:11:37 +0000 (12:11 -0400)]
Merge PR #52199 into main

* refs/pull/52199/head:
mds: continue linking if targeti is temporarily located in stray dir
Revert "mds: wait unlink to finish to avoid conflict when creating same dentries"
Revert "mds: clear the STATE_UNLINKING state when the unlink fails"
Revert "mds: wait reintegrate to finish when unlinking"
Revert "mds: notify the waiters in replica MDSs"
Revert "mds: wait the linkmerge/migrate to finish after unlink"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
22 months agoMerge pull request #52933 from dang/wip-dang-posix-driver
Daniel Gryniewicz [Fri, 15 Sep 2023 12:58:52 +0000 (08:58 -0400)]
Merge pull request #52933 from dang/wip-dang-posix-driver

RGW - Add POSIX Driver

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
22 months agoMerge pull request #53296 from phlogistonjohn/jjm-fix-git-ls-files 52810/head
Adam King [Thu, 14 Sep 2023 17:10:16 +0000 (13:10 -0400)]
Merge pull request #53296 from phlogistonjohn/jjm-fix-git-ls-files

cephadm: remove duplicates when listing files in scan

Reviewed-by: Adam King <adking@redhat.com>
22 months agoRGW - Add POSIX Driver 52933/head
Daniel Gryniewicz [Tue, 24 Jan 2023 15:02:07 +0000 (10:02 -0500)]
RGW - Add POSIX Driver

This is the MVP for a driver for RGW that operates on top of a POSIX
filesystem.  It supports get, put, list, copy, multipart, external
access via the filesystem itself, and ordered bucket listings via an
LRU-based cache.

Note that this is currently a Filter, indended to run on top of dbstore.
This is because it currently doesn't have any User implementation, so it
depends on dbstore's User.  Everything else is implemented in
POSIXDriver.  Once there is a User implementation, this will become a
Store, instead of a Filter.

Commit messages from bucket listing cache:

  rgw/posixdriver: recycle lmdb database handles as required

    While LMDB workflows often do not close/return database handles,
    ours continually reuses them.  This requires us to close each
    handle (atomically) when a cache entry is recycled.

  rgw/posixdriver: don't instantiate bucket cache entries from notify events

  rgw/posixdriver: incorporate lmdb-safe for now

    The current inclusion is based on https://github.com/Martchus/lmdb-safe,
    which is actively maintained but currently has some packaging issues the
    author has agreed to accept fixes for.

    For now, skip the submodule to save time and remove an external dependency.

  rgw/posixdriver: fix listing of cached, empty bucket

    * check lmdb enumeration result in all cases and w/better style
    * add unit test for enumeration of an empty cached directory

  rgw/posixdriver: nest lmdbs in a directory under the dbroot path to avoid cleanup issues

  rgw/posixdriver: refactor for posix integration

    * Derive BucketCache types as templates on a SAL driver and SAL
      bucket pair.

    * Integrate cache fills as callbacks into SAL layer (or mock, for
      tests)

    * Renaming and cleanups

  rgw/posixdriver: add bucket cache implementation and tests

    Adds free-standing cache of buckets and object names, with
    bucket names (and listing attributes, upcoming) managed in
    a hashed set of lmdb databases, which provides ordering and
    a high-performance listing cache.

    An framework for notification on new object creation (e.g.,
    outside S3 workflow) is provided, and a Linux implementation
    using inotify.

    FindLMDB.cmake taken with attribution and license.

  rgw/posixdriver: add zpp_bits serialization (FAST)

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
22 months agocephadm: replace get_data_dir with DaemonIdentity.data_dir 53467/head
John Mulligan [Tue, 12 Sep 2023 18:23:06 +0000 (14:23 -0400)]
cephadm: replace get_data_dir with DaemonIdentity.data_dir

Replace the near trivial get_data_dir call with a method on
the DaemonIdentity type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: add a data_dir method to DaemonIdentity
John Mulligan [Tue, 12 Sep 2023 18:22:25 +0000 (14:22 -0400)]
cephadm: add a data_dir method to DaemonIdentity

This will replace `get_data_dir` in a future commit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #52690 from phlogistonjohn/jjm-cephadm-daemon-ident
Adam King [Thu, 14 Sep 2023 15:23:38 +0000 (11:23 -0400)]
Merge pull request #52690 from phlogistonjohn/jjm-cephadm-daemon-ident

cephadm: refactor code to make use of DaemonIdentity type

Reviewed-by: Adam King <adking@redhat.com>
22 months agoMerge pull request #53450 from zdover23/wip-doc-2023-09-14-man-ceph-monstore-tool
zdover23 [Thu, 14 Sep 2023 11:44:15 +0000 (21:44 +1000)]
Merge pull request #53450 from zdover23/wip-doc-2023-09-14-man-ceph-monstore-tool

doc/man: edit ceph-monstore-tool.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agoMerge pull request #53426 from ronen-fr/wip-rf-fshut
Ronen Friedman [Thu, 14 Sep 2023 10:33:46 +0000 (13:33 +0300)]
Merge pull request #53426 from ronen-fr/wip-rf-fshut

osd: cleaning stop_for_fast_shutdown()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
22 months agoMerge PR #53070 into main
Venky Shankar [Thu, 14 Sep 2023 10:09:07 +0000 (15:39 +0530)]
Merge PR #53070 into main

* refs/pull/53070/head:
PendingReleaseNotes: add note about 'm' and 'M' periods
doc/cephfs: add note about monthly scheduling and retention period
qa: update minute period multiplier to m
qa: change minute duration specifier from M to m
mgr/snap_schedule: change minute duration specifier from M to m

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53433 from myoungwon/fix-53232
Yingxin [Thu, 14 Sep 2023 07:15:24 +0000 (15:15 +0800)]
Merge pull request #53433 from myoungwon/fix-53232

crimson/tools/perf_staged_fltree: fix compile error

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Aravind Ramesh <aravind.ramesh@wdc.com>
22 months agoMerge PR #51891 into main
Venky Shankar [Thu, 14 Sep 2023 03:59:07 +0000 (09:29 +0530)]
Merge PR #51891 into main

* refs/pull/51891/head:
qa: fix cephfs-top tests for the new fields
cephfs-top: include the missing fields in --dump output

Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
22 months agoMerge pull request #53444 from zdover23/wip-doc-2023-09-14-architecture-4-of-x
zdover23 [Thu, 14 Sep 2023 03:27:52 +0000 (13:27 +1000)]
Merge pull request #53444 from zdover23/wip-doc-2023-09-14-architecture-4-of-x

doc/architecture: edit "High Avail. Monitors"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agodoc/man: edit ceph-monstore-tool.rst 53450/head
Zac Dover [Thu, 14 Sep 2023 03:24:11 +0000 (13:24 +1000)]
doc/man: edit ceph-monstore-tool.rst

Clean up the English in doc/man/8/ceph-monstore-tool.rst. Put all
commands in the imperative mood; make sentences more direct.

Follows https://github.com/ceph/ceph/pull/52860.

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agodoc/architecture: edit "High Avail. Monitors" 53444/head
Zac Dover [Wed, 13 Sep 2023 14:09:45 +0000 (00:09 +1000)]
doc/architecture: edit "High Avail. Monitors"

Improve the sentence structure in the "High Availability Monitors"
section of doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #53178 from aclamk/wip-aclamk-bs-esb-5-enabled
Radoslaw Zarzynski [Wed, 13 Sep 2023 19:45:47 +0000 (21:45 +0200)]
Merge pull request #53178 from aclamk/wip-aclamk-bs-esb-5-enabled

bluestore: Elastic Shared Blobs 5 - enabled, final

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com
22 months agocephadm: fix an error running iscsi services 52690/head
John Mulligan [Wed, 13 Sep 2023 19:44:40 +0000 (15:44 -0400)]
cephadm: fix an error running iscsi services

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #52793 from linuxbox2/wip-fix-test-librgw-xattr
Matt Benjamin [Wed, 13 Sep 2023 16:57:52 +0000 (12:57 -0400)]
Merge pull request #52793 from linuxbox2/wip-fix-test-librgw-xattr

rgw/file:  fix build of librgw_file_xattr.cc unit test

22 months agocephadm: fix cephadm binary mount when --shared_ceph_folder is used 53415/head
Redouane Kachach [Tue, 12 Sep 2023 09:36:16 +0000 (11:36 +0200)]
cephadm: fix cephadm binary mount when --shared_ceph_folder is used
Fixes: https://tracker.ceph.com/issues/62814
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
22 months agoMerge pull request #51368 from dang/wip-dang-zipper-motr
Daniel Gryniewicz [Wed, 13 Sep 2023 13:35:39 +0000 (09:35 -0400)]
Merge pull request #51368 from dang/wip-dang-zipper-motr

Fix MOTR build, and move MOTR and DAOS to proper location

22 months agoAdding service_name and daemon_type and fixing mypy issues 52251/head
Redouane Kachach [Wed, 13 Sep 2023 11:46:42 +0000 (13:46 +0200)]
Adding service_name and daemon_type and fixing mypy issues

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
22 months agoMerge pull request #53391 from shreyanshjain7174/bucket_quota
Ali Maredia [Wed, 13 Sep 2023 11:46:19 +0000 (07:46 -0400)]
Merge pull request #53391 from shreyanshjain7174/bucket_quota

RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage

Reviewed-by: Ali Maredia <amaredia@redhat.com>
22 months agocrimson/tools/perf_staged_fltree: fix compile error 53433/head
Myoungwon Oh [Wed, 13 Sep 2023 05:16:16 +0000 (14:16 +0900)]
crimson/tools/perf_staged_fltree: fix compile error

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
22 months agoMerge pull request #53423 from zdover23/wip-doc-2023-09-12-architecture-3-of-x
zdover23 [Wed, 13 Sep 2023 07:07:26 +0000 (17:07 +1000)]
Merge pull request #53423 from zdover23/wip-doc-2023-09-12-architecture-3-of-x

doc/architecture: edit "Cluster Map"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agoMerge PR #53153 into main
Venky Shankar [Wed, 13 Sep 2023 05:40:59 +0000 (11:10 +0530)]
Merge PR #53153 into main

* refs/pull/53153/head:
Revert "mds: align quota.max_bytes to 4MB or 4KB"

Reviewed-by: Xiubo Li <xiubli@redhat.com>
22 months agoMerge pull request #53370 from wowly/rgw-cache-nginx-if-match-header
Casey Bodley [Tue, 12 Sep 2023 18:27:00 +0000 (19:27 +0100)]
Merge pull request #53370 from wowly/rgw-cache-nginx-if-match-header

examples: fixed errors with the if-match header in the examples of rgw-cache nginx configs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agodoc/architecture: edit "Cluster Map" 53423/head
Zac Dover [Tue, 12 Sep 2023 11:17:40 +0000 (21:17 +1000)]
doc/architecture: edit "Cluster Map"

Edit the section "Cluster Map" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #53422 from zdover23/wip-doc-2023-09-12-glossary-primary-affinity
Anthony D'Atri [Tue, 12 Sep 2023 16:00:08 +0000 (12:00 -0400)]
Merge pull request #53422 from zdover23/wip-doc-2023-09-12-glossary-primary-affinity

doc/glossary: add "primary affinity" to glossary

22 months agoosd: cleaning stop_for_fast_shutdown() 53426/head
Ronen Friedman [Tue, 12 Sep 2023 13:15:40 +0000 (08:15 -0500)]
osd: cleaning stop_for_fast_shutdown()

Removed unsued variables to prevent compiler warnings.
Protected the shard lock.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoMerge pull request #52407 from rhcs-dashboard/osd-op-scraper
Adam Kupczyk [Tue, 12 Sep 2023 15:47:52 +0000 (17:47 +0200)]
Merge pull request #52407 from rhcs-dashboard/osd-op-scraper

os/bluestore: osd scrape read/write operations

22 months agoos/bluestore: osd scrape read/write operations 52407/head
Pere Diaz Bou [Wed, 12 Jul 2023 11:05:17 +0000 (13:05 +0200)]
os/bluestore: osd scrape read/write operations

To run:
```bash
../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph
```

Example:
```bash
$ ../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph                                                                           [13:06:39]
collecting from osd. 0
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~4194304 4:c4b97d2c:::be151868021c187609fb97e4c22ae1f9:head 4.3 2023-07-12T13:06:34.959082+0200 client.4831.0:351
write 0~4194304 4:d392075e:::b068dce954dca8ca56d72e02e87f39c8:head 4.b 2023-07-12T13:06:35.093439+0200 client.4831.0:358
write 0~4194304 4:bc1ac643:::7b815bd25a40695c891dc06bdf2b65be:head 4.1d 2023-07-12T13:06:35.128097+0200 client.4831.0:360
write 0~4194304 4:d0d21bd5:::cc531199dd2707370bb461cce82bdef6:head 4.b 2023-07-12T13:06:36.023082+0200 client.4831.0:367
write 0~4194304 4:a9abc92c:::0703d6f43d14bc33d69f94ecb96c522c:head 4.15 2023-07-12T13:06:36.367886+0200 client.4831.0:369
write 0~4194304 4:9b4684ea:::d5856dc0ecad96f811012e0937d5e0ac:head 4.19 2023-07-12T13:06:36.654584+0200 client.4831.0:370
write 0~4194304 4:d15b8c87:::c3344e5de8c8936876965d9fb056ad89:head 4.b 2023-07-12T13:06:36.835690+0200 client.4831.0:371
write 0~4194304 4:ad41de2a:::8a1e2743b577a67d6bc8a1d514391b53:head 4.15 2023-07-12T13:06:37.102741+0200 client.4831.0:373
write 0~4194304 4:fc35d54f:::39cd961e7e33680f4bfd1ca0fe43dda4:head 4.1f 2023-07-12T13:06:38.047612+0200 client.4831.0:377

collecting from osd. 1
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~4194304 4:4c9ed2e8:::86206ccc35d8aceaf78670a751a6b550:head 4.12 2023-07-12T13:06:34.771423+0200 client.4831.0:340
write 0~4194304 4:106d9524:::fe7534cc604b0eb50498cb23aaca9161:head 4.8 2023-07-12T13:06:35.128105+0200 client.4831.0:359
write 0~4194304 4:a6e3fdb1:::2692491bfc4c584ca5b1efcb896936c2:head 4.5 2023-07-12T13:06:35.567070+0200 client.4831.0:363
write 0~4194304 4:2d9a471a:::ba51255646d00c496c2638bad03e0869:head 4.14 2023-07-12T13:06:35.568290+0200 client.4831.0:364
write 0~4194304 4:2b1381b1:::c58fa31e5d3a4f5550ee143bcb07a49f:head 4.14 2023-07-12T13:06:35.764699+0200 client.4831.0:365
write 0~4194304 4:a22e158f:::b2116e08700da18595b96d58120fe81c:head 4.5 2023-07-12T13:06:36.023101+0200 client.4831.0:366
```

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
22 months agoMerge pull request #53419 from AliMasarweh/wip-alimasa-persistent-q-test
Ali Masarwa [Tue, 12 Sep 2023 13:51:25 +0000 (16:51 +0300)]
Merge pull request #53419 from AliMasarweh/wip-alimasa-persistent-q-test

RGW | bucket notification: adding more sleeping time to prevent test from failing

Reviewed-by: yuvalif <ylifshit@redhat.com>
22 months agoqa: fix cephfs-top tests for the new fields 51891/head
Jos Collin [Fri, 2 Jun 2023 11:09:49 +0000 (16:39 +0530)]
qa: fix cephfs-top tests for the new fields

Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
22 months agocephfs-top: include the missing fields in --dump output
Jos Collin [Fri, 2 Jun 2023 11:08:29 +0000 (16:38 +0530)]
cephfs-top: include the missing fields in --dump output

Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
22 months agodoc/glossary: add "primary affinity" to glossary 53422/head
Zac Dover [Tue, 12 Sep 2023 10:56:03 +0000 (20:56 +1000)]
doc/glossary: add "primary affinity" to glossary

Add "Primary Affinity" to the glossary and link to the information in
the RADOS documentation that (more or less) explains it.

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoRGW | bucket notification: adding more sleeping time to prevent test from failing 53419/head
Ali Masarwa [Tue, 12 Sep 2023 10:31:31 +0000 (13:31 +0300)]
RGW | bucket notification: adding more sleeping time to prevent test from failing

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
22 months agomgr/dashboard: align charts of landing page 53411/head
Pedro Gonzalez Gomez [Tue, 12 Sep 2023 08:08:38 +0000 (10:08 +0200)]
mgr/dashboard: align charts of landing page

Fixes: https://tracker.ceph.com/issues/62813
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
22 months agoPendingReleaseNotes: add note about 'm' and 'M' periods 53070/head
Milind Changire [Tue, 12 Sep 2023 08:39:39 +0000 (14:09 +0530)]
PendingReleaseNotes: add note about 'm' and 'M' periods

Signed-off-by: Milind Changire <mchangir@redhat.com>
22 months agodoc/cephfs: add note about monthly scheduling and retention period
Milind Changire [Thu, 24 Aug 2023 02:19:43 +0000 (07:49 +0530)]
doc/cephfs: add note about monthly scheduling and retention period

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
22 months agoqa: update minute period multiplier to m
Milind Changire [Tue, 12 Sep 2023 08:30:33 +0000 (14:00 +0530)]
qa: update minute period multiplier to m

Signed-off-by: Milind Changire <mchangir@redhat.com>
22 months agoqa: change minute duration specifier from M to m
Milind Changire [Thu, 24 Aug 2023 02:18:17 +0000 (07:48 +0530)]
qa: change minute duration specifier from M to m

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
22 months agomgr/snap_schedule: change minute duration specifier from M to m
Milind Changire [Thu, 24 Aug 2023 02:17:46 +0000 (07:47 +0530)]
mgr/snap_schedule: change minute duration specifier from M to m

Problem:
As per the issue tracker, the period spec specifier 'M' is not
consistent with what is used elsewhere, like the period specifiers
displayed in the 'ceph status' command output.
The 'M' period specifier is used as a 'minute' level period specifier by
the cephfs team.
The issue reporter suggests to use 'M' as a 'month' period specifier.

Solution:
Since the 'minute' level period specifer, 'M', is used internally by
the development team, it is failrly easy to swap the 'minute' ('M')
level and 'month' ('m') level period specifers to finally mean that 'm'
implies 'minute' level period and 'M' implies 'month' level period.
Also, since this is the first time that somebody has ever reported that
neither the 'M' nor the 'm' level specifiers work in production, it is a
good idea to fix them once and for all.

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>