]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge PR #42319 into master
Sage Weil [Tue, 13 Jul 2021 22:20:21 +0000 (18:20 -0400)]
Merge PR #42319 into master

* refs/pull/42319/head:
qa/tasks/rebuild_mondb: fix rebuild vs logmonitor external_log_to

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42294 from neha-ojha/wip-51101
Neha Ojha [Tue, 13 Jul 2021 21:10:43 +0000 (14:10 -0700)]
Merge pull request #42294 from neha-ojha/wip-51101

qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #42278 into master
Sage Weil [Tue, 13 Jul 2021 20:26:33 +0000 (16:26 -0400)]
Merge PR #42278 into master

* refs/pull/42278/head:
doc/mgr/nfs: update about RGW exports
doc/cephfs/nfs: update about nfs module
doc/mgr/nfs: update cephfs export create command about client and squash arguments
doc/mgr/nfs: update nfs links
doc/mgr/nfs: add missing cluster_id to export info command
doc/cephfs: move nfs doc under mgr docs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa/tasks/rebuild_mondb: fix rebuild vs logmonitor external_log_to 42319/head
Sage Weil [Tue, 13 Jul 2021 18:44:57 +0000 (14:44 -0400)]
qa/tasks/rebuild_mondb: fix rebuild vs logmonitor external_log_to

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #42301 from tchaikov/wip-doc-crimson-nbd
Kefu Chai [Tue, 13 Jul 2021 17:04:07 +0000 (01:04 +0800)]
Merge pull request #42301 from tchaikov/wip-doc-crimson-nbd

doc/dev/crimson: add a section for testing with crimson-store-nbd

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #42305 from rhcs-dashboard/remove-rgw-frontend-ssl-key
Ernesto Puerta [Tue, 13 Jul 2021 16:54:37 +0000 (18:54 +0200)]
Merge pull request #42305 from rhcs-dashboard/remove-rgw-frontend-ssl-key

mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agodoc/dev/crimson: add a section for testing with crimson-store-nbd 42301/head
Kefu Chai [Tue, 13 Jul 2021 10:01:37 +0000 (18:01 +0800)]
doc/dev/crimson: add a section for testing with crimson-store-nbd

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #42227 into master
Sage Weil [Tue, 13 Jul 2021 16:04:55 +0000 (12:04 -0400)]
Merge PR #42227 into master

* refs/pull/42227/head:
doc/man/8/cephadm: add --log-to-file (and --single-host-defaults)
cephadm: add bootstrap --log-to-file option

Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge PR #42014 into master
Sage Weil [Tue, 13 Jul 2021 16:04:38 +0000 (12:04 -0400)]
Merge PR #42014 into master

* refs/pull/42014/head:
mon/LogMonitor: make 'ceph log last' work with new storage
mon/LogMonitor: transition log entries on upgrade to quincy
mon/LogMonitor: store logentries in separate keys
mon/LogMonitor: replace semi-broken pending_summary with pending_keys
mon/LogMonitor: externally log events from when we were out of quorum
mon/LogMonitor: factor logging to file out of update_from_paxos
mon: funnel SIGHUP handler through handle_mon_signal
common/LogEntry: add some constness
mon/LogMonitor: log mkfs in cluster channel
common/LRUSet: combine lru and hash-based lookup
mon/LogMonitor: remove stray derr

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/dev/crimson: add a missing not
Kefu Chai [Tue, 13 Jul 2021 12:06:00 +0000 (20:06 +0800)]
doc/dev/crimson: add a missing not

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42312 from rzarzynski/wip-crimson-boot_epoch-in-MOSDBoot
Kefu Chai [Tue, 13 Jul 2021 15:39:11 +0000 (23:39 +0800)]
Merge pull request #42312 from rzarzynski/wip-crimson-boot_epoch-in-MOSDBoot

crimson/osd: use boot_epoch for MOSDBoot.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42313 from rzarzynski/wip-crimson-drop-temporary-from-pg
Kefu Chai [Tue, 13 Jul 2021 15:36:15 +0000 (23:36 +0800)]
Merge pull request #42313 from rzarzynski/wip-crimson-drop-temporary-from-pg

crimson/osd: drop a temporary in PG::handle_initialize().

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3 42294/head
Neha Ojha [Mon, 12 Jul 2021 22:25:55 +0000 (22:25 +0000)]
qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3

To workaround the libarchive dependency issue seen with centos 8, which
has been causing consistent failures like

```
2021-06-04T04:52:51.147 INFO:tasks.workunit.client.0.smithi071.stdout:Installed:
2021-06-04T04:52:51.148 INFO:tasks.workunit.client.0.smithi071.stdout:  cmake-3.18.2-9.el8.x86_64                 cmake-data-3.18.2-9.el8.noarch
...
2021-06-04T04:52:57.554 INFO:tasks.workunit.client.0.smithi071.stderr:+ cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_TESTS=ON -DWITH_LIBRADOS=ON -DWITH_SNAPPY=ON -DWITH_GFLAGS=OFF -DFAIL_ON_WARNINGS=OFF ..
2021-06-04T04:52:57.579 DEBUG:teuthology.orchestra.run:got remote process result: 127
2021-06-04T04:52:57.580 INFO:tasks.workunit.client.0.smithi071.stderr:cmake: symbol lookup error: cmake: undefined symbol: archive_write_add_filter_zstd
```
More details in https://tracker.ceph.com/issues/51101#note-5

Fixes: https://tracker.ceph.com/issues/51101
Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42303 from tchaikov/wip-crimson-nbd-graceful-shutdown
Kefu Chai [Tue, 13 Jul 2021 15:08:45 +0000 (23:08 +0800)]
Merge pull request #42303 from tchaikov/wip-crimson-nbd-graceful-shutdown

crimson/tools/store_nbd: fixes to implement graceful shutdown

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge PR #42029 into master
Patrick Donnelly [Tue, 13 Jul 2021 15:07:22 +0000 (08:07 -0700)]
Merge PR #42029 into master

* refs/pull/42029/head:
vstart_runner: use FileNotFoundError when os.stat() fails

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge PR #42030 into master
Patrick Donnelly [Tue, 13 Jul 2021 15:05:25 +0000 (08:05 -0700)]
Merge PR #42030 into master

* refs/pull/42030/head:
vstart_runner: maintain log level when --debug is passed

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #42033 into master
Patrick Donnelly [Tue, 13 Jul 2021 15:03:18 +0000 (08:03 -0700)]
Merge PR #42033 into master

* refs/pull/42033/head:
vstart_runner: add log messages to vstart_runner.py

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocrimson/osd: drop a temporary in PG::handle_initialize(). 42313/head
Radoslaw Zarzynski [Mon, 12 Jul 2021 14:42:36 +0000 (14:42 +0000)]
crimson/osd: drop a temporary in PG::handle_initialize().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: use boot_epoch for MOSDBoot. 42312/head
Radoslaw Zarzynski [Wed, 7 Jul 2021 14:46:37 +0000 (14:46 +0000)]
crimson/osd: use boot_epoch for MOSDBoot.

For the sake of compliance and similarity with the classcial OSD.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #42267 from ifed01/wip-fix-missing-shared-blob
Kefu Chai [Tue, 13 Jul 2021 14:15:34 +0000 (22:15 +0800)]
Merge pull request #42267 from ifed01/wip-fix-missing-shared-blob

os/bluestore: fix erroneous SharedBlob record removal during repair.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42209 from sebastian-philipp/doc-dev-cephadm-define-vars
Sebastian Wagner [Tue, 13 Jul 2021 14:06:01 +0000 (16:06 +0200)]
Merge pull request #42209 from sebastian-philipp/doc-dev-cephadm-define-vars

doc/dev/cephadm: Define variables

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/dashboard: remove usage of 'rgw_frontend_ssl_key' 42305/head
Avan Thakkar [Tue, 13 Jul 2021 12:41:10 +0000 (18:11 +0530)]
mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'

Fixes: https://tracker.ceph.com/issues/51643
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Removing the usage of rgw_frontend_ssl_key from the rgw service form.

4 years agoMerge pull request #42274 from liewegas/cleanup-blkdev
Kefu Chai [Tue, 13 Jul 2021 13:29:29 +0000 (21:29 +0800)]
Merge pull request #42274 from liewegas/cleanup-blkdev

common/blkdev: remove stray debug output

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42206 from ronen-fr/wip-ronenf-list-object
Kefu Chai [Tue, 13 Jul 2021 13:23:43 +0000 (21:23 +0800)]
Merge pull request #42206 from ronen-fr/wip-ronenf-list-object

common/hobject: a minor fix and performance gain to hobjects listing

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #42210 from sebastian-philipp/options-ms-bind-port-max
Kefu Chai [Tue, 13 Jul 2021 12:25:15 +0000 (20:25 +0800)]
Merge pull request #42210 from sebastian-philipp/options-ms-bind-port-max

common/options: global.yaml: change ms_bind_port_max to 7568

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42268 from neha-ojha/wip-health-cleanup
Kefu Chai [Tue, 13 Jul 2021 12:24:31 +0000 (20:24 +0800)]
Merge pull request #42268 from neha-ojha/wip-health-cleanup

mon/PGMap: remove get_stuck_counts because there are no callers

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42235 from tchaikov/tools/kvstore-tool
Kefu Chai [Tue, 13 Jul 2021 12:23:07 +0000 (20:23 +0800)]
Merge pull request #42235 from tchaikov/tools/kvstore-tool

tools/kvstore_tool: add "std::" before ostream and string

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/tools/store_nbd: handle ECONNABORTED returned by accept() 42303/head
Kefu Chai [Tue, 13 Jul 2021 11:42:19 +0000 (19:42 +0800)]
crimson/tools/store_nbd: handle ECONNABORTED returned by accept()

if we abort accept() call, an ECONNABORTED is expected. and we should
handle it, otherwise unhandled exception will be noticed by seastar's
reactor. and it complains in that case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/tools/store_nbd: call segment_manager->close() after tm->close()
Kefu Chai [Tue, 13 Jul 2021 11:22:57 +0000 (19:22 +0800)]
crimson/tools/store_nbd: call segment_manager->close() after tm->close()

TransactionManager::close() calls into journal->close(), which in turn
calls BlockSegmentManager::segment_close(). and
SegmentStateTracker::write_out() is then called by
BlockSegmentManager::segment_close().

but BlockSegmentManager::close() closes the underlying seastar::file,
we are not able to write to the file after closing it.

in this change, to ensure that we can close a segment correctly in
TMDriver::close(), tm->close() is called before
segment_manager->close().

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/mgr/nfs: update about RGW exports 42278/head
Varsha Rao [Mon, 12 Jul 2021 07:31:49 +0000 (13:01 +0530)]
doc/mgr/nfs: update about RGW exports

This patch just moves the RGW exports created using nfs module to mgr/nfs
document. The RGW requirements will be updated in a different PR.

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs/nfs: update about nfs module
Varsha Rao [Mon, 12 Jul 2021 07:08:36 +0000 (12:38 +0530)]
doc/cephfs/nfs: update about nfs module

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/mgr/nfs: update cephfs export create command about client and squash arguments
Varsha Rao [Mon, 12 Jul 2021 06:29:59 +0000 (11:59 +0530)]
doc/mgr/nfs: update cephfs export create command about client and squash arguments

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/mgr/nfs: update nfs links
Varsha Rao [Tue, 29 Jun 2021 13:39:44 +0000 (19:09 +0530)]
doc/mgr/nfs: update nfs links

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/mgr/nfs: add missing cluster_id to export info command
Varsha Rao [Tue, 29 Jun 2021 13:08:03 +0000 (18:38 +0530)]
doc/mgr/nfs: add missing cluster_id to export info command

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs: move nfs doc under mgr docs
Varsha Rao [Tue, 29 Jun 2021 13:03:36 +0000 (18:33 +0530)]
doc/cephfs: move nfs doc under mgr docs

Fixes: https://tracker.ceph.com/issues/51428
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #42269 from zdover23/wip-doc-dev-essentials-irc-2021-07-10
zdover23 [Mon, 12 Jul 2021 20:20:29 +0000 (06:20 +1000)]
Merge pull request #42269 from zdover23/wip-doc-dev-essentials-irc-2021-07-10

doc/dev: add IRC information to dev guide

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #42113 from zdover23/wip-doc-upgrading-ceph-potential-problems...
zdover23 [Mon, 12 Jul 2021 20:19:30 +0000 (06:19 +1000)]
Merge pull request #42113 from zdover23/wip-doc-upgrading-ceph-potential-problems-2021-06-30

doc/cephadm: improve "Potential Problems"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #42092 from zdover23/wip-doc-upgrading-ceph-starting-the-upgrade...
zdover23 [Mon, 12 Jul 2021 20:18:56 +0000 (06:18 +1000)]
Merge pull request #42092 from zdover23/wip-doc-upgrading-ceph-starting-the-upgrade-2021-06-29

doc/cephadm: improving "Starting the Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agodoc/man/8/cephadm: add --log-to-file (and --single-host-defaults) 42227/head
Sage Weil [Mon, 12 Jul 2021 17:45:35 +0000 (13:45 -0400)]
doc/man/8/cephadm: add --log-to-file (and --single-host-defaults)

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #42217 from clwluvw/osd-device-details-grafana
Ernesto Puerta [Mon, 12 Jul 2021 15:30:19 +0000 (17:30 +0200)]
Merge pull request #42217 from clwluvw/osd-device-details-grafana

monitoring: fix Physical Device Latency unit

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: mykaul <NOT@FOUND>
Reviewed-by: p-se <NOT@FOUND>
4 years agoMerge pull request #41034 from nSedrickm/auth-storage-directive
Ernesto Puerta [Mon, 12 Jul 2021 15:03:30 +0000 (17:03 +0200)]
Merge pull request #41034 from nSedrickm/auth-storage-directive

mgr/dashboard: create directive for AuthStorage service

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/dashboard: create directive for AuthStorage service 41034/head
Ngwa Sedrick Meh [Mon, 26 Apr 2021 20:17:08 +0000 (21:17 +0100)]
mgr/dashboard: create directive for AuthStorage service

This commit adds a directive that can be used to conditionally display elements based on authorization/scopes criteria

Fixes: https://tracker.ceph.com/issues/47355
Signed-off-by: Ngwa Sedrick Meh <nsedrick101@gmail.com>
4 years agocommon/blkdev: remove stray debug output 42274/head
Sage Weil [Sun, 11 Jul 2021 17:24:00 +0000 (13:24 -0400)]
common/blkdev: remove stray debug output

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #41834 from grajoria/master
Gal Salomon [Sun, 11 Jul 2021 20:03:02 +0000 (23:03 +0300)]
Merge pull request #41834 from grajoria/master

doc: Correction and improvisation for Timestamp part of the doc

4 years agoMerge pull request #42272 from smithfarm/wip-51622
Kefu Chai [Sat, 10 Jul 2021 14:09:49 +0000 (22:09 +0800)]
Merge pull request #42272 from smithfarm/wip-51622

rpm: remove macro invocation from comment line

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agorpm: remove macro invocation from comment line 42272/head
Nathan Cutler [Sat, 10 Jul 2021 13:04:07 +0000 (15:04 +0200)]
rpm: remove macro invocation from comment line

In RPM spec files, comment lines should not include macro invocations,
because RPM can and will expand them, with unpredictable results.

Fixes: https://tracker.ceph.com/issues/51622
Signed-off-by: Nathan Cutler <ncutler@suse.com>
4 years agoMerge pull request #42212 from wjwithagen/wjw-fix-signal
Mark Kogan [Sat, 10 Jul 2021 12:09:35 +0000 (15:09 +0300)]
Merge pull request #42212 from wjwithagen/wjw-fix-signal

rgw: Use signaling compatible with POSIX

4 years agodoc/dev: add IRC information to dev guide 42269/head
Zac Dover [Fri, 9 Jul 2021 21:32:45 +0000 (07:32 +1000)]
doc/dev: add IRC information to dev guide

In days of yore, the Developer Guide linked to the
IRC page at ceph.io. After the 2021 rewriting of
ceph.io, a new era began, and that page was no
longer easily accessible (it could still be found
at old.ceph.com, but this was deprecated).

Anyway, that IRC information should be included in
the docs. This PR makes sure that the IRC
information is included in the docs.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agomon/PGMap: remove get_stuck_counts because there are no callers 42268/head
Neha Ojha [Fri, 9 Jul 2021 19:26:01 +0000 (19:26 +0000)]
mon/PGMap: remove get_stuck_counts because there are no callers

The only callers were removed in 729a08923fb4403b88baafa0ae75643cfefe6510,
as a part of the broader health reporting improvements.

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoos/bluestore: fix erroneous SharedBlob record removal during repair. 42267/head
Igor Fedotov [Fri, 9 Jul 2021 18:13:48 +0000 (21:13 +0300)]
os/bluestore: fix erroneous SharedBlob record removal during repair.

Fixes: https://tracker.ceph.com/issues/51619
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #42246 from ceph/1625
Ilya Dryomov [Fri, 9 Jul 2021 16:06:28 +0000 (18:06 +0200)]
Merge pull request #42246 from ceph/1625

doc: 16.2.5 Release Notes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agodoc: 16.2.5 Release Notes 42246/head
David Galloway [Thu, 8 Jul 2021 15:22:52 +0000 (11:22 -0400)]
doc: 16.2.5 Release Notes

Signed-off-by: David Galloway <dgallowa@redhat.com>
4 years agoMerge pull request #42253 from liewegas/doc-linode
Sage Weil [Fri, 9 Jul 2021 13:44:51 +0000 (08:44 -0500)]
Merge pull request #42253 from liewegas/doc-linode

doc/foundation: add linode

4 years agorgw: Use signaling compatible with POSIX 42212/head
Willem Jan Withagen [Wed, 7 Jul 2021 10:51:12 +0000 (12:51 +0200)]
rgw: Use signaling compatible with POSIX

FreeBSD does not have signal_t.
Linux specifies also to use `sigval` as POSIX compliant.

tracker: https://tracker.ceph.com/issues/51610
fixes: https://github.com/ceph/ceph/pull/36266
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #42112 from CongMinYin/fix-common-buffer-SIGABRT
Ilya Dryomov [Fri, 9 Jul 2021 11:31:41 +0000 (13:31 +0200)]
Merge pull request #42112 from CongMinYin/fix-common-buffer-SIGABRT

common/buffer: fix stack corruption in rebuild_aligned_size_and_memory()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agocommon/buffer: fix SIGABRT in rebuild_aligned_size_and_memory 42112/head
Yin Congmin [Wed, 30 Jun 2021 08:56:23 +0000 (16:56 +0800)]
common/buffer: fix SIGABRT in  rebuild_aligned_size_and_memory

There is such a bl, which needs to satisfy two conditions:
1)all ptrs' length sum except last ptr is aligned with 4K;
2)the length of last ptr is 0.
This bl will cause stack corruption when calling
bufferlist::rebuild_aligned_size_and_memory().

Deal with this special scenario in rebuild_aligned_size_and_memory() to
solve the bug. And added a specialtest-case to reproduce this scenario.

Fixes: https://tracker.ceph.com/issues/51419
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
4 years agoMerge pull request #42234 from sseshasa/wip-fix-standalone-cleanups
Sridhar Seshasayee [Fri, 9 Jul 2021 08:21:33 +0000 (13:51 +0530)]
Merge pull request #42234 from sseshasa/wip-fix-standalone-cleanups

qa/standalone: Add missing cleanups after completion of a subset of osd and scrub tests.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42034 from neha-ojha/wip-51354
Kefu Chai [Fri, 9 Jul 2021 05:04:36 +0000 (13:04 +0800)]
Merge pull request #42034 from neha-ojha/wip-51354

pybind/mgr/progress: fix type of reported_epoch

Reviewed-by: Kamoltat <ksirivad@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41965 from sg7801/master
Kefu Chai [Fri, 9 Jul 2021 01:49:06 +0000 (09:49 +0800)]
Merge pull request #41965 from sg7801/master

README.md: fix couple minor syntax issues

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/foundation: add linode 42253/head
Sage Weil [Thu, 8 Jul 2021 20:54:56 +0000 (15:54 -0500)]
doc/foundation: add linode

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoREADME.md: fix couple minor syntax issues 41965/head
Srishti Guleria [Tue, 22 Jun 2021 05:48:09 +0000 (11:18 +0530)]
README.md: fix couple minor syntax issues

Signed-off-by: Srishti Guleria <guleriasrishti01@gmail.com>
4 years agoMerge pull request #42055 from leseb/crash-various-fixes
Sébastien Han [Thu, 8 Jul 2021 10:26:48 +0000 (12:26 +0200)]
Merge pull request #42055 from leseb/crash-various-fixes

src/ceph-crash.in: various enhancements and fixes

4 years agoqa/standalone: Add missing teardowns to a subset of osd-scrub-repair tests 42234/head
Sridhar Seshasayee [Wed, 7 Jul 2021 19:09:26 +0000 (00:39 +0530)]
qa/standalone: Add missing teardowns to a subset of osd-scrub-repair tests

Tests identified with missing teardown within osd-scrub-repair.sh:
  1. TEST_periodic_scrub_replicated()
  2. TEST_scrub_warning()
  3. TEST_request_scrub_priority()

Centralize setup and teardown within the run() function for all the tests.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
4 years agoqa/standalone: Add missing teardowns to a subset of osd tests
Sridhar Seshasayee [Wed, 7 Jul 2021 18:59:30 +0000 (00:29 +0530)]
qa/standalone: Add missing teardowns to a subset of osd tests

The following files and tests in them did not teardown the
cluster after a test completed.

1. osd/osd-force-create.sh
2. osd/osd-reuse-id.sh
3. osd/pg-split-merge.sh

This wouldn't cause issues if the tests are run individually. But when
running all the tests in the files mentioned above, it could introduce
unexpected test failures down the line. For e.g., multiple tests may
create pools with same name and if they are not cleaned up properly, this
could result in unexpected failures in a subsequent test.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
4 years agocommon/hobject: speed up non-printables handling when listing hobjects 42206/head
Ronen Friedman [Wed, 7 Jul 2021 08:36:27 +0000 (08:36 +0000)]
common/hobject: speed up non-printables handling when listing hobjects

Using charconv::to_chars().
Anecdotal speedup, handling an assorted mix of strings: 2.4X
(tested on my laptop)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #42236 from tchaikov/wip-crimson-no-protobuf
Kefu Chai [Thu, 8 Jul 2021 06:42:27 +0000 (14:42 +0800)]
Merge pull request #42236 from tchaikov/wip-crimson-no-protobuf

ceph.spec,debian: drop protobuf dependencies

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #42185 from qiuxinyidian/dev2
Kefu Chai [Thu, 8 Jul 2021 01:17:50 +0000 (09:17 +0800)]
Merge pull request #42185 from qiuxinyidian/dev2

cmake: set denc_plugin_dir with the full path

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoceph.spec,debian: drop protobuf dependencies 42236/head
Kefu Chai [Thu, 8 Jul 2021 00:49:02 +0000 (08:49 +0800)]
ceph.spec,debian: drop protobuf dependencies

since Seastar has dropped the protobuf dependencies, there is no
need to prepare them for building crimson anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoseastar: bump seastar to latest upstream master
Kefu Chai [Thu, 8 Jul 2021 00:48:13 +0000 (08:48 +0800)]
seastar: bump seastar to latest upstream master

to pick up the change to drop protobuf dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotools/kvstore_tool: add "std::" before ostream and string 42235/head
Kefu Chai [Thu, 8 Jul 2021 00:42:34 +0000 (08:42 +0800)]
tools/kvstore_tool: add "std::" before ostream and string

also use <iosfwd> instead of <iostream> for forward declaration of
std::ostream, so that we don't need to include the whole iostream
header file in kvstore_tool.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42141 from trociny/wip-38931
Xie Xingguo [Thu, 8 Jul 2021 00:33:58 +0000 (08:33 +0800)]
Merge pull request #42141 from trociny/wip-38931

osd: move down peers out from peer_purged

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
4 years agoMerge pull request #42164 from yuriw/wip-yuriw-crontab-master
Ilya Dryomov [Wed, 7 Jul 2021 22:57:55 +0000 (00:57 +0200)]
Merge pull request #42164 from yuriw/wip-yuriw-crontab-master

qa/tests: removed nautilus suites as EOL

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoqa/tests: removed nautilus suites as EOL, kept kcephfs and krbd suites 42164/head
Yuri Weinstein [Fri, 2 Jul 2021 17:54:39 +0000 (10:54 -0700)]
qa/tests: removed nautilus suites as EOL, kept kcephfs and krbd suites

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agocephadm: add bootstrap --log-to-file option
Sage Weil [Wed, 7 Jul 2021 16:14:36 +0000 (12:14 -0400)]
cephadm: add bootstrap --log-to-file option

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #42176 from tchaikov/wip-doc-option
Kefu Chai [Wed, 7 Jul 2021 13:58:30 +0000 (21:58 +0800)]
Merge pull request #42176 from tchaikov/wip-doc-option

doc/rados/troubleshooting/log-and-debug: document option using :confval:

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #42136 from dvanders/dvanders_lec_merging
Kefu Chai [Wed, 7 Jul 2021 13:22:53 +0000 (21:22 +0800)]
Merge pull request #42136 from dvanders/dvanders_lec_merging

mon/OSDMonitor: resize oversized Lec::epoch_by_pg, after PG merging, preventing osdmap trimming

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomonitoring: fix Physical Device Latency unit 42217/head
Seena Fallah [Wed, 7 Jul 2021 12:30:30 +0000 (17:00 +0430)]
monitoring: fix Physical Device Latency unit

Based on the expr it should be seconds

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
4 years agoMerge pull request #42190 from ivancich/wip-unused-store-warning
J. Eric Ivancich [Wed, 7 Jul 2021 12:30:09 +0000 (08:30 -0400)]
Merge pull request #42190 from ivancich/wip-unused-store-warning

rgw: remove unused store variable

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kalpesh Pandya <kapandya@redhat.com>
4 years agomon/OSDMonitor: account for PG merging in epoch_by_pg accounting 42136/head
Dan van der Ster [Thu, 1 Jul 2021 14:12:26 +0000 (16:12 +0200)]
mon/OSDMonitor: account for PG merging in epoch_by_pg accounting

After a pool has merged PGs, the epoch_by_pg accounting will refer
to osdmap epochs of PGs that no longer exist. We'll never again get
OSD beacons for these PGs, so the min epoch in epoch_by_pg will not
advance until the mon leader has restarted. The effect of this is
that osdmaps are not trimmed after a pool has undergone PG merging,
until the mon leader restarts. To fix, we unconditionally resize
epoch_by_pg to the pg_num of the pool during each beacon report.

Fixes: https://tracker.ceph.com/issues/48212
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
4 years agoMerge pull request #42040 from wjwithagen/wjw-wip-bluestore-choose_fdv3
Kefu Chai [Wed, 7 Jul 2021 11:23:48 +0000 (19:23 +0800)]
Merge pull request #42040 from wjwithagen/wjw-wip-bluestore-choose_fdv3

blk: use choose_fd for all filehandle references

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/options: global.yaml: change ms_bind_port_max to 7568 42210/head
Sebastian Wagner [Wed, 7 Jul 2021 09:59:12 +0000 (11:59 +0200)]
common/options: global.yaml: change ms_bind_port_max to 7568

If the cluster is set to have very dense nodes (>60 OSDs per host)
we have to make sure to assign sufficient ports for Ceph OSDs. The
current default (6800-7300) currently allows for no more than 62 OSDs
per host. For cluster with dense nodes like this we have to adjust the
setting "ms_bind_port_max" to a suitable value. Each OSD will consume
8 additional ports. For example, given a host that is set to run 96 OSDs,
768 ports will be needed. "ms_bind_port_max" should be set at least to
7568.

Fixes: https://tracker.ceph.com/issues/48292
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoblk/kernel: Only use file hint capabilities if available. 42040/head
Willem Jan Withagen [Wed, 7 Jul 2021 09:50:50 +0000 (11:50 +0200)]
blk/kernel: Only use file hint capabilities if available.

Without WRITE_LIFE capabilities, only one file is used.
And rocksdb sets this value also to > 0, so we need to catch this here
instead of trusting rocksdb to set write_hint.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoblk/kernel: reorganise and use fd in debug
Willem Jan Withagen [Thu, 1 Jul 2021 16:08:20 +0000 (18:08 +0200)]
blk/kernel: reorganise and use fd in debug

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoblk: use choose_fd for all filehandle references
Willem Jan Withagen [Sat, 26 Jun 2021 13:47:24 +0000 (15:47 +0200)]
blk: use choose_fd for all filehandle references

This is a leading part of the changes to implement bluestore on
FreeBSD. Without this invalid indexing in the descriptor arrays
will occur.

Creates a new enum: `blk_access_mode_t`
to describe `BUFFERED` and `DIRECT `mode access
with `choose_fd()` to get the correct file for the typed access
and adds
a pretty-printer
boolean convertor `blk_access_mode_t::buffermode(bool)`

This PR is a redo for PR #37258, since that one was lost
in rebasing errors. But the review notes are still there.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agodoc/dev/cephadm: Define variables 42209/head
Sebastian Wagner [Wed, 7 Jul 2021 09:47:59 +0000 (11:47 +0200)]
doc/dev/cephadm: Define variables

Fixes: https://tracker.ceph.com/issues/47142
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #42183 from SMIL-Infra/fix-ac-lang
Alfonso Martínez [Wed, 7 Jul 2021 09:15:58 +0000 (11:15 +0200)]
Merge pull request #42183 from SMIL-Infra/fix-ac-lang

mgr/dashboard: fix Accept-Language header parsing

4 years agoMerge pull request #36266 from amathuria/wip-rgw-d3n-cache-upstreaming
Mark Kogan [Wed, 7 Jul 2021 09:11:47 +0000 (12:11 +0300)]
Merge pull request #36266 from amathuria/wip-rgw-d3n-cache-upstreaming

rgw: D3N Cache changes for Upstream

4 years agoMerge pull request #42158 from tchaikov/wip-e2e-centos
Kefu Chai [Wed, 7 Jul 2021 08:37:08 +0000 (16:37 +0800)]
Merge pull request #42158 from tchaikov/wip-e2e-centos

qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>
4 years agoMerge pull request #42175 from tchaikov/wip-45457
Kefu Chai [Wed, 7 Jul 2021 08:19:41 +0000 (16:19 +0800)]
Merge pull request #42175 from tchaikov/wip-45457

log: set hostname and fsid for graylog

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #42205 from tchaikov/wip-doc-crimson-metrics
Kefu Chai [Wed, 7 Jul 2021 08:11:23 +0000 (16:11 +0800)]
Merge pull request #42205 from tchaikov/wip-doc-crimson-metrics

doc/dev/crimson: add a section for metrics

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #42179 from brucen1030/wip-fix-mon-crash-when-osd-pool-get
Kefu Chai [Wed, 7 Jul 2021 08:04:41 +0000 (16:04 +0800)]
Merge pull request #42179 from brucen1030/wip-fix-mon-crash-when-osd-pool-get

mon: return -EINVAL when handling unknown option in 'ceph osd pool get'

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42177 from sebastian-philipp/cephadm-ls-garbage
Kefu Chai [Wed, 7 Jul 2021 07:47:34 +0000 (15:47 +0800)]
Merge pull request #42177 from sebastian-philipp/cephadm-ls-garbage

mgr/cephadm: avoid saving daemons of unknown type

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/dev/crimson: add a section for metrics 42205/head
Kefu Chai [Wed, 7 Jul 2021 06:26:26 +0000 (14:26 +0800)]
doc/dev/crimson: add a section for metrics

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: set denc_plugin_dir with the full path 42185/head
zhipeng li [Mon, 5 Jul 2021 14:15:02 +0000 (22:15 +0800)]
cmake: set denc_plugin_dir with the full path

This commit changes denc_plugin_dir from relative to absolute path, so
that CEPH_DENC_MOD_DIR points to the absolute path and ceph-dencoder
is able to load the shared libraries from the specified path.

Fixes:https://tracker.ceph.com/issues/51519
Signed-off-by: zhipeng li <qiuxinyidian@gmail.com>
4 years agoMerge pull request #42168 from tchaikov/wip-mgr-unsigned-pgp
Kefu Chai [Wed, 7 Jul 2021 07:16:45 +0000 (15:16 +0800)]
Merge pull request #42168 from tchaikov/wip-mgr-unsigned-pgp

mgr/DaemonServer: use "unsigned" to represent the pgp num

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #42203 from tchaikov/wip-crimson-metrics
Kefu Chai [Wed, 7 Jul 2021 02:51:42 +0000 (10:51 +0800)]
Merge pull request #42203 from tchaikov/wip-crimson-metrics

crimson/admin: dump DERIVE metrics as signed integer

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/admin: dump DERIVE metrics as signed integer 42203/head
Kefu Chai [Wed, 7 Jul 2021 02:29:59 +0000 (10:29 +0800)]
crimson/admin: dump DERIVE metrics as signed integer

per include/seastar/core/metrics.hh,

// The value binding data types
enum class data_type : uint8_t {
    COUNTER, // unsigned int 64
    GAUGE, // double
    DERIVE, // signed int 64
    ABSOLUTE, // unsigned int 64
    HISTOGRAM,
};

the DERIVE values are signed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #42139 into master
Patrick Donnelly [Tue, 6 Jul 2021 20:08:32 +0000 (13:08 -0700)]
Merge PR #42139 into master

* refs/pull/42139/head:
src/pybind/mgr/mirroring/fs/snapshot_mirror.py: do not assume a cephfs-mirror daemon is always running

Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge PR #42137 into master
Patrick Donnelly [Tue, 6 Jul 2021 20:07:53 +0000 (13:07 -0700)]
Merge PR #42137 into master

* refs/pull/42137/head:
msg/async/net_handler: do not define variable unless it is used
mds/cephfs_features: print size_t using "%zu"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #42155 from yuvalif/wip-yuval-fix-51347
Yuval Lifshitz [Tue, 6 Jul 2021 19:02:02 +0000 (22:02 +0300)]
Merge pull request #42155 from yuvalif/wip-yuval-fix-51347

rgw/notifications: cache object size to avoid accessing invalid memoery