]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge PR #41786 into master 41599/head
Patrick Donnelly [Mon, 14 Jun 2021 02:57:37 +0000 (19:57 -0700)]
Merge PR #41786 into master

* refs/pull/41786/head:
mon/MonCommands: remove obsolete mds commands
mon/MonCommands: remove obsolete mon commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41580 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:57:05 +0000 (19:57 -0700)]
Merge PR #41580 into master

* refs/pull/41580/head:
cephfs: make ceph_fallocate() platform independent

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41482 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:56:34 +0000 (19:56 -0700)]
Merge PR #41482 into master

* refs/pull/41482/head:
qa: remove obsolete deactivate routines

Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge PR #41481 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:56:04 +0000 (19:56 -0700)]
Merge PR #41481 into master

* refs/pull/41481/head:
client: Add ceph.caps vxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge PR #41431 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:54:45 +0000 (19:54 -0700)]
Merge PR #41431 into master

* refs/pull/41431/head:
mds: progress the recover queue immediately after the inode is enqueued
mds: switch to mdr->get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41422 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:52:48 +0000 (19:52 -0700)]
Merge PR #41422 into master

* refs/pull/41422/head:
qa/tasks/cephfs/test_sessionmap: reap connections immediately
msg/async: configurable threshold for reaping dead connections

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41403 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:51:52 +0000 (19:51 -0700)]
Merge PR #41403 into master

* refs/pull/41403/head:
mgr/volumes: Add config to insert delay at the beginning of the clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40740 from batrick/i50278
Kefu Chai [Sun, 13 Jun 2021 15:30:20 +0000 (23:30 +0800)]
Merge pull request #40740 from batrick/i50278

pybind/mgr: add support for common sqlite3 databases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41809 from linuxbox2/wip-returnvec-64
Kefu Chai [Sun, 13 Jun 2021 15:27:58 +0000 (23:27 +0800)]
Merge pull request #41809 from linuxbox2/wip-returnvec-64

rados: increase osd_max_write_op_reply_len default to 64 bytes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #41746 from strenuous-life/wip-cephadm-osd-create
Kefu Chai [Sun, 13 Jun 2021 15:26:46 +0000 (23:26 +0800)]
Merge pull request #41746 from strenuous-life/wip-cephadm-osd-create

mgr/cephadm: When create osd which include db, _create_daemon will be…

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge PR #41781 into master
Sage Weil [Sat, 12 Jun 2021 13:02:19 +0000 (09:02 -0400)]
Merge PR #41781 into master

* refs/pull/41781/head:
mgr/mgr_module:delete invalid judgment

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41699 into master
Sage Weil [Sat, 12 Jun 2021 13:01:38 +0000 (09:01 -0400)]
Merge PR #41699 into master

* refs/pull/41699/head:
mgr/telemetry: pass leaderboard flag even w/o ident

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
4 years agoqa: choose victim pg from rbd pool 40740/head
Patrick Donnelly [Sat, 12 Jun 2021 03:04:53 +0000 (20:04 -0700)]
qa: choose victim pg from rbd pool

Right now scrub_test picks any pg in ceph. Unfortunately, it picked the
.mgr pool's only pg in [1]:

2021-05-16T11:36:35.035 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool rbd setomapval main.db-journal.0000000000000000 key val

Instead, only pick a pg in the rbd pool.

[1] /ceph/teuthology-archive/kchai-2021-05-16_11:19:39-rados-wip-kefu-testing-2021-05-16-1043-distro-basic-smithi/6117396/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa,pybind/mgr: allow disabling .mgr pool
Patrick Donnelly [Sat, 1 May 2021 02:31:35 +0000 (19:31 -0700)]
qa,pybind/mgr: allow disabling .mgr pool

This is mostly for testing: a lot of tests assume that there are no
existing pools. These tests relied on a config to turn off creating the
"device_health_metrics" pool which generally exists for any new Ceph
cluster. It would be better to make these tests tolerant of the new .mgr
pool but clearly there's a lot of these. So just convert the config to
make it work.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agodoc: document new .mgr pool
Patrick Donnelly [Tue, 13 Apr 2021 02:42:57 +0000 (19:42 -0700)]
doc: document new .mgr pool

And that pool names beginning with `.` are reserved names.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: add upgrade test for devicehealth
Patrick Donnelly [Mon, 19 Apr 2021 19:58:01 +0000 (12:58 -0700)]
qa: add upgrade test for devicehealth

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: fix api test failures
Patrick Donnelly [Sun, 11 Apr 2021 20:16:25 +0000 (13:16 -0700)]
qa: fix api test failures

"device_health_metrics" pool is gone -- .mgr pool is in.

I don't think the pool removal code in some test cases is necessary any
longer with recent changes to remove those warnings; so that code is
gone too.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agopybind/mgr/devicehealth: load legacy device metrics pool
Patrick Donnelly [Fri, 9 Apr 2021 23:56:53 +0000 (16:56 -0700)]
pybind/mgr/devicehealth: load legacy device metrics pool

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agopybind/mgr/devicehealth: update to store metrics in sqlite
Patrick Donnelly [Fri, 9 Apr 2021 22:00:06 +0000 (15:00 -0700)]
pybind/mgr/devicehealth: update to store metrics in sqlite

This commit just modifies the module to use the sqlite3 database for
storing health metrics. A followup commit loads the legacy pool's data.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agopybind/mgr: add sqlite3 db module helpers
Patrick Donnelly [Fri, 9 Apr 2021 21:58:47 +0000 (14:58 -0700)]
pybind/mgr: add sqlite3 db module helpers

This creates a new '.mgr' pool for storing a default sqlite3 database
for each mgr module. Each module's database is stored in:

        file:///.mgr:<mgr module name>/main.db?vfs=ceph

The "main.db" is the only one used presently but perhaps a module may
want extra databases for some reason. The module name is used for the
RADOS namespace.

Databases are versioned in a common table called MgrModuleKV using the
"__version" key. A mechanism is in place (SCHEMA_VERSIONED) to allow
modules to upgrade their databases over time in a consistent way.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40480 from kamoltat/wip-ksirivad-fix-bug-49988
Kefu Chai [Sat, 12 Jun 2021 00:37:35 +0000 (08:37 +0800)]
Merge pull request #40480 from kamoltat/wip-ksirivad-fix-bug-49988

pybind/mgr/progress: Disregard unreported pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #41772 from cyx1231st/wip-seastore-onode-tree-node-size
Kefu Chai [Sat, 12 Jun 2021 00:11:45 +0000 (08:11 +0800)]
Merge pull request #41772 from cyx1231st/wip-seastore-onode-tree-node-size

crimson/onode-staged-tree: extend tree node sizes to fit insert upper-bounds

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #39505 into master
Patrick Donnelly [Sat, 12 Jun 2021 00:10:41 +0000 (17:10 -0700)]
Merge PR #39505 into master

* refs/pull/39505/head:
qa: test nowsync option in kernel client workflows
qa: deep merge top level overrides for fuse/kclient

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge pull request #40952 from tchaikov/wip-deb-build-profile
Kefu Chai [Sat, 12 Jun 2021 00:08:47 +0000 (08:08 +0800)]
Merge pull request #40952 from tchaikov/wip-deb-build-profile

debian: use build-profile to manage build deps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41683 into master
Patrick Donnelly [Sat, 12 Jun 2021 00:07:06 +0000 (17:07 -0700)]
Merge PR #41683 into master

* refs/pull/41683/head:
qa: update RHEL to 8.4

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41550 from aaryanporwal/navbar-fix
Ernesto Puerta [Fri, 11 Jun 2021 17:46:15 +0000 (19:46 +0200)]
Merge pull request #41550 from aaryanporwal/navbar-fix

mgr/dashboard: fix for right sidebar nav icon not clickable

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agocrimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print 41772/head
Yingxin Cheng [Wed, 9 Jun 2021 03:06:28 +0000 (11:06 +0800)]
crimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: merge with single-value tail internal node
Yingxin Cheng [Wed, 9 Jun 2021 02:48:43 +0000 (10:48 +0800)]
crimson/onode-staged-tree: merge with single-value tail internal node

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: switch to and test larger node sizes
Yingxin Cheng [Tue, 8 Jun 2021 07:19:33 +0000 (15:19 +0800)]
crimson/onode-staged-tree: switch to and test larger node sizes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys
Yingxin Cheng [Tue, 8 Jun 2021 06:11:52 +0000 (14:11 +0800)]
crimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: add the missing retired validation
Yingxin Cheng [Tue, 8 Jun 2021 06:03:07 +0000 (14:03 +0800)]
crimson/onode-staged-tree: add the missing retired validation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: encode/decode stage_size instead
Yingxin Cheng [Tue, 8 Jun 2021 01:59:32 +0000 (09:59 +0800)]
crimson/onode-staged-tree: encode/decode stage_size instead

end-offset may risk overflow the node_offset_t, but stage_size will not.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: use extent_len_t and node_offset_t correctly
Yingxin Cheng [Tue, 8 Jun 2021 01:55:15 +0000 (09:55 +0800)]
crimson/onode-staged-tree: use extent_len_t and node_offset_t correctly

extent_len_t represents a value that may include the node size, but
node_offset_t cannot and may overflow. Also add validations when
try to cast a larger type to node_offset_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove unused code
Yingxin Cheng [Tue, 8 Jun 2021 01:42:34 +0000 (09:42 +0800)]
crimson/onode-staged-tree: remove unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: validate node size where possible
Yingxin Cheng [Tue, 8 Jun 2021 01:41:44 +0000 (09:41 +0800)]
crimson/onode-staged-tree: validate node size where possible

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE
Yingxin Cheng [Mon, 7 Jun 2021 05:57:13 +0000 (13:57 +0800)]
crimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE

Switch to run-time node_size.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: pass node_size to lower node stages
Yingxin Cheng [Mon, 7 Jun 2021 05:15:58 +0000 (13:15 +0800)]
crimson/onode-staged-tree: pass node_size to lower node stages

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove compile-time node SIZE
Yingxin Cheng [Mon, 7 Jun 2021 02:45:18 +0000 (10:45 +0800)]
crimson/onode-staged-tree: remove compile-time node SIZE

Switch to run-time node_extent_t::node_size instead.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup internal_fields_3_t template
Yingxin Cheng [Mon, 7 Jun 2021 02:43:31 +0000 (10:43 +0800)]
crimson/onode-staged-tree: cleanup internal_fields_3_t template

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: pass node_size into node_extent_t
Yingxin Cheng [Thu, 3 Jun 2021 02:28:50 +0000 (10:28 +0800)]
crimson/onode-staged-tree: pass node_size into node_extent_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: extend num_keys_t to support larger nodes
Yingxin Cheng [Wed, 2 Jun 2021 15:44:07 +0000 (23:44 +0800)]
crimson/onode-staged-tree: extend num_keys_t to support larger nodes

Also adjust the accroding layout related unit tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE
Yingxin Cheng [Wed, 2 Jun 2021 08:25:05 +0000 (16:25 +0800)]
crimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: check the maximum tree height
Yingxin Cheng [Wed, 2 Jun 2021 06:07:33 +0000 (14:07 +0800)]
crimson/onode-staged-tree: check the maximum tree height

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: validate safe-split from tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:36:11 +0000 (13:36 +0800)]
crimson/onode-staged-tree: validate safe-split from tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: add and use node sizes in tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:33:50 +0000 (13:33 +0800)]
crimson/onode-staged-tree: add and use node sizes in tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: implement size upper-bounds to value
Yingxin Cheng [Wed, 2 Jun 2021 05:18:48 +0000 (13:18 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: implement size upper-bounds to ns/oid
Yingxin Cheng [Wed, 2 Jun 2021 04:47:55 +0000 (12:47 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to ns/oid

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: introduce compile-time tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 03:52:33 +0000 (11:52 +0800)]
crimson/onode-staged-tree: introduce compile-time tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue
Yingxin Cheng [Mon, 31 May 2021 07:58:30 +0000 (15:58 +0800)]
crimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/seastore: introduce TransactionManager::read_extent(t, offset)
Yingxin Cheng [Mon, 31 May 2021 02:11:01 +0000 (10:11 +0800)]
crimson/seastore: introduce TransactionManager::read_extent(t, offset)

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: correct the node size equation
Yingxin Cheng [Wed, 26 May 2021 06:26:59 +0000 (14:26 +0800)]
crimson/onode-staged-tree: correct the node size equation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: try merge peer if node has one value
Yingxin Cheng [Wed, 26 May 2021 04:45:27 +0000 (12:45 +0800)]
crimson/onode-staged-tree: try merge peer if node has one value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup and introduce has_single_value()
Yingxin Cheng [Wed, 26 May 2021 04:44:13 +0000 (12:44 +0800)]
crimson/onode-staged-tree: cleanup and introduce has_single_value()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #41814 from tchaikov/wip-crimson-fmt-seq
Kefu Chai [Fri, 11 Jun 2021 11:59:46 +0000 (19:59 +0800)]
Merge pull request #41814 from tchaikov/wip-crimson-fmt-seq

crimson/osd: specialize fmt::formatter<> for OpSequencer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #41759 from rhcs-dashboard/wip-api_label-master
Ernesto Puerta [Fri, 11 Jun 2021 09:08:18 +0000 (11:08 +0200)]
Merge pull request #41759 from rhcs-dashboard/wip-api_label-master

.github/labeler: add api-change label

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41242 from zdover23/wip-doc-cephadm-monitoring-deploying-monitori...
Sebastian Wagner [Fri, 11 Jun 2021 08:45:47 +0000 (10:45 +0200)]
Merge pull request #41242 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-second-half-2021-05-10

doc/cephadm: rewrite "deploy. mon. w/cephadm" 2/2

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #41241 from zdover23/wip-doc-cephadm-monitoring-deploying-monitori...
Sebastian Wagner [Fri, 11 Jun 2021 08:45:04 +0000 (10:45 +0200)]
Merge pull request #41241 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-first-half-2021-05-10

doc/cephadm: rewrite "deploy. mon. w/cephadm"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agocrimson/osd: specialize fmt::formatter<> for OpSequencer 41814/head
Kefu Chai [Fri, 11 Jun 2021 06:18:22 +0000 (14:18 +0800)]
crimson/osd: specialize fmt::formatter<> for OpSequencer

so we can print it like fmt::format("{}", sequencer)

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/cephadm: When test osd active, it need to add "mock call" to make sure execute... 41746/head
jianglong01 [Fri, 11 Jun 2021 02:56:30 +0000 (10:56 +0800)]
mgr/cephadm: When test osd active, it need to add "mock call" to make sure execute correctly

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
4 years agoMerge pull request #41812 from agayev/zoned-fix-block-size
Kefu Chai [Fri, 11 Jun 2021 00:41:36 +0000 (08:41 +0800)]
Merge pull request #41812 from agayev/zoned-fix-block-size

os/bluestore: Fix the size of the block in the Allocator base class t…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #41744 from tchaikov/wip-cmake-no-python-bindings-on-wnin32
Kefu Chai [Fri, 11 Jun 2021 00:32:30 +0000 (08:32 +0800)]
Merge pull request #41744 from tchaikov/wip-cmake-no-python-bindings-on-wnin32

cmake: vstart do not depend on cython_rbd if WIN32

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41442 into master
Patrick Donnelly [Thu, 10 Jun 2021 21:53:21 +0000 (14:53 -0700)]
Merge PR #41442 into master

* refs/pull/41442/head:
doc/cephfs/nfs: add ganesha configuration hierarchy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge PR #41797 into master
Patrick Donnelly [Thu, 10 Jun 2021 20:55:12 +0000 (13:55 -0700)]
Merge PR #41797 into master

* refs/pull/41797/head:
qa: update cephfs-shell distro to ubuntu 20.04

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41575 from ifed01/wip-ifed-fix-no-osd-daemonperf
Igor Fedotov [Thu, 10 Jun 2021 20:32:00 +0000 (23:32 +0300)]
Merge pull request #41575 from ifed01/wip-ifed-fix-no-osd-daemonperf

common/PriorityCache: low perf counters priorities for submodules.

Reviewed-by: Mark Nelson <mnelson@readhat.com>
4 years agoos/bluestore: Fix the size of the block in the Allocator base class to avoid 41812/head
Abutalib Aghayev [Thu, 10 Jun 2021 19:59:45 +0000 (15:59 -0400)]
os/bluestore: Fix the size of the block in the Allocator base class to avoid
the confusing log message about the block size.

Signed-off-by: Abutalib Aghayev <agayev@psu.edu>
4 years agorados: increase osd_max_write_op_reply_len default to 64 bytes 41809/head
Matt Benjamin [Thu, 10 Jun 2021 14:27:58 +0000 (10:27 -0400)]
rados: increase osd_max_write_op_reply_len default to 64 bytes

Agreed in #ceph-devel on 6/10.  The current controlling
rationale is that the default value should be sufficient to
marshall a SHA-512 checksum.

Fixes: https://tracker.ceph.com/issues/51166
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
4 years agoMerge pull request #41585 from pritha-srivastava/wip-rgw-sts-session-policy-eval
Matt Benjamin [Thu, 10 Jun 2021 11:56:52 +0000 (07:56 -0400)]
Merge pull request #41585 from pritha-srivastava/wip-rgw-sts-session-policy-eval

rgw/sts: correcting the evaluation of session policies

4 years agoMerge pull request #41735 from pritha-srivastava/wip-rgw-sts-ops-log-updates
Matt Benjamin [Thu, 10 Jun 2021 11:56:31 +0000 (07:56 -0400)]
Merge pull request #41735 from pritha-srivastava/wip-rgw-sts-ops-log-updates

rgw/sts: adding role name and role session to ops log.

4 years agoMerge pull request #41653 from zdover23/wip-doc-cephadm-serve-man-setting-a-limit...
Sebastian Wagner [Thu, 10 Jun 2021 10:52:17 +0000 (12:52 +0200)]
Merge pull request #41653 from zdover23/wip-doc-cephadm-serve-man-setting-a-limit-2021-06-02

doc/cephadm: enriching "setting a limit"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #41693 from sebastian-philipp/cephadm-devenv-bootstrap-mount
Sebastian Wagner [Thu, 10 Jun 2021 10:13:40 +0000 (12:13 +0200)]
Merge pull request #41693 from sebastian-philipp/cephadm-devenv-bootstrap-mount

doc/dev/cephadm: cephadm bootstrap --shared_ceph_folder

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #41741 from liu-chunmei/seastore-fixe-read-invalid
Liu-Chunmei [Thu, 10 Jun 2021 00:17:07 +0000 (17:17 -0700)]
Merge pull request #41741 from liu-chunmei/seastore-fixe-read-invalid

crimson/seastore: fix OTree read invalid extent

4 years agoqa: update cephfs-shell distro to ubuntu 20.04 41797/head
Patrick Donnelly [Wed, 9 Jun 2021 23:20:05 +0000 (16:20 -0700)]
qa: update cephfs-shell distro to ubuntu 20.04

18.04 is no longer built.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocrimson/seastore: fix OTree read invalid extent 41741/head
chunmei-liu [Mon, 7 Jun 2021 23:48:20 +0000 (16:48 -0700)]
crimson/seastore: fix OTree read invalid extent

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
4 years agoMerge PR #41229 into master
Sage Weil [Wed, 9 Jun 2021 19:55:25 +0000 (15:55 -0400)]
Merge PR #41229 into master

* refs/pull/41229/head:
qa/suites/upgrade/pacific-x/stress-split: add

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
4 years agoMerge pull request #41729 from tchaikov/wip-std-filesystem-gcc-8
Kefu Chai [Wed, 9 Jun 2021 15:41:47 +0000 (23:41 +0800)]
Merge pull request #41729 from tchaikov/wip-std-filesystem-gcc-8

*: stop using <experimental/filesystem> as an alternative

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #41791 from rzarzynski/wip-crimson-monc-no_auth_req_when_inactive
Kefu Chai [Wed, 9 Jun 2021 15:40:21 +0000 (23:40 +0800)]
Merge pull request #41791 from rzarzynski/wip-crimson-monc-no_auth_req_when_inactive

crimson/monc: don't serve auth requests without active mon connection.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41782 from sseshasa/wip-fix-standalone-test
Neha Ojha [Wed, 9 Jun 2021 15:28:05 +0000 (08:28 -0700)]
Merge pull request #41782 from sseshasa/wip-fix-standalone-test

qa/standalone: Use osd op queue = wpq in activate_osd() within ceph-helpers.sh.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agocommon/strtol: do not check for existence of <charconv> 41729/head
Kefu Chai [Mon, 7 Jun 2021 09:18:19 +0000 (17:18 +0800)]
common/strtol: do not check for existence of <charconv>

<charconv> is available since GCC-8, see https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2017

> Elementary string conversions P0067R5 11.1 (integral types supported since 8.1) __has_include(<charconv>), __cpp_lib_to_chars >= 201611

since we always have the access to GCC-8.1 and up, there is no need to
detect the existence of <charconv> anymore.

also, because GCC-11 introduced the support of float types support,
update the comment to reflect the change.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years ago*: always include <filesystem>
Kefu Chai [Mon, 7 Jun 2021 08:31:11 +0000 (16:31 +0800)]
*: always include <filesystem>

since there is no need to be compatible with GCC older than GCC-8, so
there is no need to use <experimental/filesystem> as an alternative of
<filesystem> anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoos/bluestore/bluestore_tool: do not use boost:filesystem as alternative
Kefu Chai [Mon, 7 Jun 2021 08:21:22 +0000 (16:21 +0800)]
os/bluestore/bluestore_tool: do not use boost:filesystem as alternative

reverts 9dedabde520907f99727edf9f916ebb55ccdaa87

since there is no need to be compatible with GCC older than GCC-8, so
there is no need to use boost::filesystem as an alternative of
std::filesystem anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: stop detecting <experimental/filesystem>
Kefu Chai [Mon, 7 Jun 2021 08:39:22 +0000 (16:39 +0800)]
cmake: stop detecting <experimental/filesystem>

since we've dropped the support of GCC older than v8.0, there is no need
to detect <experimental/filesystem>

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: require GCC-8.1 and up
Kefu Chai [Mon, 7 Jun 2021 08:17:09 +0000 (16:17 +0800)]
cmake: require GCC-8.1 and up

for better C++17 support, for instance for a better std::filesystem
support.

the reason why 8.1 is required is that ubuntu focal provides GCC-8.1,
and RHEL/CentOS8 provides GCC-8.4.1. so we only test the build on
GCC-8.1 and up so far.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mg/progress: Disregard unreported pgs 40480/head
Kamoltat [Thu, 13 May 2021 17:38:10 +0000 (17:38 +0000)]
pybind/mg/progress: Disregard unreported pgs

The global recovery event progress calculations only
takes into account pgs with `reported_epoch < start_epoch_of_event`
but sometimes the pgs doesn't get move before or after the creation
of the global recovery event, therefore this might result in a bug
where the global event gets stuck forever unless there is another
event that specifically makes the pgs that get stuck moves and updates
its `reported_epoch`.

Therefore, we decided to disregard pgs that are in active+clean state
but has `reported_epoch < start_epoch_of_event`.

Fixes: https://tracker.ceph.com/issues/49988
Signed-off-by: Kamoltat <ksirivad@redhat.com>
4 years agoMerge PR #41740 into master
Sage Weil [Wed, 9 Jun 2021 14:51:34 +0000 (10:51 -0400)]
Merge PR #41740 into master

* refs/pull/41740/head:
mgr/nfs: do not depend on cephadm.utils

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocrimson/monc: don't serve auth requests without active mon connection. 41791/head
Radoslaw Zarzynski [Wed, 9 Jun 2021 14:31:50 +0000 (14:31 +0000)]
crimson/monc: don't serve auth requests without active mon connection.

It's yet another racing issue which happens when auth request
handling is performed during the `active_con` reset sequence.
It caused the following `nullptr` dereference at Sepia:

```
DEBUG 2021-06-09 10:27:24,059 [shard 0] ms - [osd.6(client) v2:172.21.15.170:6809/33397 >> client.? -@39840] GOT AuthRequestFrame: method=2, p
referred_modes={2, 1}, payload_len=174
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4977-g65cb255e/rpm/el8/BUILD/ceph-17.0.0-4977-g65cb255e/src/crimson/mon/MonClient.cc:595:26: runtime error: member call on null pointer of type 'struct Connection'
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4977-g65cb255e/rpm/el8/BUILD/ceph-17.0.0-4977-g65cb255e/src/crimson/mon/MonClient.cc:178:11: runtime error: member access within null pointer of type 'struct Connection'
Segmentation fault on shard 0.
Backtrace:
 0# 0x0000563F9C00395F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F4A064D0B20 in /lib64/libpthread.so.0
 4# crimson::mon::Connection::get_keys() in ceph-osd
 5# crimson::mon::Client::handle_auth_request(seastar::shared_ptr<crimson::net::Connection>, seastar::lw_shared_ptr<AuthConnectionMeta>, bool, unsigned int, ceph::buffer::v15_2_0::list const&, ceph::buffer::v15_2_0::list*) in ceph-osd
 6# crimson::net::ProtocolV2::_handle_auth_request(ceph::buffer::v15_2_0::list&, bool) in ceph-osd
 7# 0x0000563F9D007B39 in ceph-osd
 8# 0x0000563F9D008C45 in ceph-osd
 9# 0x0000563F95FF8D70 in ceph-osd
10# 0x0000563FA1A560BF in ceph-osd
11# 0x0000563FA1A5B600 in ceph-osd
12# 0x0000563FA1C0D66B in ceph-osd
13# 0x0000563FA176B0EA in ceph-osd
14# 0x0000563FA177520E in ceph-osd
15# main in ceph-osd
16# __libc_start_main in /lib64/libc.so.6
17# _start in ceph-osd
Fault at location: 0xb0
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #41761 from dang/wip-dang-setattrs
Daniel Gryniewicz [Wed, 9 Jun 2021 13:46:58 +0000 (09:46 -0400)]
Merge pull request #41761 from dang/wip-dang-setattrs

RGW - Don't move attrs before setting them

4 years agomon/MonCommands: remove obsolete mds commands 41786/head
Kefu Chai [Wed, 9 Jun 2021 09:31:42 +0000 (17:31 +0800)]
mon/MonCommands: remove obsolete mds commands

* "mds dump"
  "mds getmap",
  "mds stop",
  "mds set_max_mds",
  "mds set",
  "mds rmfailed"
  "mds add_data_pool"
  "mds rm_data_pool"
  "mds remove_data_pool"
  the commands above were marked "OBSOLETE" back in
  a8fc92933bb24fe8ed06407335746f1262d6aea8, which was included in v13.0.1,
* "mds tell" was marked obsolete in
  e0d1127205bb45bb623a50b4b68ae85f7b083e04, which was included in v12.0.2,
* "mds deactivate" was marked obsolete in
  c7bd6f02c74513f29be12ae47808ca1a1ddaec9f, which was included in v13.1.0,
* "mds newfs" was marked obsolete in
  072c41e3492a13525d2a45465ed580fc1426fbd1, which was included by v12.0.2

so according to our command retirement policy proposed by
https://ceph-users.ceph.narkive.com/iUh4e0nj/rfc-deprecating-ceph-tool-commands

> Once two major releases go by, the command will then enter the OBSOLETE
> period. This would be one major release, during which the command would
> no longer work although still acknowledged. A simple message down the
> lines of 'This command is now obsolete; please check the docs' would
> suffice to inform the user.

since the next release will be v17. it's been long enough for retiring
these OBSOLETE commands.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomon/MonCommands: remove obsolete mon commands
Kefu Chai [Wed, 9 Jun 2021 09:38:39 +0000 (17:38 +0800)]
mon/MonCommands: remove obsolete mon commands

the "scrub" command was marked obsolete in
e9a5ce0897efc6126caeebea9900bf05ec3d2174, which was included by
v15.1.0, but the next release will be v17, so it's long enough to retire
this command.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/standalone: Use osd op queue = wpq in activate_osd() 41782/head
Sridhar Seshasayee [Wed, 9 Jun 2021 06:14:11 +0000 (11:44 +0530)]
qa/standalone: Use osd op queue = wpq in activate_osd()

This change is a follow-up to commit
b6e9c0903d5ad9a699b675f9fa7739e9cce9a5f3 that set the scheduler to wpq in
run_osd() and run_osd_filestore(). In addition, activate_osd() too has to
set the scheduler type to 'wpq' in order to be consistent and avoid test
failures.

The above is a temporary measure until all the standalone tests are
modified to run well with the mclock_scheduler.

Fixes: https://tracker.ceph.com/issues/51074
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
4 years agorgw/sts: adding role name and role session to ops log. 41735/head
Pritha Srivastava [Mon, 7 Jun 2021 15:25:11 +0000 (20:55 +0530)]
rgw/sts: adding role name and role session to ops log.
Also adding authentication type for all ops.

Fixes: https://tracker.ceph.com/issues/51152
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
4 years agoMerge pull request #41727 from yanqiang-ux/handle_error_ret_FillInVerifyExtent
Kefu Chai [Wed, 9 Jun 2021 09:03:05 +0000 (17:03 +0800)]
Merge pull request #41727 from yanqiang-ux/handle_error_ret_FillInVerifyExtent

osd: set r only if succeed in FillInVerifyExtent

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/mgr_module:delete invalid judgment 41781/head
dengchl01 [Wed, 9 Jun 2021 08:59:51 +0000 (16:59 +0800)]
mgr/mgr_module:delete invalid judgment

Signed-off-by: dengchl01 <dengchl01@inspur.com>
4 years agoosd: set r only if succeed in FillInVerifyExtent 41727/head
yanqiang-ux [Mon, 7 Jun 2021 07:54:44 +0000 (15:54 +0800)]
osd: set r only if succeed in FillInVerifyExtent

When read failed, ret can be taken as data len in FillInVerifyExtent, which should be avoided.
It may cause errors in crc repair or retry read because of the data len. In my case, we use FillInVerifyExtent for EC read,
when meet -EIO,we will try crc repair, which need read data from other shard accrding to data len.
And I meet assert in ECBackend.cc (loc: line 2288 ceph_assert(range.first != range.second) ), But it seems master branch not support EC crc repair.
In shot, when reuse the readop may cause unpredictable error.

Fixes: https://tracker.ceph.com/issues/51115
Signed-off-by: yanqiang-ux <yanqiang_ux@163.com>
4 years agoMerge pull request #41718 from tchaikov/wip-doc-ci-deps
Kefu Chai [Wed, 9 Jun 2021 08:36:01 +0000 (16:36 +0800)]
Merge pull request #41718 from tchaikov/wip-doc-ci-deps

doc/dev: add continuous-integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agodoc/dev: add continuous-integration 41718/head
Kefu Chai [Sun, 6 Jun 2021 10:22:06 +0000 (18:22 +0800)]
doc/dev: add continuous-integration

for noting down the architecture of our CI. and how we prepare
some of the build dependencies.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41123 from rhcs-dashboard/host-addr-and-labels
Ernesto Puerta [Wed, 9 Jun 2021 08:23:34 +0000 (10:23 +0200)]
Merge pull request #41123 from rhcs-dashboard/host-addr-and-labels

mgr/dashboard: Include Network address and labels on Host Creation form

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
4 years agoMerge pull request #41548 from rhcs-dashboard/revert-npm
Ernesto Puerta [Wed, 9 Jun 2021 08:08:16 +0000 (10:08 +0200)]
Merge pull request #41548 from rhcs-dashboard/revert-npm

Revert "mgr/dashboard: Generate NPM dependencies manifest"

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agomgr/cephadm: When create osd which include db, _create_daemon will be executed twice...
jianglong01 [Tue, 8 Jun 2021 06:00:48 +0000 (14:00 +0800)]
mgr/cephadm: When create osd which include db, _create_daemon will be executed twice. So we should filter the db type

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
4 years agoMerge pull request #38135 from joke-lee/fix-list-parts-show-id-and-displayname
J. Eric Ivancich [Tue, 8 Jun 2021 17:20:05 +0000 (13:20 -0400)]
Merge pull request #38135 from joke-lee/fix-list-parts-show-id-and-displayname

rgw: RGWListMultipart show Owner ID and DisplayName

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #40975 from trociny/wip-50415
J. Eric Ivancich [Tue, 8 Jun 2021 17:19:33 +0000 (13:19 -0400)]
Merge pull request #40975 from trociny/wip-50415

cls/rgw: look for plain entries in non-ascii plain namespace too

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>