ceph.git
6 hours agoMerge PR #41799 into master master
Sage Weil [Mon, 14 Jun 2021 15:02:31 +0000 (11:02 -0400)]
Merge PR #41799 into master

* refs/pull/41799/head:
cephadm: validate `--fsid` during bootstrap

Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 hours agoMerge PR #41805 into master
Sage Weil [Mon, 14 Jun 2021 15:02:16 +0000 (11:02 -0400)]
Merge PR #41805 into master

* refs/pull/41805/head:
cephadm: set TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728

Reviewed-by: Sebastian Wagner <swagner@suse.com>
7 hours agoMerge pull request #39624 from sebastian-philipp/mypy-812
Kefu Chai [Mon, 14 Jun 2021 14:53:02 +0000 (22:53 +0800)]
Merge pull request #39624 from sebastian-philipp/mypy-812

src,qa: Upgrade to mypy 0.901

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 hours agoMerge pull request #41632 from jmolmo/fix_osd_rm_help_msg
Sebastian Wagner [Mon, 14 Jun 2021 14:09:07 +0000 (16:09 +0200)]
Merge pull request #41632 from jmolmo/fix_osd_rm_help_msg

mgr/cephadm: fix errors in command <ceph orch osd> help and documentation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
7 hours agoMerge pull request #41556 from adk3798/daemon-count
Sebastian Wagner [Mon, 14 Jun 2021 14:08:46 +0000 (16:08 +0200)]
Merge pull request #41556 from adk3798/daemon-count

mgr/cephadm: fix --service-type flag in orch ls when service has id

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
8 hours agoMerge PR #41827 into master
Sage Weil [Mon, 14 Jun 2021 13:11:04 +0000 (09:11 -0400)]
Merge PR #41827 into master

* refs/pull/41827/head:
qa: move dashboard e2e from cephadm -> rados suite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
8 hours agoMerge pull request #41796 from rzarzynski/wip-crimson-cltreq-dbgs
Kefu Chai [Mon, 14 Jun 2021 13:00:04 +0000 (21:00 +0800)]
Merge pull request #41796 from rzarzynski/wip-crimson-cltreq-dbgs

crimson/osd: improve debugs in ClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 hours agocrimson/osd: improve debugs in OpSequencer. 41796/head
Radoslaw Zarzynski [Wed, 9 Jun 2021 20:55:34 +0000 (20:55 +0000)]
crimson/osd: improve debugs in OpSequencer.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 hours agocrimson/osd: should_abort_request() dumps Operation's details.
Radoslaw Zarzynski [Wed, 9 Jun 2021 19:18:33 +0000 (19:18 +0000)]
crimson/osd: should_abort_request() dumps Operation's details.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 hours agocrimson/osd: print prev_op_id in ClientRequest's debugs.
Radoslaw Zarzynski [Wed, 9 Jun 2021 19:16:56 +0000 (19:16 +0000)]
crimson/osd: print prev_op_id in ClientRequest's debugs.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
11 hours agoMerge pull request #41832 from inspur-wyq/patch-4
Kefu Chai [Mon, 14 Jun 2021 10:13:21 +0000 (18:13 +0800)]
Merge pull request #41832 from inspur-wyq/patch-4

os/ObjectStore: fix typo in comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
18 hours agoMerge PR #41786 into master 41599/head
Patrick Donnelly [Mon, 14 Jun 2021 02:57:37 +0000 (19:57 -0700)]
Merge PR #41786 into master

* refs/pull/41786/head:
mon/MonCommands: remove obsolete mds commands
mon/MonCommands: remove obsolete mon commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
18 hours agoMerge PR #41580 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:57:05 +0000 (19:57 -0700)]
Merge PR #41580 into master

* refs/pull/41580/head:
cephfs: make ceph_fallocate() platform independent

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
18 hours agoMerge PR #41482 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:56:34 +0000 (19:56 -0700)]
Merge PR #41482 into master

* refs/pull/41482/head:
qa: remove obsolete deactivate routines

Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 hours agoMerge PR #41481 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:56:04 +0000 (19:56 -0700)]
Merge PR #41481 into master

* refs/pull/41481/head:
client: Add ceph.caps vxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
19 hours agoMerge PR #41431 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:54:45 +0000 (19:54 -0700)]
Merge PR #41431 into master

* refs/pull/41431/head:
mds: progress the recover queue immediately after the inode is enqueued
mds: switch to mdr->get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
19 hours agoMerge PR #41422 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:52:48 +0000 (19:52 -0700)]
Merge PR #41422 into master

* refs/pull/41422/head:
qa/tasks/cephfs/test_sessionmap: reap connections immediately
msg/async: configurable threshold for reaping dead connections

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
19 hours agoMerge PR #41403 into master
Patrick Donnelly [Mon, 14 Jun 2021 02:51:52 +0000 (19:51 -0700)]
Merge PR #41403 into master

* refs/pull/41403/head:
mgr/volumes: Add config to insert delay at the beginning of the clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
19 hours agoos/ObjectStore: fix typo in comments 41832/head
wangyunqing [Mon, 14 Jun 2021 02:42:14 +0000 (10:42 +0800)]
os/ObjectStore: fix typo in comments

Signed-off-by: wangyunqing <wangyunqing@inspur.com>
30 hours agoMerge pull request #40740 from batrick/i50278 feature-50336-cluster-creation-wizard
Kefu Chai [Sun, 13 Jun 2021 15:30:20 +0000 (23:30 +0800)]
Merge pull request #40740 from batrick/i50278

pybind/mgr: add support for common sqlite3 databases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
30 hours agoMerge pull request #41809 from linuxbox2/wip-returnvec-64
Kefu Chai [Sun, 13 Jun 2021 15:27:58 +0000 (23:27 +0800)]
Merge pull request #41809 from linuxbox2/wip-returnvec-64

rados: increase osd_max_write_op_reply_len default to 64 bytes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
30 hours agoMerge pull request #41746 from strenuous-life/wip-cephadm-osd-create
Kefu Chai [Sun, 13 Jun 2021 15:26:46 +0000 (23:26 +0800)]
Merge pull request #41746 from strenuous-life/wip-cephadm-osd-create

mgr/cephadm: When create osd which include db, _create_daemon will be…

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 days agoMerge PR #41781 into master
Sage Weil [Sat, 12 Jun 2021 13:02:19 +0000 (09:02 -0400)]
Merge PR #41781 into master

* refs/pull/41781/head:
mgr/mgr_module:delete invalid judgment

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 days agoMerge PR #41699 into master
Sage Weil [Sat, 12 Jun 2021 13:01:38 +0000 (09:01 -0400)]
Merge PR #41699 into master

* refs/pull/41699/head:
mgr/telemetry: pass leaderboard flag even w/o ident

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2 days agoqa: move dashboard e2e from cephadm -> rados suite 41827/head
Sage Weil [Sat, 12 Jun 2021 12:52:42 +0000 (07:52 -0500)]
qa: move dashboard e2e from cephadm -> rados suite

This test fails ~20% of the time.

Signed-off-by: Sage Weil <sage@newdream.net>
2 days agoqa: choose victim pg from rbd pool 40740/head
Patrick Donnelly [Sat, 12 Jun 2021 03:04:53 +0000 (20:04 -0700)]
qa: choose victim pg from rbd pool

Right now scrub_test picks any pg in ceph. Unfortunately, it picked the
.mgr pool's only pg in [1]:

2021-05-16T11:36:35.035 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool rbd setomapval main.db-journal.0000000000000000 key val

Instead, only pick a pg in the rbd pool.

[1] /ceph/teuthology-archive/kchai-2021-05-16_11:19:39-rados-wip-kefu-testing-2021-05-16-1043-distro-basic-smithi/6117396/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoqa,pybind/mgr: allow disabling .mgr pool
Patrick Donnelly [Sat, 1 May 2021 02:31:35 +0000 (19:31 -0700)]
qa,pybind/mgr: allow disabling .mgr pool

This is mostly for testing: a lot of tests assume that there are no
existing pools. These tests relied on a config to turn off creating the
"device_health_metrics" pool which generally exists for any new Ceph
cluster. It would be better to make these tests tolerant of the new .mgr
pool but clearly there's a lot of these. So just convert the config to
make it work.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agodoc: document new .mgr pool
Patrick Donnelly [Tue, 13 Apr 2021 02:42:57 +0000 (19:42 -0700)]
doc: document new .mgr pool

And that pool names beginning with `.` are reserved names.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoqa: add upgrade test for devicehealth
Patrick Donnelly [Mon, 19 Apr 2021 19:58:01 +0000 (12:58 -0700)]
qa: add upgrade test for devicehealth

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoqa: fix api test failures
Patrick Donnelly [Sun, 11 Apr 2021 20:16:25 +0000 (13:16 -0700)]
qa: fix api test failures

"device_health_metrics" pool is gone -- .mgr pool is in.

I don't think the pool removal code in some test cases is necessary any
longer with recent changes to remove those warnings; so that code is
gone too.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agopybind/mgr/devicehealth: load legacy device metrics pool
Patrick Donnelly [Fri, 9 Apr 2021 23:56:53 +0000 (16:56 -0700)]
pybind/mgr/devicehealth: load legacy device metrics pool

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agopybind/mgr/devicehealth: update to store metrics in sqlite
Patrick Donnelly [Fri, 9 Apr 2021 22:00:06 +0000 (15:00 -0700)]
pybind/mgr/devicehealth: update to store metrics in sqlite

This commit just modifies the module to use the sqlite3 database for
storing health metrics. A followup commit loads the legacy pool's data.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agopybind/mgr: add sqlite3 db module helpers
Patrick Donnelly [Fri, 9 Apr 2021 21:58:47 +0000 (14:58 -0700)]
pybind/mgr: add sqlite3 db module helpers

This creates a new '.mgr' pool for storing a default sqlite3 database
for each mgr module. Each module's database is stored in:

        file:///.mgr:<mgr module name>/main.db?vfs=ceph

The "main.db" is the only one used presently but perhaps a module may
want extra databases for some reason. The module name is used for the
RADOS namespace.

Databases are versioned in a common table called MgrModuleKV using the
"__version" key. A mechanism is in place (SCHEMA_VERSIONED) to allow
modules to upgrade their databases over time in a consistent way.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoMerge pull request #40480 from kamoltat/wip-ksirivad-fix-bug-49988
Kefu Chai [Sat, 12 Jun 2021 00:37:35 +0000 (08:37 +0800)]
Merge pull request #40480 from kamoltat/wip-ksirivad-fix-bug-49988

pybind/mgr/progress: Disregard unreported pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2 days agoMerge pull request #41772 from cyx1231st/wip-seastore-onode-tree-node-size
Kefu Chai [Sat, 12 Jun 2021 00:11:45 +0000 (08:11 +0800)]
Merge pull request #41772 from cyx1231st/wip-seastore-onode-tree-node-size

crimson/onode-staged-tree: extend tree node sizes to fit insert upper-bounds

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 days agoMerge PR #39505 into master
Patrick Donnelly [Sat, 12 Jun 2021 00:10:41 +0000 (17:10 -0700)]
Merge PR #39505 into master

* refs/pull/39505/head:
qa: test nowsync option in kernel client workflows
qa: deep merge top level overrides for fuse/kclient

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2 days agoMerge pull request #40952 from tchaikov/wip-deb-build-profile
Kefu Chai [Sat, 12 Jun 2021 00:08:47 +0000 (08:08 +0800)]
Merge pull request #40952 from tchaikov/wip-deb-build-profile

debian: use build-profile to manage build deps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoMerge PR #41683 into master
Patrick Donnelly [Sat, 12 Jun 2021 00:07:06 +0000 (17:07 -0700)]
Merge PR #41683 into master

* refs/pull/41683/head:
qa: update RHEL to 8.4

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 days agoMerge pull request #41550 from aaryanporwal/navbar-fix
Ernesto Puerta [Fri, 11 Jun 2021 17:46:15 +0000 (19:46 +0200)]
Merge pull request #41550 from aaryanporwal/navbar-fix

mgr/dashboard: fix for right sidebar nav icon not clickable

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 days agocrimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print 41772/head
Yingxin Cheng [Wed, 9 Jun 2021 03:06:28 +0000 (11:06 +0800)]
crimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: merge with single-value tail internal node
Yingxin Cheng [Wed, 9 Jun 2021 02:48:43 +0000 (10:48 +0800)]
crimson/onode-staged-tree: merge with single-value tail internal node

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: switch to and test larger node sizes
Yingxin Cheng [Tue, 8 Jun 2021 07:19:33 +0000 (15:19 +0800)]
crimson/onode-staged-tree: switch to and test larger node sizes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys
Yingxin Cheng [Tue, 8 Jun 2021 06:11:52 +0000 (14:11 +0800)]
crimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: add the missing retired validation
Yingxin Cheng [Tue, 8 Jun 2021 06:03:07 +0000 (14:03 +0800)]
crimson/onode-staged-tree: add the missing retired validation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: encode/decode stage_size instead
Yingxin Cheng [Tue, 8 Jun 2021 01:59:32 +0000 (09:59 +0800)]
crimson/onode-staged-tree: encode/decode stage_size instead

end-offset may risk overflow the node_offset_t, but stage_size will not.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: use extent_len_t and node_offset_t correctly
Yingxin Cheng [Tue, 8 Jun 2021 01:55:15 +0000 (09:55 +0800)]
crimson/onode-staged-tree: use extent_len_t and node_offset_t correctly

extent_len_t represents a value that may include the node size, but
node_offset_t cannot and may overflow. Also add validations when
try to cast a larger type to node_offset_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: remove unused code
Yingxin Cheng [Tue, 8 Jun 2021 01:42:34 +0000 (09:42 +0800)]
crimson/onode-staged-tree: remove unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: validate node size where possible
Yingxin Cheng [Tue, 8 Jun 2021 01:41:44 +0000 (09:41 +0800)]
crimson/onode-staged-tree: validate node size where possible

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE
Yingxin Cheng [Mon, 7 Jun 2021 05:57:13 +0000 (13:57 +0800)]
crimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE

Switch to run-time node_size.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: pass node_size to lower node stages
Yingxin Cheng [Mon, 7 Jun 2021 05:15:58 +0000 (13:15 +0800)]
crimson/onode-staged-tree: pass node_size to lower node stages

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: remove compile-time node SIZE
Yingxin Cheng [Mon, 7 Jun 2021 02:45:18 +0000 (10:45 +0800)]
crimson/onode-staged-tree: remove compile-time node SIZE

Switch to run-time node_extent_t::node_size instead.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: cleanup internal_fields_3_t template
Yingxin Cheng [Mon, 7 Jun 2021 02:43:31 +0000 (10:43 +0800)]
crimson/onode-staged-tree: cleanup internal_fields_3_t template

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: pass node_size into node_extent_t
Yingxin Cheng [Thu, 3 Jun 2021 02:28:50 +0000 (10:28 +0800)]
crimson/onode-staged-tree: pass node_size into node_extent_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: extend num_keys_t to support larger nodes
Yingxin Cheng [Wed, 2 Jun 2021 15:44:07 +0000 (23:44 +0800)]
crimson/onode-staged-tree: extend num_keys_t to support larger nodes

Also adjust the accroding layout related unit tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE
Yingxin Cheng [Wed, 2 Jun 2021 08:25:05 +0000 (16:25 +0800)]
crimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: check the maximum tree height
Yingxin Cheng [Wed, 2 Jun 2021 06:07:33 +0000 (14:07 +0800)]
crimson/onode-staged-tree: check the maximum tree height

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: validate safe-split from tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:36:11 +0000 (13:36 +0800)]
crimson/onode-staged-tree: validate safe-split from tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: add and use node sizes in tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:33:50 +0000 (13:33 +0800)]
crimson/onode-staged-tree: add and use node sizes in tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: implement size upper-bounds to value
Yingxin Cheng [Wed, 2 Jun 2021 05:18:48 +0000 (13:18 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: implement size upper-bounds to ns/oid
Yingxin Cheng [Wed, 2 Jun 2021 04:47:55 +0000 (12:47 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to ns/oid

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: introduce compile-time tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 03:52:33 +0000 (11:52 +0800)]
crimson/onode-staged-tree: introduce compile-time tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue
Yingxin Cheng [Mon, 31 May 2021 07:58:30 +0000 (15:58 +0800)]
crimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/seastore: introduce TransactionManager::read_extent(t, offset)
Yingxin Cheng [Mon, 31 May 2021 02:11:01 +0000 (10:11 +0800)]
crimson/seastore: introduce TransactionManager::read_extent(t, offset)

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: correct the node size equation
Yingxin Cheng [Wed, 26 May 2021 06:26:59 +0000 (14:26 +0800)]
crimson/onode-staged-tree: correct the node size equation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: try merge peer if node has one value
Yingxin Cheng [Wed, 26 May 2021 04:45:27 +0000 (12:45 +0800)]
crimson/onode-staged-tree: try merge peer if node has one value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agocrimson/onode-staged-tree: cleanup and introduce has_single_value()
Yingxin Cheng [Wed, 26 May 2021 04:44:13 +0000 (12:44 +0800)]
crimson/onode-staged-tree: cleanup and introduce has_single_value()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 days agoMerge pull request #41814 from tchaikov/wip-crimson-fmt-seq
Kefu Chai [Fri, 11 Jun 2021 11:59:46 +0000 (19:59 +0800)]
Merge pull request #41814 from tchaikov/wip-crimson-fmt-seq

crimson/osd: specialize fmt::formatter<> for OpSequencer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 days agoMerge pull request #41759 from rhcs-dashboard/wip-api_label-master
Ernesto Puerta [Fri, 11 Jun 2021 09:08:18 +0000 (11:08 +0200)]
Merge pull request #41759 from rhcs-dashboard/wip-api_label-master

.github/labeler: add api-change label

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 days agoMerge pull request #41242 from zdover23/wip-doc-cephadm-monitoring-deploying-monitori...
Sebastian Wagner [Fri, 11 Jun 2021 08:45:47 +0000 (10:45 +0200)]
Merge pull request #41242 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-second-half-2021-05-10

doc/cephadm: rewrite "deploy. mon. w/cephadm" 2/2

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 days agoMerge pull request #41241 from zdover23/wip-doc-cephadm-monitoring-deploying-monitori...
Sebastian Wagner [Fri, 11 Jun 2021 08:45:04 +0000 (10:45 +0200)]
Merge pull request #41241 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-first-half-2021-05-10

doc/cephadm: rewrite "deploy. mon. w/cephadm"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 days agocrimson/osd: specialize fmt::formatter<> for OpSequencer 41814/head
Kefu Chai [Fri, 11 Jun 2021 06:18:22 +0000 (14:18 +0800)]
crimson/osd: specialize fmt::formatter<> for OpSequencer

so we can print it like fmt::format("{}", sequencer)

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agomgr/cephadm: When test osd active, it need to add "mock call" to make sure execute... 41746/head
jianglong01 [Fri, 11 Jun 2021 02:56:30 +0000 (10:56 +0800)]
mgr/cephadm: When test osd active, it need to add "mock call" to make sure execute correctly

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
3 days agoMerge pull request #41812 from agayev/zoned-fix-block-size
Kefu Chai [Fri, 11 Jun 2021 00:41:36 +0000 (08:41 +0800)]
Merge pull request #41812 from agayev/zoned-fix-block-size

os/bluestore: Fix the size of the block in the Allocator base class t…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 days agoMerge pull request #41744 from tchaikov/wip-cmake-no-python-bindings-on-wnin32
Kefu Chai [Fri, 11 Jun 2021 00:32:30 +0000 (08:32 +0800)]
Merge pull request #41744 from tchaikov/wip-cmake-no-python-bindings-on-wnin32

cmake: vstart do not depend on cython_rbd if WIN32

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 days agoMerge PR #41442 into master
Patrick Donnelly [Thu, 10 Jun 2021 21:53:21 +0000 (14:53 -0700)]
Merge PR #41442 into master

* refs/pull/41442/head:
doc/cephfs/nfs: add ganesha configuration hierarchy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 days agoMerge PR #41797 into master
Patrick Donnelly [Thu, 10 Jun 2021 20:55:12 +0000 (13:55 -0700)]
Merge PR #41797 into master

* refs/pull/41797/head:
qa: update cephfs-shell distro to ubuntu 20.04

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 days agoMerge pull request #41575 from ifed01/wip-ifed-fix-no-osd-daemonperf
Igor Fedotov [Thu, 10 Jun 2021 20:32:00 +0000 (23:32 +0300)]
Merge pull request #41575 from ifed01/wip-ifed-fix-no-osd-daemonperf

common/PriorityCache: low perf counters priorities for submodules.

Reviewed-by: Mark Nelson <mnelson@readhat.com>
4 days agoos/bluestore: Fix the size of the block in the Allocator base class to avoid 41812/head
Abutalib Aghayev [Thu, 10 Jun 2021 19:59:45 +0000 (15:59 -0400)]
os/bluestore: Fix the size of the block in the Allocator base class to avoid
the confusing log message about the block size.

Signed-off-by: Abutalib Aghayev <agayev@psu.edu>
4 days agorados: increase osd_max_write_op_reply_len default to 64 bytes 41809/head
Matt Benjamin [Thu, 10 Jun 2021 14:27:58 +0000 (10:27 -0400)]
rados: increase osd_max_write_op_reply_len default to 64 bytes

Agreed in #ceph-devel on 6/10.  The current controlling
rationale is that the default value should be sufficient to
marshall a SHA-512 checksum.

Fixes: https://tracker.ceph.com/issues/51166

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
4 days agocephadm: set TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728 41805/head
Sage Weil [Thu, 10 Jun 2021 12:39:11 +0000 (08:39 -0400)]
cephadm: set TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728

This used to live in /etc/{sysconfig,defaults}/ceph, but that does not
apply inside the container.

Signed-off-by: Sage Weil <sage@newdream.net>
4 days agoMerge pull request #41585 from pritha-srivastava/wip-rgw-sts-session-policy-eval
Matt Benjamin [Thu, 10 Jun 2021 11:56:52 +0000 (07:56 -0400)]
Merge pull request #41585 from pritha-srivastava/wip-rgw-sts-session-policy-eval

rgw/sts: correcting the evaluation of session policies

4 days agoMerge pull request #41735 from pritha-srivastava/wip-rgw-sts-ops-log-updates
Matt Benjamin [Thu, 10 Jun 2021 11:56:31 +0000 (07:56 -0400)]
Merge pull request #41735 from pritha-srivastava/wip-rgw-sts-ops-log-updates

rgw/sts: adding role name and role session to ops log.

4 days agocephadm: Upgrade to mypy 0.901 39624/head
Sebastian Wagner [Wed, 9 Jun 2021 13:17:35 +0000 (15:17 +0200)]
cephadm: Upgrade to mypy 0.901

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
4 days agopybind/mgr: Upgrade to mypy 0.901
Sebastian Wagner [Wed, 9 Jun 2021 12:08:10 +0000 (14:08 +0200)]
pybind/mgr: Upgrade to mypy 0.901

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
4 days agoMerge pull request #41653 from zdover23/wip-doc-cephadm-serve-man-setting-a-limit...
Sebastian Wagner [Thu, 10 Jun 2021 10:52:17 +0000 (12:52 +0200)]
Merge pull request #41653 from zdover23/wip-doc-cephadm-serve-man-setting-a-limit-2021-06-02

doc/cephadm: enriching "setting a limit"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 days agoMerge pull request #41693 from sebastian-philipp/cephadm-devenv-bootstrap-mount
Sebastian Wagner [Thu, 10 Jun 2021 10:13:40 +0000 (12:13 +0200)]
Merge pull request #41693 from sebastian-philipp/cephadm-devenv-bootstrap-mount

doc/dev/cephadm: cephadm bootstrap --shared_ceph_folder

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 days agoMerge pull request #41741 from liu-chunmei/seastore-fixe-read-invalid
Liu-Chunmei [Thu, 10 Jun 2021 00:17:07 +0000 (17:17 -0700)]
Merge pull request #41741 from liu-chunmei/seastore-fixe-read-invalid

crimson/seastore: fix OTree read invalid extent

4 days agocephadm: validate `--fsid` during bootstrap 41799/head
Michael Fritch [Wed, 9 Jun 2021 23:45:03 +0000 (17:45 -0600)]
cephadm: validate `--fsid` during bootstrap

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 days agoqa: update cephfs-shell distro to ubuntu 20.04 41797/head
Patrick Donnelly [Wed, 9 Jun 2021 23:20:05 +0000 (16:20 -0700)]
qa: update cephfs-shell distro to ubuntu 20.04

18.04 is no longer built.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 days agocrimson/seastore: fix OTree read invalid extent 41741/head
chunmei-liu [Mon, 7 Jun 2021 23:48:20 +0000 (16:48 -0700)]
crimson/seastore: fix OTree read invalid extent

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
5 days agoMerge PR #41229 into master
Sage Weil [Wed, 9 Jun 2021 19:55:25 +0000 (15:55 -0400)]
Merge PR #41229 into master

* refs/pull/41229/head:
qa/suites/upgrade/pacific-x/stress-split: add

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
5 days agoMerge pull request #41729 from tchaikov/wip-std-filesystem-gcc-8
Kefu Chai [Wed, 9 Jun 2021 15:41:47 +0000 (23:41 +0800)]
Merge pull request #41729 from tchaikov/wip-std-filesystem-gcc-8

*: stop using <experimental/filesystem> as an alternative

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 days agoMerge pull request #41791 from rzarzynski/wip-crimson-monc-no_auth_req_when_inactive
Kefu Chai [Wed, 9 Jun 2021 15:40:21 +0000 (23:40 +0800)]
Merge pull request #41791 from rzarzynski/wip-crimson-monc-no_auth_req_when_inactive

crimson/monc: don't serve auth requests without active mon connection.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 days agoMerge pull request #41782 from sseshasa/wip-fix-standalone-test
Neha Ojha [Wed, 9 Jun 2021 15:28:05 +0000 (08:28 -0700)]
Merge pull request #41782 from sseshasa/wip-fix-standalone-test

qa/standalone: Use osd op queue = wpq in activate_osd() within ceph-helpers.sh.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 days agocommon/strtol: do not check for existence of <charconv> 41729/head
Kefu Chai [Mon, 7 Jun 2021 09:18:19 +0000 (17:18 +0800)]
common/strtol: do not check for existence of <charconv>

<charconv> is available since GCC-8, see https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2017

> Elementary string conversions P0067R5 11.1 (integral types supported since 8.1) __has_include(<charconv>), __cpp_lib_to_chars >= 201611

since we always have the access to GCC-8.1 and up, there is no need to
detect the existence of <charconv> anymore.

also, because GCC-11 introduced the support of float types support,
update the comment to reflect the change.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 days ago*: always include <filesystem>
Kefu Chai [Mon, 7 Jun 2021 08:31:11 +0000 (16:31 +0800)]
*: always include <filesystem>

since there is no need to be compatible with GCC older than GCC-8, so
there is no need to use <experimental/filesystem> as an alternative of
<filesystem> anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 days agoos/bluestore/bluestore_tool: do not use boost:filesystem as alternative
Kefu Chai [Mon, 7 Jun 2021 08:21:22 +0000 (16:21 +0800)]
os/bluestore/bluestore_tool: do not use boost:filesystem as alternative

reverts 9dedabde520907f99727edf9f916ebb55ccdaa87

since there is no need to be compatible with GCC older than GCC-8, so
there is no need to use boost::filesystem as an alternative of
std::filesystem anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 days agocmake: stop detecting <experimental/filesystem>
Kefu Chai [Mon, 7 Jun 2021 08:39:22 +0000 (16:39 +0800)]
cmake: stop detecting <experimental/filesystem>

since we've dropped the support of GCC older than v8.0, there is no need
to detect <experimental/filesystem>

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 days agocmake: require GCC-8.1 and up
Kefu Chai [Mon, 7 Jun 2021 08:17:09 +0000 (16:17 +0800)]
cmake: require GCC-8.1 and up

for better C++17 support, for instance for a better std::filesystem
support.

the reason why 8.1 is required is that ubuntu focal provides GCC-8.1,
and RHEL/CentOS8 provides GCC-8.4.1. so we only test the build on
GCC-8.1 and up so far.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 days agopybind/mg/progress: Disregard unreported pgs 40480/head
Kamoltat [Thu, 13 May 2021 17:38:10 +0000 (17:38 +0000)]
pybind/mg/progress: Disregard unreported pgs

The global recovery event progress calculations only
takes into account pgs with `reported_epoch < start_epoch_of_event`
but sometimes the pgs doesn't get move before or after the creation
of the global recovery event, therefore this might result in a bug
where the global event gets stuck forever unless there is another
event that specifically makes the pgs that get stuck moves and updates
its `reported_epoch`.

Therefore, we decided to disregard pgs that are in active+clean state
but has `reported_epoch < start_epoch_of_event`.

Fixes: https://tracker.ceph.com/issues/49988

Signed-off-by: Kamoltat <ksirivad@redhat.com>