]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agorgw,test: Fix build errors in 32 bit platforms 45135/head
Luciano Lo Giudice [Wed, 23 Feb 2022 19:01:58 +0000 (16:01 -0300)]
rgw,test: Fix build errors in 32 bit platforms

When building on 32 bit platforms (tested on armhf), 2 errors were detected:
The first involves the compiler being unable to disambiguate a call to
deserialize an integer (apparently because there isn't a specialization for it).
The second involves invalid usage of the builtin reinterpret_cast.

Signed-off-by: Luciano Lo Giudice <luciano.logiudice@canonical.com>
2 years agoMerge pull request #47287 from tchaikov/wip-crimson-cleanup
Samuel Just [Tue, 26 Jul 2022 19:33:00 +0000 (12:33 -0700)]
Merge pull request #47287 from tchaikov/wip-crimson-cleanup

crimson/osd: remove unused variable

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agocrimson/osd: remove unused variable 47287/head
Kefu Chai [Tue, 26 Jul 2022 16:41:24 +0000 (00:41 +0800)]
crimson/osd: remove unused variable

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47279 from rzarzynski/wip-crimson-fix-repopreply-capture
Kefu Chai [Tue, 26 Jul 2022 15:24:50 +0000 (23:24 +0800)]
Merge pull request #47279 from rzarzynski/wip-crimson-fix-repopreply-capture

crimson/osd: don't capture a local, on-stack address of MOSDRepOpReply

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47277 from rzarzynski/wip-crimson-renew_on_auth_reply
Kefu Chai [Tue, 26 Jul 2022 15:23:44 +0000 (23:23 +0800)]
Merge pull request #47277 from rzarzynski/wip-crimson-renew_on_auth_reply

crimson/monc: renew tickets and rotating keys on MAuthReply

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47082 from rzarzynski/wip-doc-crimson-cephadm
zdover23 [Tue, 26 Jul 2022 14:45:09 +0000 (00:45 +1000)]
Merge pull request #47082 from rzarzynski/wip-doc-crimson-cephadm

doc/crimson: document installing crimson with cephadm

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #46945 from rhcs-dashboard/rbd-e2e
Nizamudeen A [Tue, 26 Jul 2022 14:21:06 +0000 (19:51 +0530)]
Merge pull request #46945 from rhcs-dashboard/rbd-e2e

mgr/dashboard: rbd-mirroring e2e tests

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2 years agodoc, crimson: document installing crimson with cephadm 47082/head
Radosław Zarzyński [Wed, 13 Jul 2022 12:33:06 +0000 (14:33 +0200)]
doc, crimson: document installing crimson with cephadm

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: comment on `dispatched` in OSD::ms_dispatch() 47279/head
Radoslaw Zarzynski [Tue, 26 Jul 2022 12:46:01 +0000 (12:46 +0000)]
crimson/osd: comment on `dispatched` in OSD::ms_dispatch()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge PR #47218 into main
Patrick Donnelly [Tue, 26 Jul 2022 12:59:56 +0000 (08:59 -0400)]
Merge PR #47218 into main

* refs/pull/47218/head:
mon/MDSMonitor: fix check for standby reversion
mds: update comment about standby-replay daemons holding a rank
qa: test for standby-replay removal in MDSMonitor::prepare_beacon
mds: add inject config for synthetic health message

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agocrimson/osd: move `conn` into the closure in OSD::ms_dispatch()
Radoslaw Zarzynski [Tue, 26 Jul 2022 12:45:07 +0000 (12:45 +0000)]
crimson/osd: move `conn` into the closure in OSD::ms_dispatch()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: don't capture a local, on-stack address of MOSDRepOpReply
Radoslaw Zarzynski [Tue, 26 Jul 2022 12:18:55 +0000 (12:18 +0000)]
crimson/osd: don't capture a local, on-stack address of MOSDRepOpReply

This flaw can explain the following crash observed at Sepia:

```
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-13661-g3a14f283/rpm/el8/BUILD/ceph-17.0.0-13661-g3a14f283/x86_64-redhat-linux-gnu/boost/include/boost/smart_ptr/intrusive_ptr.hpp:199: T* boost::intrusive_ptr<T>::operator->() const [with T = MOSDRepOpReply]: Assertion `px != 0' failed.
Aborting on shard 0.
Backtrace:
 0# gsignal in /lib64/libc.so.6
 1# abort in /lib64/libc.so.6
 2# 0x00007FB1346F4CD9 in /lib64/libc.so.6
 3# 0x00007FB13471A3F6 in /lib64/libc.so.6
 4# crimson::osd::OSD::handle_rep_op_reply(seastar::shared_ptr<crimson::net::Connection>, boost::intrusive_ptr<MOSDRepOpReply>) in ceph-osd
 5# 0x000055C4100FF8D0 in ceph-osd
 6# 0x000055C410100639 in ceph-osd
 7# crimson::osd::OSD::ms_dispatch(seastar::shared_ptr<crimson::net::Connection>, boost::intrusive_ptr<Message>) in ceph-osd
 8# crimson::net::ChainedDispatchers::ms_dispatch(seastar::shared_ptr<crimson::net::Connection>, boost::intrusive_ptr<Message>) in ceph-osd
 9# 0x000055C4165220EF in ceph-osd
10# 0x000055C416532791 in ceph-osd
11# 0x000055C4165328B2 in ceph-osd
12# 0x000055C40FDD33D5 in ceph-osd
13# 0x000055C40FDEE2CE in ceph-osd
14# 0x000055C40FDEEEF0 in ceph-osd
15# 0x000055C41CA41215 in ceph-osd
16# 0x000055C41CA97959 in ceph-osd
17# 0x000055C41CBF73A0 in ceph-osd
18# 0x000055C41CBF93F5 in ceph-osd
19# 0x000055C41C629A46 in ceph-osd
20# 0x000055C41C62E012 in ceph-osd
21# main in ceph-osd
22# __libc_start_main in /lib64/libc.so.6
23# _start in ceph-osd
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/monc: renew_rotating_keyring() doesn't assume the clock is monotonic 47277/head
Radoslaw Zarzynski [Tue, 26 Jul 2022 09:39:08 +0000 (09:39 +0000)]
crimson/monc: renew_rotating_keyring() doesn't assume the clock is monotonic

According to the `seastar::lowres_system_clock` reference:

> This clock has the same granularity as lowres_clock,
> but it is not required to be monotonic and its time
> points correspond to system time.

we should similar check as the classical `MonClient` does:

```cpp
  if ((now > last_rotating_renew_sent) &&
      double(now - last_rotating_renew_sent) < 1) {
    ldout(cct, 10) << __func__ << " called too often (last: "
                   << last_rotating_renew_sent << "), skipping refresh" << dendl;
    return 0;
  }
```

This check is rather paranoidal and the main reason behind it
in crimson is replicating the classical behaviour.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/monc: unify handling auth_service_ticket_ttl with classical OSD
Radoslaw Zarzynski [Tue, 26 Jul 2022 09:31:38 +0000 (09:31 +0000)]
crimson/monc: unify handling auth_service_ticket_ttl with classical OSD

In the classical `MonClient` the `auth_service_ticket_ttl` is lower
bounded to `30` units.

```cpp
  utime_t now = ceph_clock_now();
  utime_t cutoff = now;
  cutoff -= std::min(30.0, cct->_conf->auth_service_ticket_ttl / 4.0);
  utime_t issued_at_lower_bound = now;
  issued_at_lower_bound -= cct->_conf->auth_service_ticket_ttl;
  if (!rotating_secrets->need_new_secrets(cutoff)) {
    ldout(cct, 10) << "_check_auth_rotating have uptodate secrets (they expire after " << cutoff << ")" << dendl;
    rotating_secrets->dump_rotating();
    return 0;
  }
```

The unification affects also the debug mesages.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/monc: improve debugs around Connection::renew_tickets()
Radoslaw Zarzynski [Mon, 25 Jul 2022 14:48:07 +0000 (14:48 +0000)]
crimson/monc: improve debugs around Connection::renew_tickets()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/monc: renew tickets and rotating keys on MAuthReply
Radoslaw Zarzynski [Mon, 25 Jul 2022 14:42:00 +0000 (14:42 +0000)]
crimson/monc: renew tickets and rotating keys on MAuthReply

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocmake: add rbd-mirror executable to run-make-check 46945/head
Deepika Upadhyay [Mon, 4 Jul 2022 15:27:29 +0000 (20:57 +0530)]
cmake: add rbd-mirror executable to run-make-check

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit e2b46020317e4d4e37c281a6b1817711e0826a37)

2 years agomgr/dashboard: rbd-mirroring e2e tests
Nizamudeen A [Mon, 4 Jul 2022 07:39:43 +0000 (13:09 +0530)]
mgr/dashboard: rbd-mirroring e2e tests

Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #47168 from guits/get-devices-vgs-per-dev
Guillaume Abrioux [Tue, 26 Jul 2022 07:56:32 +0000 (09:56 +0200)]
Merge pull request #47168 from guits/get-devices-vgs-per-dev

ceph-volume: do not call get_device_vgs() per devices

2 years agoMerge pull request #47239 from cyx1231st/wip-seastore-cleaner-alloc-trim
Yingxin [Tue, 26 Jul 2022 07:18:22 +0000 (15:18 +0800)]
Merge pull request #47239 from cyx1231st/wip-seastore-cleaner-alloc-trim

crimson/os/seastore: cleanups and improvements to trimming

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoMerge pull request #46554 from adamemerson/wip-up-the-boost
Kefu Chai [Mon, 25 Jul 2022 23:39:10 +0000 (07:39 +0800)]
Merge pull request #46554 from adamemerson/wip-up-the-boost

build: Bump Boost version to 1.79

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47003 from liu-chunmei/seastore-list-objects
Liu-Chunmei [Mon, 25 Jul 2022 23:14:59 +0000 (16:14 -0700)]
Merge pull request #47003 from liu-chunmei/seastore-list-objects

crimson/seastore: fix list objects

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #47217 from tchaikov/wip-ntstatus
Kefu Chai [Mon, 25 Jul 2022 22:56:36 +0000 (06:56 +0800)]
Merge pull request #47217 from tchaikov/wip-ntstatus

common/win32,dokan: include bcrypt.h for NTSTATUS

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #47164 from Dazul/add-config-availability-drive-group
Adam King [Mon, 25 Jul 2022 20:19:33 +0000 (16:19 -0400)]
Merge pull request #47164 from Dazul/add-config-availability-drive-group

mgr/cephadm: add parsing for config on osd specs

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47120 from yehudasa/wip-56576
Casey Bodley [Mon, 25 Jul 2022 19:49:07 +0000 (15:49 -0400)]
Merge pull request #47120 from yehudasa/wip-56576

rgw: bucket sync: fetch source bucket info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47083 from cbodley/wip-rgw-sysobjctx-rm
Casey Bodley [Mon, 25 Jul 2022 19:03:35 +0000 (15:03 -0400)]
Merge pull request #47083 from cbodley/wip-rgw-sysobjctx-rm

rgw: sysobj service cleanup

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #35870 from soumyakoduri/json_fix
Casey Bodley [Mon, 25 Jul 2022 16:17:58 +0000 (12:17 -0400)]
Merge pull request #35870 from soumyakoduri/json_fix

JSONFormattable incorrectly sets a string starting with digit as int

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #47255 from tchaikov/wip-fmt-containers
Kefu Chai [Mon, 25 Jul 2022 15:41:03 +0000 (23:41 +0800)]
Merge pull request #47255 from tchaikov/wip-fmt-containers

include/types_fmt: use fmt::join() to format containers

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2 years agocrimson/os/seastore/async_cleaner: fix and improve update_journal_tails() 47239/head
Yingxin Cheng [Mon, 25 Jul 2022 08:28:42 +0000 (16:28 +0800)]
crimson/os/seastore/async_cleaner: fix and improve update_journal_tails()

* Accept JOURNAL_SEQ_NULL dirty_tail when it is pending.
* Enforce monotone increasing to the journal tails.
* Enforce journal tails to be always behind the journal head.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore/cache: add logs to debug alloc_tail trimming
Yingxin Cheng [Mon, 25 Jul 2022 08:26:09 +0000 (16:26 +0800)]
crimson/os/seastore/cache: add logs to debug alloc_tail trimming

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #47256 from anthonyeleven/anthonyeleven/how-to-fix-pg-autoscaler...
Anthony D'Atri [Mon, 25 Jul 2022 14:17:18 +0000 (07:17 -0700)]
Merge pull request #47256 from anthonyeleven/anthonyeleven/how-to-fix-pg-autoscaler-empty-output

doc/radosgw/operations: Document how to fix empty autoscale-status output

2 years agoinclude/types_fmt: use fmt::join() to format containers 47255/head
Kefu Chai [Sun, 24 Jul 2022 16:16:33 +0000 (00:16 +0800)]
include/types_fmt: use fmt::join() to format containers

* add formatter for classes with stream insertion operator.
  as in fmtlib v6.1.2, which is the one shipped with ubuntu focal,
  fmt::join() requires the printed element to have formatter.
  see https://github.com/fmtlib/fmt/issues/2040 and
  https://github.com/fmtlib/fmt/issues/1462
* use template parameter pack to represent container template argument,
  simpler this way. also, this enables us to print specialized
  classes which uses non-default template parameters.
* use fmt::join() to print container elements. see also
  https://fmt.dev/latest/api.html#_CPPv4I0EN3fmt4joinE9join_viewIN6detail10iterator_tI5RangeEEN6detail10sentinel_tI5RangeEEERR5Range11string_view

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agofmt: pickup v8.1.1
Kefu Chai [Mon, 25 Jul 2022 12:42:58 +0000 (20:42 +0800)]
fmt: pickup v8.1.1

update fmt submodule so we can use fmt::join() without defining
formatter specialization for types which already have `operator<<()`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agocmake: disable FMT_USE_TZSET for WIN32
Kefu Chai [Mon, 25 Jul 2022 13:57:57 +0000 (21:57 +0800)]
cmake: disable FMT_USE_TZSET for WIN32

otherwise we'd have following FTBFS with MinGW, like:

../src/fmt/include/fmt/chrono.h: In lambda function:
../src/fmt/include/fmt/chrono.h:952:5: error: '_tzset' was not declared
in this scope; did you mean 'tzset'?
  952 |     _tzset();
      |     ^~~~~~
      |     tzset

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agomon/MDSMonitor: fix check for standby reversion 47218/head
Patrick Donnelly [Thu, 21 Jul 2022 20:34:05 +0000 (16:34 -0400)]
mon/MDSMonitor: fix check for standby reversion

A standby-replay daemon always has a rank, so this check is completely
wrong. If a beacon from a standby-replay daemon reaches
MDSMonitor::prepare_beacon, it will always be evicted/removed by the
mons. This is rare (usually a reply occurs directly from
MDSMonitor::preprocess_beacon) but can happen in certain circumstances,
like a health warning included in the beacon.

Fixes: https://tracker.ceph.com/issues/56666
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agomds: update comment about standby-replay daemons holding a rank
Patrick Donnelly [Fri, 22 Jul 2022 17:22:01 +0000 (13:22 -0400)]
mds: update comment about standby-replay daemons holding a rank

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoqa: test for standby-replay removal in MDSMonitor::prepare_beacon
Patrick Donnelly [Thu, 21 Jul 2022 20:28:08 +0000 (16:28 -0400)]
qa: test for standby-replay removal in MDSMonitor::prepare_beacon

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agomds: add inject config for synthetic health message
Patrick Donnelly [Thu, 21 Jul 2022 19:46:58 +0000 (15:46 -0400)]
mds: add inject config for synthetic health message

This will be used to force the MDSMonitor to process a beacon in
MDSMonitor::prepare_beacon.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocmake: bump up required fmt version
Kefu Chai [Mon, 25 Jul 2022 12:23:14 +0000 (20:23 +0800)]
cmake: bump up required fmt version

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agocrimson/os/seastore/cache: write cleaner dirty tail if the oldest dirty extent is...
Yingxin Cheng [Mon, 25 Jul 2022 08:20:04 +0000 (16:20 +0800)]
crimson/os/seastore/cache: write cleaner dirty tail if the oldest dirty extent is still pending

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: drop journal_seq_t NO_DELTAS
Yingxin Cheng [Mon, 25 Jul 2022 05:35:37 +0000 (13:35 +0800)]
crimson/os/seastore: drop journal_seq_t NO_DELTAS

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: improve logs
Yingxin Cheng [Mon, 25 Jul 2022 05:22:32 +0000 (13:22 +0800)]
crimson/os/seastore: improve logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agodoc/radosgw/operations: Document how to fix empty autoscale-status output 47256/head
Anthony D'Atri [Mon, 25 Jul 2022 03:14:02 +0000 (20:14 -0700)]
doc/radosgw/operations: Document how to fix empty autoscale-status output

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agoMerge pull request #47014 from ktdreyer/no-system-pmdk-s390x
Ilya Dryomov [Sun, 24 Jul 2022 07:37:55 +0000 (09:37 +0200)]
Merge pull request #47014 from ktdreyer/no-system-pmdk-s390x

ceph.spec.in: disable system_pmdk on s390x

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #47242 from idryomov/wip-56676
Ilya Dryomov [Sat, 23 Jul 2022 09:33:47 +0000 (11:33 +0200)]
Merge pull request #47242 from idryomov/wip-56676

librbd: tweak misleading "image is still primary" error message

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2 years agocrimson/test: add more list_objects test in test_seastore 47003/head
chunmei-liu [Sat, 16 Jul 2022 06:09:42 +0000 (23:09 -0700)]
crimson/test: add more list_objects test in test_seastore

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agocrimson/seastore: fix test_seastore
chunmei-liu [Fri, 8 Jul 2022 08:52:09 +0000 (01:52 -0700)]
crimson/seastore: fix test_seastore

collection default has pool(0), and NO_SHARD,
so the ghobject_t should have the same setting.
and ghobject_t default hash not within 1<<(32-4),
so set collection bits to be 0.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2 years agocrimson/seastore: fix list_onodes bug
chunmei-liu [Fri, 15 Jul 2022 05:16:27 +0000 (22:16 -0700)]
crimson/seastore: fix list_onodes bug

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2 years agocrimson/seastore: calculate list_objects ranges by coll_range_(temp)_start/end
chunmei-liu [Wed, 6 Jul 2022 09:10:39 +0000 (02:10 -0700)]
crimson/seastore: calculate list_objects ranges by coll_range_(temp)_start/end

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #46711 from 5cs/fix-backoff-retries
J. Eric Ivancich [Fri, 22 Jul 2022 18:39:22 +0000 (14:39 -0400)]
Merge pull request #46711 from 5cs/fix-backoff-retries

rgwlc: fix backoff retries counter

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #47005 from 5cs/fix-rm-LCEntry
J. Eric Ivancich [Fri, 22 Jul 2022 18:38:11 +0000 (14:38 -0400)]
Merge pull request #47005 from 5cs/fix-rm-LCEntry

rgwlc: fix removing LCEntry

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2 years agoMerge pull request #47078 from Huber-ming/nonexists_noaccounted
J. Eric Ivancich [Fri, 22 Jul 2022 18:37:41 +0000 (14:37 -0400)]
Merge pull request #47078 from Huber-ming/nonexists_noaccounted

cls_rgw: nonexists object shoud not be accounted when check_index

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agoMerge pull request #47229 from kotreshhr/mgr-legacy-to-upgrade-config-fix
David Galloway [Fri, 22 Jul 2022 17:42:27 +0000 (13:42 -0400)]
Merge pull request #47229 from kotreshhr/mgr-legacy-to-upgrade-config-fix

  mgr/volumes: Fix subvolume discover during upgrade

2 years agoMerge pull request #47244 from ceph/fix-release-order
David Galloway [Fri, 22 Jul 2022 16:18:34 +0000 (12:18 -0400)]
Merge pull request #47244 from ceph/fix-release-order

doc/releases: Fix quincy order

2 years agotest/encoding: refactor EncodingException::Macros 47217/head
Kefu Chai [Fri, 22 Jul 2022 13:08:41 +0000 (21:08 +0800)]
test/encoding: refactor EncodingException::Macros

so it is more compacted

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/encoding: verify that e.what() starts with expected str
Kefu Chai [Fri, 22 Jul 2022 12:54:43 +0000 (20:54 +0800)]
test/encoding: verify that e.what() starts with expected str

boost changes the way how it prints boost::system::system_error in
boost 1.79 -- it appends the stringified error_category at end of
exception::what(), and our buffer::malformed_input is a subclass
of boost::system::system_error.

so we cannot just compare the return value of what() with the
expected string, to be more future proof, let's check if i
starts with the expected string instead.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agocommon/win32,dokan: include bcrypt.h for NTSTATUS
Kefu Chai [Thu, 21 Jul 2022 23:28:37 +0000 (07:28 +0800)]
common/win32,dokan: include bcrypt.h for NTSTATUS

to avoid the conflicting declaration of NTSTATUS from bcrypt.h and our
own typedef. as after switching to boost 1.79, we would have following compiling
failure:

In file included from ../src/dokan/options.cc:14:
../src/dokan/ceph_dokan.h:16:15: error: conflicting declaration 'typedef DWORD NTSTATUS'
   16 | typedef DWORD NTSTATUS;
      |               ^~~~~~~~
In file included from ../build.deps/mingw/boost/include/boost/asio/impl/connect_pipe.ipp:29,
                 from ../build.deps/mingw/boost/include/boost/asio/connect_pipe.hpp:79,
                 from ../build.deps/mingw/boost/include/boost/asio.hpp:64,
                 from ../src/include/win32/winsock_wrapper.h:20,
                 from <command-line>:
/usr/share/mingw-w64/include/bcrypt.h:27:16: note: previous declaration as 'typedef LONG NTSTATUS'
   27 |   typedef LONG NTSTATUS,*PNTSTATUS;
      |                ^~~~~~~~

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47124 from tchaikov/spec-mibs
Kefu Chai [Fri, 22 Jul 2022 15:54:03 +0000 (23:54 +0800)]
Merge pull request #47124 from tchaikov/spec-mibs

ceph.spec.in: correct "%files mib" section

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Justin Caratzas <jcaratza@redhat.com>
2 years agoMerge pull request #47243 from tchaikov/wip-gantt
Kefu Chai [Fri, 22 Jul 2022 15:50:59 +0000 (23:50 +0800)]
Merge pull request #47243 from tchaikov/wip-gantt

doc/_ext: sort the release date

Reviewed-by: David Galloway <dgallowa@redhat.com>
2 years agodoc/releases: Fix quincy order 47244/head
David Galloway [Fri, 22 Jul 2022 15:08:05 +0000 (11:08 -0400)]
doc/releases: Fix quincy order

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agobuild: Bump Boost version to 1.79 46554/head
Adam C. Emerson [Mon, 7 Mar 2022 22:14:50 +0000 (17:14 -0500)]
build: Bump Boost version to 1.79

Also remove bind_allocator, as Boost.Asio now provides this function.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agodoc/_ext: sort the release date 47243/head
Kefu Chai [Fri, 22 Jul 2022 14:50:50 +0000 (22:50 +0800)]
doc/_ext: sort the release date

so the release in reverse chronologically order, so that the newer
one is listed at the top.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agolibrbd: tweak misleading "image is still primary" error message 47242/head
Ilya Dryomov [Fri, 22 Jul 2022 13:23:57 +0000 (15:23 +0200)]
librbd: tweak misleading "image is still primary" error message

m_promotion_state == PROMOTION_STATE_NON_PRIMARY doesn't say anything
about the remote image.  It could still be primary but it could also be
demoted.

Fixes: https://tracker.ceph.com/issues/56676
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoqa: Remove unused variable 47229/head
Kotresh HR [Fri, 22 Jul 2022 13:50:33 +0000 (19:20 +0530)]
qa: Remove unused variable

Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agoqa: validate subvolume discover on upgrade
Kotresh HR [Fri, 4 Feb 2022 09:58:39 +0000 (15:28 +0530)]
qa: validate subvolume discover on upgrade

Validate subvolume discover on upgrade from
legacy subvolume to v1. The handcrafted
`.meta' file on legacy subvolume root should
not be used for any subvolume apis like getpath,
authorize.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #47237 from rosinL/fix-move
Kefu Chai [Fri, 22 Jul 2022 12:14:36 +0000 (20:14 +0800)]
Merge pull request #47237 from rosinL/fix-move

crimson/osd/osd: fix sigsegv caused by reusing the moved object

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agocrimson/os/seastore/async_cleaner: clarify invariants
Yingxin Cheng [Fri, 22 Jul 2022 09:18:52 +0000 (17:18 +0800)]
crimson/os/seastore/async_cleaner: clarify invariants

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: consider dirty tail during replay
Yingxin Cheng [Fri, 22 Jul 2022 09:00:51 +0000 (17:00 +0800)]
crimson/os/seastore: consider dirty tail during replay

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore/async_cleaner: decouple gc_trim_alloc() and gc_trim_dirty()
Yingxin Cheng [Fri, 22 Jul 2022 08:58:30 +0000 (16:58 +0800)]
crimson/os/seastore/async_cleaner: decouple gc_trim_alloc() and gc_trim_dirty()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: cleanup, consolidate alloc_tail/dirty_tail naming
Yingxin Cheng [Fri, 22 Jul 2022 08:54:25 +0000 (16:54 +0800)]
crimson/os/seastore: cleanup, consolidate alloc_tail/dirty_tail naming

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: update journal tail when writting the segment header in mkfs
Yingxin Cheng [Fri, 22 Jul 2022 06:18:56 +0000 (14:18 +0800)]
crimson/os/seastore: update journal tail when writting the segment header in mkfs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/osd/osd: fix sigsegv caused by reusing the moved objcect 47237/head
luo rixin [Fri, 22 Jul 2022 08:48:14 +0000 (16:48 +0800)]
crimson/osd/osd: fix sigsegv caused by reusing the moved objcect

In some case, `m` will be moved and setted nullptr before
`m->get_spg()` be called, causing a segmentation fault.

Signed-off-by: luo rixin <luorixin@huawei.com>
2 years agoMerge pull request #47206 from cyx1231st/wip-seastore-cleaner-tail-committed
Yingxin [Fri, 22 Jul 2022 08:27:59 +0000 (16:27 +0800)]
Merge pull request #47206 from cyx1231st/wip-seastore-cleaner-tail-committed

crimson/os/seastore: simplify journal tails

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agomgr/volumes: V2 Fix for test_subvolume_retain_snapshot_invalid_recreate
Kotresh HR [Fri, 22 Jul 2022 07:55:18 +0000 (13:25 +0530)]
mgr/volumes: V2 Fix for test_subvolume_retain_snapshot_invalid_recreate

Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agomgr/volumes: Fix subvolume discover during upgrade
Kotresh HR [Fri, 22 Jul 2022 07:53:39 +0000 (13:23 +0530)]
mgr/volumes: Fix subvolume discover during upgrade

Fixes the subvolume discover to use the correct
metadata file after an upgrade from legacy subvolume
to v1. The fix makes sure, it doesn't use the
handcrafted metadata file placed in the subvolume
root of legacy subvolume.

Co-authored-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
Co-authored-by: Dan van der Ster <daniel.vanderster@cern.ch>
Co-authored-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2 years agocrimson/os/seastore: simplify journal tails 47206/head
Yingxin Cheng [Thu, 21 Jul 2022 09:16:50 +0000 (17:16 +0800)]
crimson/os/seastore: simplify journal tails

* append latest dirty tail as journal delta, in order to simplify and
  drop journal_tail_committed in the cleaner.
* simplify misc journal tails into alloc_tail and dirty_tail, with
  proper renaming.
* move journal-tail recovery logic from cleaner to journal.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: misc cleanups
Yingxin Cheng [Thu, 21 Jul 2022 08:58:47 +0000 (16:58 +0800)]
crimson/os/seastore: misc cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: store valid journal tail during mkfs
Yingxin Cheng [Thu, 21 Jul 2022 08:50:15 +0000 (16:50 +0800)]
crimson/os/seastore: store valid journal tail during mkfs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #47219 from ceph/sec
Gregory Farnum [Fri, 22 Jul 2022 02:28:07 +0000 (19:28 -0700)]
Merge pull request #47219 from ceph/sec

16.2.10 and 17.2.2 Release Notes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #47212 from tchaikov/wip-aligned-free
Kefu Chai [Thu, 21 Jul 2022 23:08:53 +0000 (07:08 +0800)]
Merge pull request #47212 from tchaikov/wip-aligned-free

include/compat: define aligned_free() as a function not a macro

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #46890 from aclamk/wip-fix-54547
Yuri Weinstein [Thu, 21 Jul 2022 22:47:55 +0000 (15:47 -0700)]
Merge pull request #46890 from aclamk/wip-fix-54547

os/bluestore: Fix collistion between BlueFS and BlueStore deferred writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2 years agoMerge pull request #36282 from rzarzynski/wip-bl-drop-clone
Yuri Weinstein [Thu, 21 Jul 2022 22:33:04 +0000 (15:33 -0700)]
Merge pull request #36282 from rzarzynski/wip-bl-drop-clone

common/bl, kv, tests: drop MemDB and simplify buffer::ptr and buffer::raw

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #47213 from Matan-B/wip-matanb-snapmapper-pendingnote
Neha Ojha [Thu, 21 Jul 2022 22:30:36 +0000 (15:30 -0700)]
Merge pull request #47213 from Matan-B/wip-matanb-snapmapper-pendingnote

PendingReleaseNotes: add a note about SnapMapper key conversion

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #46941 from Matan-B/wip-matanb-make-writeable
Yuri Weinstein [Thu, 21 Jul 2022 20:30:17 +0000 (13:30 -0700)]
Merge pull request #46941 from Matan-B/wip-matanb-make-writeable

osd/PrimaryLogPG: make_writeable nit fixes

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #46561 from NitzanMordhai/wip-nitzan-add-pglog-dups-length
Yuri Weinstein [Thu, 21 Jul 2022 20:29:07 +0000 (13:29 -0700)]
Merge pull request #46561 from NitzanMordhai/wip-nitzan-add-pglog-dups-length

osd, mon: add pglog dups length

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoPendingReleaseNotes: add a note about SnapMapper key coversion 47213/head
Matan Breizman [Thu, 21 Jul 2022 16:23:58 +0000 (16:23 +0000)]
PendingReleaseNotes: add a note about SnapMapper key coversion

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agodoc: 16.2.10 Release notes 47219/head
David Galloway [Thu, 21 Jul 2022 16:45:45 +0000 (12:45 -0400)]
doc: 16.2.10 Release notes

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agodoc: 17.2.2 Release notes
David Galloway [Thu, 21 Jul 2022 16:43:59 +0000 (12:43 -0400)]
doc: 17.2.2 Release notes

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agoMerge pull request #47001 from yaarith/fix-telemetry-changed-warning
Laura Flores [Thu, 21 Jul 2022 16:42:09 +0000 (11:42 -0500)]
Merge pull request #47001 from yaarith/fix-telemetry-changed-warning

mgr/telemetry: reset health warning after re-opting-in

2 years agoMerge pull request #47209 from yuvalif/wip-yuval-fix-56170
Yuval Lifshitz [Thu, 21 Jul 2022 16:31:48 +0000 (19:31 +0300)]
Merge pull request #47209 from yuvalif/wip-yuval-fix-56170

rgw/lua: make assignment of char* to std::string explicit

reviewed-by: tchaikov, cbodley

2 years agoinclude/compat: define aligned_free() as a function not a macro 47212/head
Kefu Chai [Thu, 21 Jul 2022 16:09:33 +0000 (00:09 +0800)]
include/compat: define aligned_free() as a function not a macro

so it does not get substituted by the preprocessor unconditionally.
this change helps to address the compiling failure when using boost
1.79 using MinGW, like:

../build.deps/mingw/boost/include/boost/align/aligned_allocator.hpp: In member function 'void boost::alignment::aligned_allocator<T, Alignment>::deallocate(boost::alignment::aligned_allocator<T, Alignment>::pointer, boost::alignment::aligned_allocator<T, Alignment>::size_type)':
../src/include/compat.h:352:27: error: '_aligned_free' is not a member of 'boost::alignment'; did you mean 'aligned_free'?
  352 | #define aligned_free(ptr) _aligned_free(ptr)
      |                           ^~~~~~~~~~~~~

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agodoc: CVE-2022-0670
David Galloway [Thu, 21 Jul 2022 16:11:11 +0000 (12:11 -0400)]
doc: CVE-2022-0670

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agodoc: Add missing 16.2.9 release
David Galloway [Thu, 21 Jul 2022 15:52:52 +0000 (11:52 -0400)]
doc: Add missing 16.2.9 release

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agorgw/lua: make assignment of char* to std::string explicit 47209/head
Yuval Lifshitz [Thu, 21 Jul 2022 12:38:33 +0000 (12:38 +0000)]
rgw/lua: make assignment of char* to std::string explicit

issue is happening on RHEL8, when using: g++ (GCC) 8.5.0

Fixes: https://tracker.ceph.com/issues/56170
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #47203 from SUSE/wip-buildrequires-ndctl-suse
Ilya Dryomov [Thu, 21 Jul 2022 14:29:54 +0000 (16:29 +0200)]
Merge pull request #47203 from SUSE/wip-buildrequires-ndctl-suse

ceph.spec.in: use BuildRequires libndctl-devel for SUSE distros

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agocrimson/os/seastore: drop journal tail info from segment_tail_t
Yingxin Cheng [Thu, 21 Jul 2022 07:49:54 +0000 (15:49 +0800)]
crimson/os/seastore: drop journal tail info from segment_tail_t

We will write a segment header right after writing a segment tail during
rolling, so there is no need to store journal tail in segment tail.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: move Transaction::src_t to seastore_types.h
Yingxin Cheng [Thu, 21 Jul 2022 07:36:43 +0000 (15:36 +0800)]
crimson/os/seastore: move Transaction::src_t to seastore_types.h

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoceph.spec.in: use BuildRequires libndctl-devel for SUSE distros 47203/head
Tim Serong [Thu, 21 Jul 2022 08:08:57 +0000 (18:08 +1000)]
ceph.spec.in: use BuildRequires libndctl-devel for SUSE distros

Fixes: b6b24852a6d1c6d8e785e15661f3bf29d662417f
Signed-off-by: Tim Serong <tserong@suse.com>
2 years agoJSONFormattable incorrectly sets a string starting with digit as int 35870/head
Soumya Koduri [Thu, 24 Jun 2021 18:00:45 +0000 (23:30 +0530)]
JSONFormattable incorrectly sets a string starting with digit as int

JSONFormattable::set calls JSONParser::parse() to check if the given string
is a valid JSON object, which internally uses json_spirit::read to determine
the string type and copy it into data.
This routine incorrectly sets data type to integer if the string starts with
digit and copies only the first set of numeric values of the string.

To work-around this issue, verify if the entire string is parsed to
determine if its valid json data type.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #46917 from ceph/releases-always-main
David Galloway [Wed, 20 Jul 2022 20:05:03 +0000 (16:05 -0400)]
Merge pull request #46917 from ceph/releases-always-main

.readthedocs.yml: Always build latest doc/releases pages