ceph-release-notes: Support for markdown for new ceph.io site

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #42026 from rhcs-dashboard/bucket-name-regex-fix

mgr/dashboard: Fix bucket name input allowing space in the value

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41353 from a16bitsysop/mgr-node_modules

pybind/mgr/CMakeLists.txt: exclude files not used at runtime

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #41954 from pkalever/notrim

rbd-nbd: support notrim option with map command

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #42089 from tchaikov/wip-xattr-cleanup

osd, crimson/osd: do_cmp_xattr() related cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #42054 from tchaikov/wip-os-xattrs

os: use transparent comparator in ObjectStore::getattrs()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #42043 from tchaikov/wip-ceph-argparse-ann

pybind/ceph_argparse: do not set self.typeargs in ctor of CephArgtype

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42087 from tchaikov/wip-common-buffers

common/buffers: check _num directly in list::c_str()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42084 from tchaikov/wip-49638

qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #42047 from tchaikov/wip-autoscaler-ann

pybind/mgr/pg_autoscaler: add typing annotations and cleanups

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42082 from fengchunsong/master

common: Use double instead of long double to improve performance

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37866 from pritha-srivastava/wip-sts-47809

rgw/sts: fix for copy object operation using sts

qa/workunits/rbd-nbd: add notrim test

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

mgr/dashboard: Fix bucket name input allows space in the value

Fixes: https://tracker.ceph.com/issues/51368
Signed-off-by: Nizamudeen A <nia@redhat.com>

pybind/mgr/pg_autoscaler: add typing annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: do not index pool_root with pool_name

pool_root is indexed by pool_id, and we never index it with pool_name.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: do not create a tuple long_desc

long_desc is supposed to be a str, not a tuple of str.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out

as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: define commands using CLICommand

simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: define options using Option

more consistent and less error-prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41919 from agayev/zoned-more-cleaning-support

os/bluestore: More support for cleaning zones.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #42100 from rzarzynski/wip-crimson-assert_moveable

crimson: introduce assert_moveable().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

pybind/ceph_argparse: add more type annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41976 from tchaikov/wip-crimson-alienstore-lockless

crimson/os: use lockfree queue for sharded queue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #42101 into master

* refs/pull/42101/head:
cephsqlite: add comment on atexit

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/alienstore: use semaphore to manage tasks in thread pool

* implement std::counting_semaphore in C++17
* use the homebrew counting_semaphore as the synchronization primitive
  to access the pending tasks, for better performance than the
  implementation using mutex and condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephsqlite: add comment on atexit

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

crimson: introduce assert_moveable().

In C++ `std::moving` a `const`-qualified value yields a constant
r-value reference (`const T&&`) which won't be matched with a callable
taking non-constant r-value reference (like move constructors) but
can play with one taking a constant l-value reference (like copy
constructors do). This behaviour is surprising especially in lambas
where adding or removing the `mutable` specifier may lead to different
behaviour. The problem isn't obvious and it's easy to wrongly drop
the `mutable` druing a clean-up. Therefore introducing a tool for
developers to fail at compile-time if that happens seems desired.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42002 from tchaikov/wip-common-armor

common/armor: mark dst_end a const pointer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42035 from batrick/i51372

libcephsqlite: shutdown RADOS in destructor

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42094 from zdover23/wip-doc-upgrading-ceph-monitoring-the-upgrade-2021-06-29

doc/cephadm: improving "Monitoring the Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42079 from zdover23/wip-doc-upgrading-ceph-first-section-2021-06-29

doc/cephadm: improve "Upgrading Ceph" (main)

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

doc/cephadm: improving "Monitoring the Upgrade"

This PR improves the section "Monitoring the Upgrade"
in the "Upgrading Ceph" chapter of the cephadm documentation.

This PR introduces a couple of section breaks with signposting
information in their titles, and rewrites some sentences in order
to reduce the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42042 from tchaikov/wip-51375

common/options: convert a millisecs opt to a chrono::milliseconds and cleanups

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os/alienstore: return attrs retrieved from bluestore directly

since ObjectStore returns a map<...,less<>> already, there is not point to
convert the returned map to the type expected by AlienStore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os: use transparent comparator in ObjectStore::getattrs()

for two reasons:

- better performance when looking on in the return map if the key
  is not a string, as we don't need to create a temporary string
  as the key
- improve the performance of crimson::AlienStore, as the latter
  uses the transparent comparator. as, without this change, we'd
  have to perform a deep copy to fill up the returned map with
  its non-transparent-comparator version.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: use lockfree queue for sharded queue

each sharded queue has multiple producers and a single consumer queue,
but the producer side is in seastar world, so ideally, we should not
guard the queue using a POSIX lock.

in this change, the lockfree multiple-writer/multiple-reader queue is
used to replace the std::queue to drop the lock guarding it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr,mon: s/boost::optional/std::optional/

since the encoding schemes of boost::optional and std::optional are
identical, they can be used interchangeably.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: s/boost::variant/std::variant/

we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: s/boost::variant/std::variant/

we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42013 from ronen-fr/wip-ronenf-scrubs-config

qa/suites/rados: add simultaneous scrubs to the thrasher

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42031 from sebastian-philipp/normalize_image_digest_local-registry

cephadm: Fix normalize_image_digest for local registries

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #41933 from liewegas/fix-48311

mgr: set debug_mgr=2/5 (so INFO goes to mgr log by default)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41999 from tchaikov/wip-rpm-deb

rpm,debian: cleanups related to python3-setuptools dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #42076 from liu-chunmei/crimson-pgnls

crimson: fix pgnls exception

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: reorder #includes

so the oder of includes complies to https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: use std::from_chars() to convert str to integer

* no need to create a temporary string for using strtoull()
* use std::from_chars() so it's consistent with its counterpart in
  in crimson.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: check invalid input in do_xattr_cmp_u64()

we should return -EINVAL if the string does not represent a decimal
integer.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: include used header

for using std::from_chars()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: extract do_cmp_xattr()

to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: extract do_cmp_xattr()

to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42039 from liu-chunmei/crimson-cmpxattr

crimson: add cmp_xatt support

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/buffers: check _num directly in list::c_str()

no need to create temporary iterator for comparing it with cend().

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: fix pgnls exception

has_pg_op is always false, since m->ops is empty at that time.
so pgnls operation will go to process_op and report unknown operations.
move m->finish_decode ahead to fill m->ops.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

qa: run e2e test on centos only

it's a regression introduced by the restrcuture of the test suites,
let's pin the test to CentOS8.

See-also: https://tracker.ceph.com/issues/49638
Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: add cmp_xatt support

fix Exceptional future ignored: std::runtime_error (op 'cmpxattr' not supported)

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

common: Use double instead of long double to improve performance

To convert namoseconds to seconds, the precision needs to be 10,
and the precision of double is 15, which is enough to use.
On aarch64, double division uses the div instruction, while long
double uses the gcc buildin _divtf3, which has poor performance.
Therefore, use double instead of long double for better performance.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>

doc/cephadm: improve "Upgrading Ceph" (main)

This PR makes a couple of minor improvements to the text under the
top-level section "Upgrading Ceph" in the "Upgrading Ceph" chapter of
the cephadm documentation.

This one, mercifully, contains only a couple of changes.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #42011 into master

* refs/pull/42011/head:
mds: just respawn mds daemon when osd op requests timeout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41988 into master

* refs/pull/41988/head:
logrotate: include cephfs-mirror daemon
cephfs-mirror: reopen logs on SIGHUP

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41917 into master

* refs/pull/41917/head:
mgr/mgr_util: switch using unshared cephfs connections whenever possible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>

Merge PR #41849 into master

* refs/pull/41849/head:
mds: try to flush the mdlog when requesting the rdlock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #42038 into master

* refs/pull/42038/head:
mds: fix compile warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42049 from zdover23/wip-doc-cephadm-serve-man-disable-auto-deploy-of-daemons

doc/cephadm: enrich "Disabling Automatic Deploy..."

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #41681 from TRYTOBE8TME/wip-rgw-dpp

src/rgw: DPP addition

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #42050 from rzarzynski/wip-crimson-alienstore-fix-attrs-conv

crimson/os: fix memory corruption in AlienStore::get_attrs().

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os: fix memory corruption in AlienStore::get_attrs().

`FuturizedStore` and `ObjectStore` use different memory layout for
conveying object attributes: map of `bufferlists` and map of `bptrs`
respectively. Unfortunately, `AlienStore` was trying to solve this
mismatch with just a `reinterpret_cast`.

Very likely this problem was the root cause behind the observed
crashes in `PGBackend::load_matadata` like the following one:

```
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: DEBUG 2021-06-15 09:24:19,199 [shard 0] osd - peering_event(id=412, detail=PeeringEvent(from=7 pgid=5.14 sent=49 requested=49 evt=epoch_sent: 49 epoch_requested: 49 MInfoRec from 7 info: 5.14( v 45'2 (0'0,45'2] local-lis/les=48/49 n=0 ec=44/44 lis/c=48/44 les/c/f=49/45/0 sis=48) pg_lease_ack(ruub 19.176788330s))): complete
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: Segmentation fault on shard 0.
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: Backtrace:
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  0# 0x000055C99757FFBF in /usr/bin/ceph-osd
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  1# FatalSignal::signaled(int, siginfo_t const*) in /usr/bin/ceph-osd
2021-06-15T09:25:07.511 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in /usr/bin/ceph-osd
2021-06-15T09:25:07.512 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  3# 0x00007F34BB632B20 in /lib64/libpthread.so.0
2021-06-15T09:25:07.512 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  4# 0x000055C99263D4D2 in /usr/bin/ceph-osd
2021-06-15T09:25:07.512 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  5# 0x000055C992740E47 in /usr/bin/ceph-osd
2021-06-15T09:25:07.512 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  6# seastar::continuation<seastar::internal::promise_base_with_type<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)84> > >::_future<crimson::errorated_future_marker<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > > > (seastar::future<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >&&)>, seastar::future<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >::then_wrapped_nrvo<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)84> > >::_future<crimson::errorated_future_marker<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > > >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)84> > >::_future<crimson::errorated_future_marker<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > > > (seastar::future<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >&&)> >(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)84> > >::_future<crimson::errorated_future_marker<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > > > (seastar::future<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >&&)>&&)::{lambda(seastar::internal::promise_base_with_type<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > >&&, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)84> > >::_future<crimson::errorated_future_marker<std::unique_ptr<PGBackend::loaded_object_md_t, std::default_delete<PGBackend::loaded_object_md_t> > > > (seastar::future<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >&&)>&, seastar::future_state<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >&&)#1}, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::buffer::v15_2_0::list, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::buffer::v15_2_0::list> > > >::run_and_dispose() in /usr/bin/ceph-osd
2021-06-15T09:25:07.512 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  7# 0x000055C99CFD195F in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  8# 0x000055C99CFD6EA0 in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]:  9# 0x000055C99D188F0B in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: 10# 0x000055C99CCE698A in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: 11# 0x000055C99CCF0AAE in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: 12# main in /usr/bin/ceph-osd
2021-06-15T09:25:07.513 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: 13# __libc_start_main in /lib64/libc.so.6
2021-06-15T09:25:07.514 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: 14# _start in /usr/bin/ceph-osd
2021-06-15T09:25:07.514 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:19 smithi100 conmon[54917]: Fault at location: 0x31dfff8000
2021-06-15T09:25:07.514 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jun 15 09:24:20 smithi100 podman[55356]: 2021-06-15 09:24:20.230341885 +0000 UTC m=+0.072958807 container died a3ea2a1d0a176286b93b8f5b94458982b9038e70d09128fb55f53b92976f0c42 (image=quay.ceph.io/ceph-ci/ceph@sha256:13ae953e3f83ee011d784d6eb9126fdc692f5bb688fe7d918be61ca7a7282b3c, name=ceph-43579b90-cdba-11eb-8c13-001a4aab830c-osd.3)
```

The fix deals with the issue by wrapping the `bptrs` in `bufferlists`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #41989 from zdover23/wip-doc-cephadm-serve-man-deploy-of-daemons-2021-06-24

doc/cephadm: enrich "deployment of daemons"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

doc/cephadm: enrich "Disabling Automatic Deploy..."

This PR rewrites and reformats the section "Disabling Automatic
Deployment of Daemons" in the "Service Management" chapter of the
cephadm guide.

I've rewritten some sentences, removed some "please"s, and added
some section titles so that the content in this is better
signposted.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw/sts: fix for copy object operation using sts
temporary credentials

Fixes: https://tracker.ceph.com/issues/47809
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rbd-nbd: support notrim option with map command

currently --notrim option works for rbd kernel mounter, but fails with rbd-nbd

$ rbd device --options notrim map rbd-pool/image0
/dev/rbd0
$ rbd device list
id  pool      namespace  image   snap  device
0   rbd-pool             image0  -     /dev/rbd0

$ rbd device --device-type nbd --options try-netlink,notrim map rbd-pool/image0
rbd-nbd: unknown args: --notrim
rbd: rbd-nbd failed with error: /data/ceph/build/bin/rbd-nbd: exit status: 1

With this changes:
$ rbd device --device-type nbd --options try-netlink,notrim map rbd-pool/image0
/dev/nbd0
$ rbd-nbd list-mapped
id    pool      namespace  image   snap  device
6945  rbd-pool             image0  -     /dev/nbd0
$ ps -eo "cmd" |grep [r]bd-nbd
/data/ceph/build/bin/rbd-nbd map rbd-pool/image0 --try-netlink --notrim

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

Merge pull request #41998 from kevinzs2048/arm64-rwl-cache-optional

ceph.spec.in, debian/rules: enable rbd-rwl-cache by default only on x86_64

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42021 from tchaikov/wip-rpm-memory-constraint

ceph.spec.in: increase memory per core to 3000MB on SUSE distros

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

pybind/ceph_argparse: do not set self.typeargs in ctor of CephArgtype

self.typeargs is never referenced, so let's just drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/uuid: define operator>() for uuid_d

it can be used by Option::validate() if
operator>(const variant<>&, const variant<>&) requires this.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/msg_types: add operator>() for entity_addrvec_t

it can be used by Option::validate() if
operator>(const variant<>&, const variant<>&) requires this.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: s/boost::lexical_cast/std::stoull/

we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: convert a millisecs opt to a chrono::milliseconds when paring it

Option always parses a new string value and convert it to a value_t
before validating it. and value_t is an alias of boost::variant<...>.
and we use "new_value < min" to tell if the new_value is out of the
bound or not, where both "new_value" and "min" are instances of
value_t. so it is critcal that these two values contain the same type
of value, otherwise boost::variant::operator< would

> Returns:
> If which() == rhs.which() then: content_this < content_rhs, where content_this is the content of *this and content_rhs is the content of rhs. Otherwise: which() < rhs.which().

where which() indicates which type of value is contained in the value_t.

before this change, instead of converting a new value of milliseconds to
std::chrono::milliseconds, we convert it to an uint64_t, whose index in
the value_t is 2, while the milliseconds value's index is 9, so
"new_value < min" evaluates to true even if "new_value" is 100 and "min"
is 30.

after this change, the new value of a milliseconds option is converted
to std::chrono::milliseconds, so it is comparable with its min value and
max value.

a minimal test is added to reproduce this issue.

the change which added the support of millisec to option was
29690a338ba4482d187e6036903e138437ae3bb4 which is not included by any
LTS branches, so no need to backport this fix.

Fixes: https://tracker.ceph.com/issues/51375
Signed-off-by: Kefu Chai <kchai@redhat.com>

libcephsqlite: register atexit handler for cleanup

We need to tear down the ceph vfs when sqlite3 (or other binaries) call
exit(). Otherwise global state gets destructed which can cause library
threads to segfault or raise exceptions.

Also pull vfs struct out of appdata. We need to be able to detect double
calls to the atexit handler which, sadly, can happen.

Fixes: https://tracker.ceph.com/issues/50503
Fixes: https://tracker.ceph.com/issues/51372
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

libcephsqlite: shutdown RADOS in destructor

Fixes: https://tracker.ceph.com/issues/50503
Fixes: https://tracker.ceph.com/issues/51372
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41574 into master

* refs/pull/41574/head:
qa/tasks/vstart_runner: add LocalCluster.run
qa/tasks/cephfs/test_nfs: fiddle with sudo
mgr/nfs/export: some cleanup, minor refactoring
mgr/nfs/cluster: remove unused @cluster_setter
nfs/mgr: fix help message case
doc/cephfs/fs-nfs-export: add note about export update behavior
mgr/nfs: move user create/delete into helper
mgr/nfs: refactor _delete_user helper
mgr/nfs: refactor create_export_from_dict() helper
mgr/nfs: keep 'nfs export get' around for backward-compat
mgr/nfs: rename method
qa/tasks/cephfs/test_nfs: test new export via apply
doc/cephfs/fs-nfs-export: be consistent with cluster_id and _ vs -
mgr/nfs: addr -> client_addr for 'nfs export create ...'
mgr/nfs: fix tests
mgr/nfs: 'nfs export get' -> 'nfs export info'
mgr/nfs: binding -> pseudo_path
mgr/nfs: more revisions based on review
mgr/nfs: adjust NFSExceptoin errno arg
doc/cephfs: update 'nfs export {get,apply}' docs
mgr/nfs: merge FSExport back into ExportMgr
doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports
mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'
mgr/nfs: test export creation and list
mgr/nfs: test export_update (+ fixes)
mgr/nfs: test Export.validate(); several fixes
mgr/nfs: test that export <-> block+dict conversions go both ways
mgr/nfs: clean up test a bit
mgr/nfs/export: fix export validation
mgr/nfs/export: fix tests
mgr/nfs: handle option addr/client block in create_export()
mgr/nfs: allow multiple addrs for new exports
mgr/nfs: fix/finish rgw export
mgr/nfs/module: clusterid -> cluster_id
mgr/nfs/export: fix export_update_1 to type check
mgr/nfs/cluster: fix type error
mgr/nfs/export: wrap long lines
mgr/nfs: ExportMgr._delete_export only works for cephfs for now
mgr/nfs: Remove pool_ns from NFSCluster
mgr/nfs: Remove ExportMgr.rados_namespace
mgr/nfs: flake8
mgr/nfs: Add type checking
mgr/nfs: Add __eq__ method to Export
mgr/nfs: Add some compatibility to mgr/dashboard
mgr/nfs: Fix whitespace handling
mgr/nfs: Copy unit tests from mgr/dashboard
mgr/nfs: partially implement rgw export support
mgr/nfs: abstract FSAL; add RGWFSAL
mgr/nfs: refactor to merge 'update' and 'import' code
mgr/nfs: add 'nfs export import' command
mgr/nfs: refactor 'nfs export update' and export validation
mgr/nfs: fix _fetch_export to distinguish between clusters
mgr/nfs: move export ganesha conf translation into caller
mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'
mgr/nfs: add --addr to 'nfs export create'
mgr/nfs: add --squash to 'nfs export create'
mgr/nfs/export_utils: include false but non-None items in config
vstart.sh: enable nfs module
mgr/cephadm: nfs: drop attr_expiration_time from top-level config
mgr/cephadm: remove Dir_Chunk = 0

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #41937 from liewegas/mgr-crash

mgr: generate crash dumps for Python exceptions in mgr modules

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41946 from liewegas/fix-51294

mgr/devicehealth: fix _get_device_metrics ValueError

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks/vstart_runner: add LocalCluster.run

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/cephfs/test_nfs: fiddle with sudo

- no sudo for 'ceph' commands
- explicit sudo for _sys_cmd (things like 'rados' don't need sudo!)

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/nfs/export: some cleanup, minor refactoring

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/nfs/cluster: remove unused @cluster_setter

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #41977 from rzarzynski/wip-crimson-common-print-more-on-crash

crimson/common: dump more on faults

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: fix compile warning

    ../src/mds/Server.cc: In member function ‘void Server::handle_set_vxattr(MDRequestRef&, CInode*)’:
    ../src/mds/Server.cc:5703:18: warning: unused variable ‘realm’ [-Wunused-variable]
           SnapRealm *realm = cur->find_snaprealm();
                      ^~~~~

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

nfs/mgr: fix help message case

Signed-off-by: Sage Weil <sage@newdream.net>

doc/cephfs/fs-nfs-export: add note about export update behavior

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/nfs: move user create/delete into helper

- Do user create or delete via a helper
- Defer until after we have validated the Export (on create or update)
- Support updates to user_id, which is needed to keep the naming consistent
and to also support changing the bucket, since the user_id is derived
from that.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #41838 from p-se/grafana-clean-up

monitoring: Clean up Grafana dashboards

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: jan--f <NOT@FOUND>
Reviewed-by: p-se <NOT@FOUND>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

qa/suites/rados/mgr: whitelist module crash during selftest

One of the selftests triggers an exception from serve().

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #41721 from aaryanporwal/telemetry-ident-fix

mgr/dashboard: telemetry activate: show ident fields when checked

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41991 from dang/wip-dang-bucket-delete

RGW - Bucket Remove Op: Pass in user

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41993 from ronen-fr/wip-ronenf-50346

osd/scrub: replace a ceph_assert() with a test

Reviewed-by: Neha Ojha <nojha@redhat.com>

cephadm: Fix normalize_image_digest for local registries

Cause they typically don't have dots in it.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42024 from rzarzynski/wip-crimson-load_obc_nocpy

crimson/osd: don't extra copy hobject in PG::load_head_obc().

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: don't extra copy hobject in PG::load_head_obc().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>