neorados: build new RADOS client using legacy librados::Rados

Client libraries like librbd cannot force the adaptation to neorados
and it would not be ideal to have two effective RADOS cluster
connections for librados and neorados.

The new neorados::RADOS::make_with_librados helper method will
allow neorados to re-use the existing thread pool, MGR, MON, and
objecter already created for librados::Rados.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

neorados: use a lightweight proxy object to abstract RADOS backend

Both neorados and (legacy) librados create MON, MGR, and OSD
(objecter) clients. Let the neorados frontend API directly access
these clients through a generic class that can be swapped out.
This will allow the neorados API to re-use a (legacy) librados
RadosClient in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

neorados: fixed valgrind memory leaks and errors

Keep the CephContext wrapped in an intrusive pointer to keep the
reference counting functional. Ensure all the messenger clients
(mon, mgr, and objecter) are shut down prior to destruction.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35913 from ideepika/wip-container-tag-change

docker/podman: tag crimson flavor container as 'sha-1-flavor'

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35920 from uweigand/entity-addrvec-fix

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35890 from matthewoliver/cephadm_ipv6_list_networks

cephadm: Make list_networks ipv6 enabled

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35546 from agayev/zoned-fm

os/bluestore: Introduce zoned freelist manager.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

docker/podman: tag crimson flavor container as 'sha-1-flavor'

see also: ceph/ceph-container#1704

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #35936 from tchaikov/wip-cmake-seastar-cxxflags

cmake: append to Seastar_CXX_FLAGS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35812 from tchaikov/wip-more-log-for-45717

mgr/dashboard: print osd log as well

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

crimson/osd: assign to bufferlist not to bufferlist*

this change addresses a regression introduced by a23a1929f5

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: append to Seastar_CXX_FLAGS

instead of overwriting Seastar_CXX_FLAGS, append to it. this change
allows user to set Seastar_CXX_FLAGS in the command line.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35610 from pcuzner/wip-grafana-container

monitoring: add grafana container build file

Merge pull request #35935 from tchaikov/wip-crimson-no-more-claim

crimson/osd: use operator=(...) instead of claim()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: use operator=(...) instead of claim()

to silence warnings like

objclass.cc:300:26: warning: 'void ceph::buffer::v15_2_0::list::claim(ceph::buffer::v15_2_0::list&)' is deprecated: in favor of operator=(list&&) [-Wdeprecated-declarations]
  300 |   outbl->claim(op.outdata);
      |                          ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35921 from uweigand/test_cdc-endianfix

unittest_cdc: fix test on big-endian hosts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35932 from zdover23/wip-doc-cephadm-suites-typo

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

This commit changes the sentence "Choose the method that best suites
your needs." to "Choose the method that best suits your needs."

Fixes: https://tracker.ceph.com/issues/46359
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35275 from AmnonHanuhov/wip-objclass_impl

crimson/osd: Add support for cls_log

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35871 from tchaikov/wip-45717

mon/PGMap: do not consider changing pg stuck

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35864 from tchaikov/wip-osdc-claim

osdc: use operator=(...) instead of claim()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35863 from wuxingyi/removeunused

osd: Remove unused function declearation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35818 from majianpeng/osd-replicatedbackend-redundant-parameter

osd/ReplicatedBackend: remove redundant parameter.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35817 from tchaikov/wip-drop-py2

qa,test,pybind: drop py2 support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

pybind/mgr/telegraf: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/status: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/selftest: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/restful: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/insights: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/influx: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/devicehealth: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/cephadm: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph_objectstore_tool: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/barbican: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/workunit: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/s3tests: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/rbd: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/radosgw_admin: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/osd_failsafe_enospc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/{ceph,ceph_manager}: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/pybind: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/dashboard: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: drop py2 support

the python bindings are compiled using python3-cython, and packaged as
python3-{rados,rbd,cephfs,rgw}, so there is no need and is wrong to
claim that they are python2 compatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/vstart_runner: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/standalone: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35855 from dillaman/wip-45986

librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs

Reviewed-by: Mykola Golub <mgolub@suse.com>

unittest_cdc: fix test on big-endian hosts

The specific_result test hardcodes a particular output.  However,
the input buffer used by the test is currently initialized to a
different byte string depending host the host byte order, which
makes the expected result not match on big-endian hosts.

Fixed by always initializing the input buffer to the same string.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

When decoding an entity_addrvec_t with marker 1, we just have
a single (non-legacy) entity_addr_t.  This should be decoded
exactly the same as done by entity_addr_t::decode, but it
currently is not.  Specifically, the sa_family member of
the sockaddr is not converted from the on-wire little-endian
format to host byte order (as done by entity_addr_t::decode).

Fixed by using the same code as in entity_addr_t::decode.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

Merge pull request #35859 from tspmelo/wip-remove-ngx-bootstrap

mgr/dashboard: Remove ngx-bootstrap

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/dev: Update zoned-storage.rst to indicate that OSDs can now be written to after restart on zoned devices.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

os/bluestore: Introduce zoned freelist manager.

Introduced ZonedFreelistManager, which tracks the write pointer and the number
of dead bytes per zone.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

Merge pull request #35875 from Devp00l/wip-46303

mgr/dashboard: Fixes regression in device selection modal

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35912 from tchaikov/wip-crimson-osd-flush-pg-stats

crimson/osd: implement flush_pg_stats command

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35374 from jschmid1/activate_osdspec_affinity

mgr/cephadm: re-enable osdspec affinity passing

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson/osd: avoid `-Wunused-but-set-variable` warning

in release build, `found` is not used at all. but GCC complains with a
`-Wunused-but-set-variable` warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35704 from Devp00l/wip-46135

mgr/dashboard: Fixes typeahead regression in the silence matcher

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

crimson/osd: update the osd stats with hb peers

just to align with classic OSD

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implement flush_pg_stats command

this command is used by teuthology based tests for updating the mgr with
the lastes pg stats on OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/{mgr,osd}: add WithStats::update_stats()

because we need to implement a tell command which forces osd to
send the latest pg stats to mgr, and the command returns the user with
the sequence id of the report, and `mgr::Client::report()` does not
return a future, so we have to update the seq id before sending
the report. the solution is to update the seq id in a separated
method, so in this change:

* add `const` to `WithStats::get_stats()
* add a dedicated method of `WithStats::update_stats()` to update
  the stats to be collected by mgr::Client.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35887 from joscollin/wip-update-kyr

.githubmap: add Kyr

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #35845 from tchaikov/wip-rpm-crimson

rpm: package crimson-osd in ceph-osd

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #35867 from jwilk-forks/spelling

doc/dev/osd_internals: fix typos

Reviewed-by: Neha Ojha <nojha@redhat.com>

cephadm: Make list_networks ipv6 enabled

Currently the list_network command and methods in cephadm only run and
parse ipv4 output from `ip route`.

This patch extends the list_network command and internal methods to be
ipv6 enabled. It now also checks `ip -6 route` and `ip -6 addr` to
gather gather all networks from both protocol families.

Signed-off-by: Matthew Oliver <moliver@suse.com>

Merge pull request #35894 from tchaikov/wip-crimson-os

crimson/os: pin the last cpu core for the alien worker threads

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #35745 from votdev/issue_46146_delete_host

mgr/dashboard: Host delete action should be disabled if not managed by Orchestrator

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35642 from tspmelo/wip-fix-flicker

mgr/dashboard: Re-enable OSD's table autoReload

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34889 from votdev/issue_43120_tabs

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

crimson/os/alienstore: assert on failure

before we have a failure recovery, better off failing fast.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: pin the last cpu core for the alien worker threads

before this change, we assume that we have at least current_shared + 10
cores. but that's not always true. so in this change, the last core
is used for performing the alien tasks scheduled by reactor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: stay on active tab (accessible via URL)

This PR refactors the /monitoring page by introducing a new component which is responsible for rendering the tabs. All tabs have a dedicated route/URL now, no fragments are used anymore, thus they will act like the iSCSI and RBD pages.

Fixes: https://tracker.ceph.com/issues/45375
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #35816 from idryomov/wip-msgr21-fix-reuse

msg/async/ProtocolV2: take care of features when replacing the socket

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35854 from tspmelo/wip-fix-nfs-loading

mgr/dashboard: Fix NFS Edit page loading problem

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #35823 from bk201/wip-46245

mgr/cephadm: reconfig ssh after setting or clearing ssh_config

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35765 from votdev/issue_46145_reset_action_button

mgr/dashboard: Action button is not reset after switching tabs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35803 from s0nea/wip-dashboard-remove-formatted-attr

mgr/dashboard: remove cdCopy2ClipboardButton `formatted` attribute

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35828 from votdev/issue_46249_rm_tab

mgr/dashboard: Remove useless tab in monitoring/alerts datatable details

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35564 from rhcs-dashboard/wip-45957-fixUiApiEndpoints

mgr/dashboard: fix ui api endpoints

.githubmap: add Kyr

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #35639 from joscollin/wip-fix-sh-as-in-teuthology

qa/tasks: make sh() in vstart_runner.py identical with teuthology.orchestra.remote.sh

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #35037 from CongMinYin/qa-test-for-immutale-object-cache

qa: add tests for immutable object cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd/image: fix immutable object cache read failed bug when daemon down

Fix bug: read child image failed when enabling immutable object cache and daemon down

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

qa: add tests for immutable object cache

based on qemu task, use immutable_object_cache task to test parent cache
based on rbd_fio task, use immutable_object_cache task to test parent cache

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
Signed-off-by: Feng Hualong <hualong.feng@intel.com>

Merge pull request #35699 from fflorens/ff/work/bug_46139

osd/OSDCap: rbd profile permits use of "rbd_info"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35596 from Songweibin/wip-fix-rbd-rm

librbd: fix potential image rm failure due to parent missing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Fixes regression in device selection modal

The 'ExpressionChangedAfterItHasBeenCheckedError' has recently
shown up in device selection modal in OSD creation form. It looks
like it's a regression due to the modal switch (PR #35370). Which was
merged yesterday and it had worked on Monday.

Fixes: https://tracker.ceph.com/issues/46303
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #35856 from tchaikov/wip-46267

test/unittest_lockdep: skip lockdep test if CEPH_DEBUG_MUTEX is not d…

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35751 from smithfarm/wip-radosgw-admin-orphans

doc: PendingReleaseNotes: fix deprecated radosgw-admin commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: Remove ngx-bootstrap

Fixes: https://tracker.ceph.com/issues/46281
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35782 from tspmelo/wip-ng-datepicker

mgr/dashboard: Use ng-bootstrap for Datepicker

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35869 from smithfarm/wip-rn-15_2_4

doc/releases/octopus: add 15.2.4 release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #35805 from ricardoasmarques/ceph-orch-status-format

cephadm: Add "--format" option to "ceph orch status"

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35811 from tchaikov/wip-46054

kv/RocksDBStore: always release column family handles

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #35723 from tspmelo/wip-fix-action-tooltip

mgr/dashboard: Fix title of disabled table actions

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>