mgr/progress: Skip pg_summary update if _events dict is empty.

The get call of pg_stats and pg_ready are expensive and are not necessary in this case. This avoids blocking other modules like the prometheus module

Fixes: https://tracker.ceph.com/issues/46416
Signed-off-by: Manuel Lausch <manuel.lausch@1und1.de>

Merge pull request #35501 from SchoolGuy/streamline-bootstrap-usage

mgr/dashboard: Introduce SASS modules

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34960 from majianpeng/vstart-spdk

vstart: specify the NVME PCI BUS ID when enable spdk.

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: specify the NVME PCI BUS ID when enable spdk.

Currently we use vendor::device to specify the nvme name when using
spdk. But it have some problems:
a)it can't use different vendor or different device with the same
device.
b)if machine has same vendor&device, we don't choose specific
device.

So we diretly specify the PCI BUS ID for NVME device.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge PR #35725 into master

* refs/pull/35725/head:
client: Fix setxattr with 0 size specified

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #35861 from dillaman/wip-write-zeroes-tweaks

librbd: allow thick-zeroing of an image extent

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #35743 into master

* refs/pull/35743/head:
qa/tasks/test_nfs: Add test for cluster info
mgr/volumes/nfs: Add cluster show info command

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35834 from yuvalif/add-amqp-topic-doc

rgw: add a doc hint that the topic-name is used as AMQP routing-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35923 from votdev/fix_various_issues

mgr/dashboard: Fix bugs in a unit test and i18n translation

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35886 from yuvalif/wip-yuval-fix-46316

rgw: make RGWToken string ctor explicit

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35707 from Devp00l/wip-45650

mgr/dashboard: Fixes ICU selection regression

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35764 from rzarzynski/wip-bl-cache-in-release

common/bl: document and slightly optimize ptr::release().

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Introduces SASS modules

Fixes: https://tracker.ceph.com/issues/45934
Signed-off-by: Enno Gotthold <egotthold@suse.de>

common/bl: document and slightly optimize ptr::release().

Before the patch GCC was generating:
```
  10f9910:   mov    (%rdi),%rax
  10f9913:   test   %rax,%rax
  10f9916:   je     10f9951 <ceph::buffer::v15_2_0::ptr::release()+0x41>
  10f9918:   mov    0x2c(%rax),%eax
  10f991b:   cmp    $0x1,%eax
  10f991e:   jne    10f9940 <ceph::buffer::v15_2_0::ptr::release()+0x30>
  10f9920:   mov    (%rdi),%rax
  10f9923:   movq   $0x0,(%rdi)
  10f992a:   test   %rax,%rax
  10f992d:   je     10f9958 <ceph::buffer::v15_2_0::ptr::release()+0x48>
  10f992f:   mov    (%rax),%rdx
  10f9932:   mov    %rax,%rdi
  10f9935:   mov    0x8(%rdx),%rdx
  10f9939:   jmpq   *%rdx
  10f993b:   nopl   0x0(%rax,%rax,1)
  10f9940:   mov    (%rdi),%rax
  10f9943:   lock subl $0x1,0x2c(%rax)
  10f9948:   je     10f9920 <ceph::buffer::v15_2_0::ptr::release()+0x10>
  10f994a:   movq   $0x0,(%rdi)
  10f9951:   retq
  10f9952:   nopw   0x0(%rax,%rax,1)
  10f9958:   retq
  10f9959:   nop
  10f995a:   nopw   0x0(%rax,%rax,1)
```

after:
```
  11209b0:   mov    (%rdi),%rax
  11209b3:   movq   $0x0,(%rdi)
  11209ba:   test   %rax,%rax
  11209bd:   je     11209e6 <ceph::buffer::v15_2_0::ptr::release()+0x36>
  11209bf:   mov    0x2c(%rax),%edx
  11209c2:   lea    0x2c(%rax),%rcx
  11209c6:   cmp    $0x1,%edx
  11209c9:   jne    11209e0 <ceph::buffer::v15_2_0::ptr::release()+0x30>
  11209cb:   mov    (%rax),%rdx
  11209ce:   mov    %rax,%rdi
  11209d1:   mov    0x8(%rdx),%rdx
  11209d5:   jmpq   *%rdx
  11209d7:   nopw   0x0(%rax,%rax,1)
  11209de:
  11209e0:   lock subl $0x1,(%rcx)
  11209e4:   je     11209cb <ceph::buffer::v15_2_0::ptr::release()+0x1b>
  11209e6:   retq
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: add a doc hint that the topic-name is used as AMQP routing-key

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

qa/tasks/test_nfs: Add test for cluster info

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Add cluster show info command

Fixes: https://tracker.ceph.com/issues/45743
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #35940 from tchaikov/wip-ceph-dencoder-struct-ver

tools/ceph-dencoder: add get_struct_v,get_struct_compat commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35914 from tchaikov/wip-crimson-clang

crimson: clang related cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #35933 from tchaikov/wip-interval-set

include/interval_set: use template as the 2nd template parameter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35496 into master

* refs/pull/35496/head:
client: introduce timeout for client shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35942 from tchaikov/wip-build-rtd

admin: move python-common deps out of doc-requirements.txt

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35354 from ishanrai05/rfs

mgr/dashboard: use relative font sizes

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

tools/ceph-dencoder: add get_struct_v,get_struct_compat commands

for understanding the current encoding version and the lower bound of
supported decoders.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35913 from ideepika/wip-container-tag-change

docker/podman: tag crimson flavor container as 'sha-1-flavor'

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35920 from uweigand/entity-addrvec-fix

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35890 from matthewoliver/cephadm_ipv6_list_networks

cephadm: Make list_networks ipv6 enabled

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35546 from agayev/zoned-fm

os/bluestore: Introduce zoned freelist manager.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

admin: move python-common deps out of doc-requirements.txt

to address the build failure of

python -m pip install --exists-action=w --no-cache-dir -r admin/doc-requirements.txt
ERROR: ../src/python-common is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

Signed-off-by: Kefu Chai <kchai@redhat.com>

docker/podman: tag crimson flavor container as 'sha-1-flavor'

see also: ceph/ceph-container#1704

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/dashboard: Fix bugs in a unit test and i18n translation

* SKIP: Unknown argument type: 'BinaryExpression' NodeObject { ... text: 'You can activate the module on the Telemetry configuration '
* WARNING: /ceph/src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-list/rgw-bucket-list.component.spec.ts:39:32 - get is deprecated: from v9.0.0 use TestBed.inject

Fixes: https://tracker.ceph.com/issues/46371
Signed-off-by: Volker Theile <vtheile@suse.com>

client: Fix setxattr with 0 size specified

When xattrs are set with 0 size, the value should be set to an empty string.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

Merge pull request #35936 from tchaikov/wip-cmake-seastar-cxxflags

cmake: append to Seastar_CXX_FLAGS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35812 from tchaikov/wip-more-log-for-45717

mgr/dashboard: print osd log as well

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

crimson/osd: assign to bufferlist not to bufferlist*

this change addresses a regression introduced by a23a1929f5

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: append to Seastar_CXX_FLAGS

instead of overwriting Seastar_CXX_FLAGS, append to it. this change
allows user to set Seastar_CXX_FLAGS in the command line.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35610 from pcuzner/wip-grafana-container

monitoring: add grafana container build file

rbd: switch thick-provision image creation to use new write_zeroes API

Also fix some of the concurrency and chunk size issues when handling
fancy striped images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add flag to force thick-provisioning during write_zeroes API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35935 from tchaikov/wip-crimson-no-more-claim

crimson/osd: use operator=(...) instead of claim()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: use operator=(...) instead of claim()

to silence warnings like

objclass.cc:300:26: warning: 'void ceph::buffer::v15_2_0::list::claim(ceph::buffer::v15_2_0::list&)' is deprecated: in favor of operator=(list&&) [-Wdeprecated-declarations]
  300 |   outbl->claim(op.outdata);
      |                          ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35921 from uweigand/test_cdc-endianfix

unittest_cdc: fix test on big-endian hosts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test/common: link unittest_interval_set against GTest::Main

its main() is offered by GTet::Main library

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/interval_set: use template as the 2nd template parameter

to avoid repeating the unit for the interval in the internal map's
parameter.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35932 from zdover23/wip-doc-cephadm-suites-typo

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

This commit changes the sentence "Choose the method that best suites
your needs." to "Choose the method that best suits your needs."

Fixes: https://tracker.ceph.com/issues/46359
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35275 from AmnonHanuhov/wip-objclass_impl

crimson/osd: Add support for cls_log

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35871 from tchaikov/wip-45717

mon/PGMap: do not consider changing pg stuck

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35864 from tchaikov/wip-osdc-claim

osdc: use operator=(...) instead of claim()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35863 from wuxingyi/removeunused

osd: Remove unused function declearation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35818 from majianpeng/osd-replicatedbackend-redundant-parameter

osd/ReplicatedBackend: remove redundant parameter.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35817 from tchaikov/wip-drop-py2

qa,test,pybind: drop py2 support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

pybind/mgr/telegraf: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/status: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/selftest: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/restful: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/insights: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/influx: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/devicehealth: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/cephadm: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph_objectstore_tool: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/barbican: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/workunit: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/s3tests: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/rbd: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/radosgw_admin: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/osd_failsafe_enospc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/{ceph,ceph_manager}: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/pybind: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/dashboard: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: drop py2 support

the python bindings are compiled using python3-cython, and packaged as
python3-{rados,rbd,cephfs,rgw}, so there is no need and is wrong to
claim that they are python2 compatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/vstart_runner: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/standalone: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: s/assert/ceph_abort_msg/ when hitting unreachable code

clang++ complains with
```
crimson/os/seastore/root_block.h:100:3: warning: control reaches end of non-void function [-Wreturn-type]
  }
  ^
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: silence -Wunused-lambda-capture warnings

they are pointed out by clang++.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: make FixedKVNodeLayout::set_val() public

otherwise clang++ fails with
```
mson/os/seastore/lba_manager/btree/lba_btree_node_impl.cc:183:10: error: 'set_val' is a private member of 'crimson::common::FixedKVNodeLayout<255, unsigned long, ceph_le<unsigned long long>,
crimson::os::seastore::paddr_t, crimson::os::seastore::paddr_le_t, true>::iter_t<false>'
      i->set_val(updated);
         ^
/home/jenkins-build/build/workspace/ceph-perf-crimson/ceph-master/src/crimson/common/fixed_kv_node_layout.h:148:10: note: declared private here
    void set_val(V val) const {
         ^
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35855 from dillaman/wip-45986

librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs

Reviewed-by: Mykola Golub <mgolub@suse.com>

unittest_cdc: fix test on big-endian hosts

The specific_result test hardcodes a particular output.  However,
the input buffer used by the test is currently initialized to a
different byte string depending host the host byte order, which
makes the expected result not match on big-endian hosts.

Fixed by always initializing the input buffer to the same string.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

When decoding an entity_addrvec_t with marker 1, we just have
a single (non-legacy) entity_addr_t.  This should be decoded
exactly the same as done by entity_addr_t::decode, but it
currently is not.  Specifically, the sa_family member of
the sockaddr is not converted from the on-wire little-endian
format to host byte order (as done by entity_addr_t::decode).

Fixed by using the same code as in entity_addr_t::decode.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

Merge pull request #35859 from tspmelo/wip-remove-ngx-bootstrap

mgr/dashboard: Remove ngx-bootstrap

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/dev: Update zoned-storage.rst to indicate that OSDs can now be written to after restart on zoned devices.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

os/bluestore: Introduce zoned freelist manager.

Introduced ZonedFreelistManager, which tracks the write pointer and the number
of dead bytes per zone.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

Merge pull request #35875 from Devp00l/wip-46303

mgr/dashboard: Fixes regression in device selection modal

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35912 from tchaikov/wip-crimson-osd-flush-pg-stats

crimson/osd: implement flush_pg_stats command

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35374 from jschmid1/activate_osdspec_affinity

mgr/cephadm: re-enable osdspec affinity passing

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson/osd: avoid `-Wunused-but-set-variable` warning

in release build, `found` is not used at all. but GCC complains with a
`-Wunused-but-set-variable` warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35704 from Devp00l/wip-46135

mgr/dashboard: Fixes typeahead regression in the silence matcher

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

crimson/osd: update the osd stats with hb peers

just to align with classic OSD

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implement flush_pg_stats command

this command is used by teuthology based tests for updating the mgr with
the lastes pg stats on OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/{mgr,osd}: add WithStats::update_stats()

because we need to implement a tell command which forces osd to
send the latest pg stats to mgr, and the command returns the user with
the sequence id of the report, and `mgr::Client::report()` does not
return a future, so we have to update the seq id before sending
the report. the solution is to update the seq id in a separated
method, so in this change:

* add `const` to `WithStats::get_stats()
* add a dedicated method of `WithStats::update_stats()` to update
  the stats to be collected by mgr::Client.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35887 from joscollin/wip-update-kyr

.githubmap: add Kyr

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #35845 from tchaikov/wip-rpm-crimson

rpm: package crimson-osd in ceph-osd

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #35867 from jwilk-forks/spelling

doc/dev/osd_internals: fix typos

Reviewed-by: Neha Ojha <nojha@redhat.com>