Merge pull request #35949 from zdover23/wip-doc-dev-guide-essentials-2020-07-07

doc/devel: repair sentences in "Essentials" page

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35966 from ivancich/wip-fix-orphan-list-quoting

rgw: rgw-orphan-list -- fix interaction, quoting, and percentage calc

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #35910 from galsalomon66/add_s3select_filter

adding filter for s3select

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35286 from theanalyst/rgw/cleanup/override-umdhandler

rgw: user: RGWUserMetadataHandler mark remove method override

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35911 from majianpeng/bluestore-check-written_slow-bytes

os/bluestore/bluefs: only bytes_written_slow > 0, we update perfcounter.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #35908 from bk201/wip-46329

mgr/cephadm: fix Dashboard ganesha setting duplication error

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35907 from agayev/fix-zoned-device-detection

os/bluestore: Fix handling the return value of zbc_device_is_zoned() call.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35825 from changchengx/watch_stress

test/stress_watch: remove unused variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35943 from tchaikov/wip-mgr-balancer

pybind/mgr/balancer: use "==" and "!=" for comparing str

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #35948 from tspmelo/wip-clean-jest

mgr/dashboard: Clean unit tests configuration and output

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35922 from tspmelo/wip-clean-lint

mgr/dashboard: Clean the output of lint and fix scripts

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35338 from myoungwon/wip-doc-manifet-snap

doc/dev/osd_internals/manifest.rst: add information about clone snap refcounting

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #35874 from tspmelo/wip-fix-pwd-notification

mgr/dashboard: Hide password notification when expiration date is far

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #35501 from SchoolGuy/streamline-bootstrap-usage

mgr/dashboard: Introduce SASS modules

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34960 from majianpeng/vstart-spdk

vstart: specify the NVME PCI BUS ID when enable spdk.

Reviewed-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: use "==" and "!=" for comparing str

we cannot assume that two values with the same value share the same
identity in Python.

also silences warnings like:

balancer/module.py:473: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if pool_ids is '':

Fixes: https://tracker.ceph.com/issues/46406
Signed-off-by: Kefu Chai <kchai@redhat.com>

vstart: specify the NVME PCI BUS ID when enable spdk.

Currently we use vendor::device to specify the nvme name when using
spdk. But it have some problems:
a)it can't use different vendor or different device with the same
device.
b)if machine has same vendor&device, we don't choose specific
device.

So we diretly specify the PCI BUS ID for NVME device.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge PR #35725 into master

* refs/pull/35725/head:
client: Fix setxattr with 0 size specified

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

rgw: rgw-orphan-list -- fix interaction, quoting, and percentage calc

The interactive mode wasn't working due to prompts going to stdout
instead of stderr. If a space should appear in temporary file, it will
generate a shell error, so quoting was added. Furthermore if there are
no objects found in a pool, a divide by zero error will be
generated. This commit addresses these issues.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: Clean the output of lint and fix scripts

Increase deprecation tslint rule from warning to error and
remove no-use-before-declare.

Fixes: https://tracker.ceph.com/issues/46339
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35861 from dillaman/wip-write-zeroes-tweaks

librbd: allow thick-zeroing of an image extent

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #35743 into master

* refs/pull/35743/head:
qa/tasks/test_nfs: Add test for cluster info
mgr/volumes/nfs: Add cluster show info command

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35834 from yuvalif/add-amqp-topic-doc

rgw: add a doc hint that the topic-name is used as AMQP routing-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35923 from votdev/fix_various_issues

mgr/dashboard: Fix bugs in a unit test and i18n translation

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35886 from yuvalif/wip-yuval-fix-46316

rgw: make RGWToken string ctor explicit

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35707 from Devp00l/wip-45650

mgr/dashboard: Fixes ICU selection regression

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35764 from rzarzynski/wip-bl-cache-in-release

common/bl: document and slightly optimize ptr::release().

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Introduces SASS modules

Fixes: https://tracker.ceph.com/issues/45934
Signed-off-by: Enno Gotthold <egotthold@suse.de>

common/bl: document and slightly optimize ptr::release().

Before the patch GCC was generating:
```
  10f9910:   mov    (%rdi),%rax
  10f9913:   test   %rax,%rax
  10f9916:   je     10f9951 <ceph::buffer::v15_2_0::ptr::release()+0x41>
  10f9918:   mov    0x2c(%rax),%eax
  10f991b:   cmp    $0x1,%eax
  10f991e:   jne    10f9940 <ceph::buffer::v15_2_0::ptr::release()+0x30>
  10f9920:   mov    (%rdi),%rax
  10f9923:   movq   $0x0,(%rdi)
  10f992a:   test   %rax,%rax
  10f992d:   je     10f9958 <ceph::buffer::v15_2_0::ptr::release()+0x48>
  10f992f:   mov    (%rax),%rdx
  10f9932:   mov    %rax,%rdi
  10f9935:   mov    0x8(%rdx),%rdx
  10f9939:   jmpq   *%rdx
  10f993b:   nopl   0x0(%rax,%rax,1)
  10f9940:   mov    (%rdi),%rax
  10f9943:   lock subl $0x1,0x2c(%rax)
  10f9948:   je     10f9920 <ceph::buffer::v15_2_0::ptr::release()+0x10>
  10f994a:   movq   $0x0,(%rdi)
  10f9951:   retq
  10f9952:   nopw   0x0(%rax,%rax,1)
  10f9958:   retq
  10f9959:   nop
  10f995a:   nopw   0x0(%rax,%rax,1)
```

after:
```
  11209b0:   mov    (%rdi),%rax
  11209b3:   movq   $0x0,(%rdi)
  11209ba:   test   %rax,%rax
  11209bd:   je     11209e6 <ceph::buffer::v15_2_0::ptr::release()+0x36>
  11209bf:   mov    0x2c(%rax),%edx
  11209c2:   lea    0x2c(%rax),%rcx
  11209c6:   cmp    $0x1,%edx
  11209c9:   jne    11209e0 <ceph::buffer::v15_2_0::ptr::release()+0x30>
  11209cb:   mov    (%rax),%rdx
  11209ce:   mov    %rax,%rdi
  11209d1:   mov    0x8(%rdx),%rdx
  11209d5:   jmpq   *%rdx
  11209d7:   nopw   0x0(%rax,%rax,1)
  11209de:
  11209e0:   lock subl $0x1,(%rcx)
  11209e4:   je     11209cb <ceph::buffer::v15_2_0::ptr::release()+0x1b>
  11209e6:   retq
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: add a doc hint that the topic-name is used as AMQP routing-key

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

qa/tasks/test_nfs: Add test for cluster info

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Add cluster show info command

Fixes: https://tracker.ceph.com/issues/45743
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #35940 from tchaikov/wip-ceph-dencoder-struct-ver

tools/ceph-dencoder: add get_struct_v,get_struct_compat commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35914 from tchaikov/wip-crimson-clang

crimson: clang related cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #35933 from tchaikov/wip-interval-set

include/interval_set: use template as the 2nd template parameter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35496 into master

* refs/pull/35496/head:
client: introduce timeout for client shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/devel: repair sentences in "Essentials" page

This commit fixes a few sentences in the "Essentials" page
that aren't as well-formed as they might have been. This commit
fixes the issue detailed in the tracker link, as well as a couple
of other improvements that I couldn't resist in the heat of the
moment.

Fixes: https://tracker.ceph.com/issues/46362
Signed-off-by: Zac Dover <zac.dover@gmail.com>

mgr/dashboard: Hide password notification when expiration date is far

Fixes: https://tracker.ceph.com/issues/46306
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Clean unit tests configuration and output

Remove outdated configurations and moment warning.
Update Jest related packages.

Fixes: https://tracker.ceph.com/issues/46341
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35942 from tchaikov/wip-build-rtd

admin: move python-common deps out of doc-requirements.txt

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35354 from ishanrai05/rfs

mgr/dashboard: use relative font sizes

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

tools/ceph-dencoder: add get_struct_v,get_struct_compat commands

for understanding the current encoding version and the lower bound of
supported decoders.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35913 from ideepika/wip-container-tag-change

docker/podman: tag crimson flavor container as 'sha-1-flavor'

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35920 from uweigand/entity-addrvec-fix

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35890 from matthewoliver/cephadm_ipv6_list_networks

cephadm: Make list_networks ipv6 enabled

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35546 from agayev/zoned-fm

os/bluestore: Introduce zoned freelist manager.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

admin: move python-common deps out of doc-requirements.txt

to address the build failure of

python -m pip install --exists-action=w --no-cache-dir -r admin/doc-requirements.txt
ERROR: ../src/python-common is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

Signed-off-by: Kefu Chai <kchai@redhat.com>

docker/podman: tag crimson flavor container as 'sha-1-flavor'

see also: ceph/ceph-container#1704

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/dashboard: Fix bugs in a unit test and i18n translation

* SKIP: Unknown argument type: 'BinaryExpression' NodeObject { ... text: 'You can activate the module on the Telemetry configuration '
* WARNING: /ceph/src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-list/rgw-bucket-list.component.spec.ts:39:32 - get is deprecated: from v9.0.0 use TestBed.inject

Fixes: https://tracker.ceph.com/issues/46371
Signed-off-by: Volker Theile <vtheile@suse.com>

client: Fix setxattr with 0 size specified

When xattrs are set with 0 size, the value should be set to an empty string.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

Merge pull request #35936 from tchaikov/wip-cmake-seastar-cxxflags

cmake: append to Seastar_CXX_FLAGS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35812 from tchaikov/wip-more-log-for-45717

mgr/dashboard: print osd log as well

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

crimson/osd: assign to bufferlist not to bufferlist*

this change addresses a regression introduced by a23a1929f5

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: append to Seastar_CXX_FLAGS

instead of overwriting Seastar_CXX_FLAGS, append to it. this change
allows user to set Seastar_CXX_FLAGS in the command line.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35610 from pcuzner/wip-grafana-container

monitoring: add grafana container build file

rbd: switch thick-provision image creation to use new write_zeroes API

Also fix some of the concurrency and chunk size issues when handling
fancy striped images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add flag to force thick-provisioning during write_zeroes API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35935 from tchaikov/wip-crimson-no-more-claim

crimson/osd: use operator=(...) instead of claim()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: use operator=(...) instead of claim()

to silence warnings like

objclass.cc:300:26: warning: 'void ceph::buffer::v15_2_0::list::claim(ceph::buffer::v15_2_0::list&)' is deprecated: in favor of operator=(list&&) [-Wdeprecated-declarations]
  300 |   outbl->claim(op.outdata);
      |                          ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35921 from uweigand/test_cdc-endianfix

unittest_cdc: fix test on big-endian hosts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test/common: link unittest_interval_set against GTest::Main

its main() is offered by GTet::Main library

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/interval_set: use template as the 2nd template parameter

to avoid repeating the unit for the interval in the internal map's
parameter.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35932 from zdover23/wip-doc-cephadm-suites-typo

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

doc/cephadm: index.rst: s/suites/suits/ typo fixed.

This commit changes the sentence "Choose the method that best suites
your needs." to "Choose the method that best suits your needs."

Fixes: https://tracker.ceph.com/issues/46359
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35275 from AmnonHanuhov/wip-objclass_impl

crimson/osd: Add support for cls_log

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35871 from tchaikov/wip-45717

mon/PGMap: do not consider changing pg stuck

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35864 from tchaikov/wip-osdc-claim

osdc: use operator=(...) instead of claim()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35863 from wuxingyi/removeunused

osd: Remove unused function declearation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35818 from majianpeng/osd-replicatedbackend-redundant-parameter

osd/ReplicatedBackend: remove redundant parameter.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35817 from tchaikov/wip-drop-py2

qa,test,pybind: drop py2 support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

pybind/mgr/telegraf: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/status: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/selftest: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/restful: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/pg_autoscaler: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/insights: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/influx: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/devicehealth: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/cephadm: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph_objectstore_tool: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/barbican: drop six dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/workunit: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/s3tests: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/rbd: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/radosgw_admin: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/osd_failsafe_enospc: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/{ceph,ceph_manager}: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/pybind: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/dashboard: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: drop py2 support

the python bindings are compiled using python3-cython, and packaged as
python3-{rados,rbd,cephfs,rgw}, so there is no need and is wrong to
claim that they are python2 compatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/vstart_runner: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/standalone: drop py2 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: s/assert/ceph_abort_msg/ when hitting unreachable code

clang++ complains with
```
crimson/os/seastore/root_block.h:100:3: warning: control reaches end of non-void function [-Wreturn-type]
  }
  ^
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: silence -Wunused-lambda-capture warnings

they are pointed out by clang++.

Signed-off-by: Kefu Chai <kchai@redhat.com>