do_cmake.sh: do not unnecessarily warn about debug build

do_cmake.sh forgets to check the arguments passed to it and ends up warning
about debug build even if a user specifies a release build.

Signed-off-by: Abutalib Aghayev <agayev@gmail.com>

Merge pull request #34969 from pritha-srivastava/wip-rgw-gc-fix

cls/rgw_gc: Fixing the iterator used to access urgent data map

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34965 from Devp00l/wip-45433

mgr/dashboard: Always use fast angular unit tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35041 from smithfarm/wip-tcmalloc-nit

doc: fix TCMalloc spelling

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #33121 from tspmelo/wip-angular-9

mgr/dashboard: Update to Angular 9

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc: fix TCMalloc spelling

The google/tcmalloc page on GitHub refers to "TCMalloc", not
"tc_malloc".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35010 from pcuzner/k8sevents-sanitise

mgr/k8sevents:sanitise kubernetes events

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34972 from tspmelo/wip-fix-45445

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #33886 from sebastian-philipp/doc-mon-command-api

doc: Add a generated reference of all mon commands.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35017 from tspmelo/wip-fix-fakefs

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34956 from tchaikov/wip-qa-tasks-cephfs-py3

qa/tasks/cephfs: remove sys.exc_clear() calls

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #34857 from tchaikov/wip-qa-tasks-no-filter

qa/tasks: use list comprehension for checking the length

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Add dashboard node script

This file should be used to agregate all external scripts we need.
Multiple flags can be used in the same call.

Available flags:

--env: Generates angular environment files.

--pre: Modifies 'angular.json' to enable the build of custom locales using
       angular --localize.
       Languages can be defined using the environment variable LANGS,
       if no value is provided all languages will be build.
       Default language is always build, even if not provided.
       p.e.: 'LANGS="pt" node cd --pre', will build EN and PT.
       For backward compatibility we accept both long and short version of
       languages, p.e.: 'pt' and 'pt-BR'

--post: Restores 'angular.json' to its original.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove ngcc warnings

Some packages are still not updated to be compatible with Angular 9
and can cause warnings.

I have whitelisted some that work well, so we don't see the warning every time.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update Node.js to latest LTS: v12.16.2

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update npm dependencies

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix SCSS imports

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix i18n configuration

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Few linting fixes

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update Angular to v9

This was done with "ng update" cli command.

Fixes: https://tracker.ceph.com/issues/44829
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35009 from ivancich/wip-orphan-pending-rel-notes

PendingReleaseNotes: rgw-orphan-list, deprecation, and fixes

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge PR #34978 into master

* refs/pull/34978/head:
qa/tasks/cephfs/mount.py: always setup the NAT rules
qa/tasks/cephfs/mount.py: fall back to use ip command to setup the bridge

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #35021 from tchaikov/wip-cephfs-shell-pep8

cephfs-shell: fix warnings reported by flake8

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

cephfs-shell: fix warnings reported by flake8

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34746 from tspmelo/wip-form-loading

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Fix HomeTest setup

'setup_controllers' was being called outside of the fake fs mock,
so the tests would fail if there was no real dist folder.

Fixes: https://tracker.ceph.com/issues/45516
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34888 from jmolmo/fix_command_parse

cephadm: Fix error parsing shell command arguments

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34945 from sebastian-philipp/cephadm-remove-async-map-resurrect

mgr/cephadm: replace async_map_completion with a simple wrapper (again)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34970 from sebastian-philipp/bluestore-revert-sharding

Revert "os/bluestore: Enable sharding by default"

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #34977 from mgfritch/cephadm-prom-encoding

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa/tasks/cephfs/mount.py: always setup the NAT rules

If the last test failed just after the 'ceph-brx' bridge is created
but with setuping the NAT rules in iptables, so if we run the test
case again, it will just skip seting the 'ceph-brx' and the NAT rules.
Then deleting the NAT rules will get the errors like:

iptables: Bad rule (does a matching rule exist in that chain?).

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/cephfs/mount.py: fall back to use ip command to setup the bridge

CentOS/RHEL8 have abandoned the bridge-utils package so the brctl
cmd is none avalible, and on Ubuntu 18.04 and 20.04 the nmcli is
buggy to setup the bridge. So this will fall back to use ip cmd
to setup the bridge stuff.

Fixes: https://tracker.ceph.com/issues/45459
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34955 from tchaikov/wip-cbt-with-bluestore

scripts/run-cbt.sh: use bluestore for testing classic osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #34962 into master

* refs/pull/34962/head:
vstart_runner.py: use tuple instead of set

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/k8sevents:sanitise kubernetes events

Kubernetes can generate events without a timestamp
or an event count. When this occurs the k8sevents 'ls'
command fails due to None type values. This patch
sanitises the event received before adding to the
internal data structures to account for these
issues.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

PendingReleaseNotes: add missing quote character

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

PendingReleaseNotes: `rgw-orphan-list` and deprecated `radosgw-admin orphans`

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #34938 from mgfritch/cephadm-systemd-stop-timeout

cephadm: extend systemd stop timeout

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34905 from tspmelo/wip-mirroring

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

vstart_runner.py: use tuple instead of set

Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34926 from sebastian-philipp/mgr-mock-mon-store

pybind/mgr: properly mock the mon store for pytest

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Revert "os/bluestore: Enable sharding by default"

This reverts commit aa1a116f9eef5d6fbe09d6ed99910223451431f1.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

non-ascii chars introduced by 653c3f66823

Fixes: https://tracker.ceph.com/issues/45458
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #34948 into master

* refs/pull/34948/head:
doc/cephfs-shell: Add quota command usage
doc/cephfs-shell: Add du command usage
doc/cephfs-shell: Add df command usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34958 from yuvalif/fix_versioning_in_notif

rgw/notifications: version id was not sent in versioned buckets

Merge pull request #34967 from tchaikov/wip-rpm-fmt

rpm: BuildRequires fmt-devel on el8

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34781 from lxbsz/run_shell1

qa/tasks/cephfs/mount.py: run_shell to use StringIO as default

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34834 from sidharthanup/qa-test-exports-fix

qa/test_exports: fix TestExports failure under new py3 compability changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34871 from tchaikov/wip-qa-tasks-mgr-set-application

qa/tasks/mgr: always add "application" label to pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34895 from mgfritch/mgr-insights-py3-warn

mgr/insights: fix python3 DeprecationWarning

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34875 from tchaikov/wip-qa-tasks-qemu

qa/tasks/qemu: open file in text mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #34941 into master

* refs/pull/34941/head:
qa/cephfs: use StringIO instead of BytesIO

Reviewed-by: Xiubo Li <xiubli@redhat.com>

mgr/dashboard: Fix random E2E error in mgr-modules

This test failed at random times when it tried to find the new value of pool_ids
in the balancer module.
This happened because the value of pool_ids is automatically reverted by ceph,
so in some situations when we tried to read the new value,
it was already reverted and failed.

Enhanced the tests to be able to use any text input, not only the ones with
empty default values.

Fixes: https://tracker.ceph.com/issues/45445
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34037 from ivancich/wip-adjust-ordered-list-attempts

rgw: clean up attempt loop in ordered listing

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #33865 from ivancich/wip-fix-ordered-marker-bug

rgw: fix bug where bucket listing end marker not always set correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30511 from tianshan/fix_41953

rgw: skip add empty multipart chain to gc in abort_multipart_upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29886 from tianshan/fix_bucket_handle_leak

rgw: fix some list buckets handle leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34961 from sebastian-philipp/cephadm-fix-node-exporter-auth-entity

mgr/cephadm: node-exporter is not a ceph service

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34932 from sebastian-philipp/cephadm-hostassignment-schedule.py

mgr/cephadm: move HostAssignment to new module

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34689 from rhcs-dashboard/45186-fix-test-orch

mgr/test_orchestrator: fix _get_ceph_daemons()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

cls/rgw_gc: Fixing the iterator used to access urgent data map
in xattr in cls_rgw_gc. The incorrect iterator caused a crash
in osd.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rpm: BuildRequires fmt-devel on el8

it's included in EPEL8 now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34950 from sebastian-philipp/cephadm-upgrade.py

mgr/cephadm: move upgrade code to upgrade.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34922 from SUSE/wip-fix-45393

mgr/cephadm: refresh osd config when mons change

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34903 from sebastian-philipp/cephadm-adopt-qa-grafana

qa/workunits/cephadm: test grafana adoption

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: Always use fast angular unit tests

This will replace the current approach having a fast testing workaround
that didn't fully work on every test. Now using ng-bullet's optimization
works with every test, therefore there is no need to keep the old
configuration files and use the slow testing on Jenkins.

Fixes: https://tracker.ceph.com/issues/45433
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #34949 from tspmelo/wip-i18n-fix

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/cephfs-shell: Add quota command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add du command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add df command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #34923 from ofriedma/wip-ofriedma-fcgi

rgw: unable to compile fcgi frontend

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #34709 from votdev/issue_45204_max_buckets

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34854 from BenoitKnecht/fix-alert-description

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/cephadm: add extra function for the auth entity

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: node-exporter is not a ceph service

Fixes: https://tracker.ceph.com/issues/45427
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

pybind/mgr: properly mock the mon store for pytest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34843 from mgfritch/cephadm-cli-servicespec

mgr/orch: Clean-up ServiceSpec handling during add/remove (mds, rgw, iscsi, nfs)

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: add some more type checks

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: cleanup async_map_completion etc.

This breaks the progress integration, but we haven't
used that anyway till now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_completion with trivial_completion

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_map_completion with a simple wrapper

There is no need to wrap everything into completions.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a78f42e225bbc041324dc3e565deb207feddad26)

mgr/cephadm: move HostAssignment to new module

(speeds up my editor)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34909 from sebastian-philipp/cephadm-doc-ceph.conf

doc/cephadm: add section about setting config settings

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34891 from sebastian-philipp/cephadm-cephadmservice

mgr/cephadm: Add CephadmService class

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

cephadm: Fix error parsing <command> arguments

Optional arguments in the <command> parameter used in:
 <cephadm shell>
 <cephadm enter>
 <cephadm ceph-volume>
are not parsed properly. This fixes this problem.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

rgw/notifications: version id was not sent in versioned buckets

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45440

qa/tasks/cephfs: remove sys.exc_clear() calls

sys.exc_clear() was removed in Python3, see
https://docs.python.org/3/whatsnew/3.0.html#index-22. so we should not
call it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34853 from yison/rwl-image-writeback-cache-seq9

rbd/cache: Replicated Write Log core codes - compare_and_write

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34952 from cbodley/wip-44958

doc/rgw: document 'rgw gc max concurrent io'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

scripts/run-cbt.sh: use bluestore for testing classic osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34929 from tspmelo/wip-e2e-script

mgr/dashboard: Improve run-frontend-e2e-tests.sh

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/rgw: document 'rgw gc max concurrent io'

Fixes: https://tracker.ceph.com/issues/44958
Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/cephadm: move upgrade code to upgrade.py

Make it easier for everyone to read the upgrade algorithm.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Fix I18N errors

We were just checking if the translation sources were correct and ignore if the
extraction simply failed.

Fixes: https://tracker.ceph.com/issues/45428
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34820 from sebastian-philipp/akamac-cephadm-fix-typo-in-postinst

debian: fix typo in cephadm user gecos

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Add IscsiService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add CrashService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add NodeExporterService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add PrometheusService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add AlertmanagerService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>