mgr/dashboard: select placement target on RGW bucket creation

* Select a placement target from the zone that the RGW daemon is running on.

Fixes: https://tracker.ceph.com/issues/40567
Signed-off-by: alfonsomthd <almartin@redhat.com>

Merge pull request #29057 from cyx1231st/wip-seastar-msgr-loggingv2

crimson/net: clean-up and fixes of messenger

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29105 from tspmelo/wip-prevent-propagation

mgr/dashboard: Disable event propagation in the helper icon

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29107 from ricardoasmarques/label-col-size-mgr-module-form

mgr/dashboard: Increase column size on mgr module form

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29112 from ricardoasmarques/iscsi-gateway-status-health-page

mgr/dashboard: Show iSCSI gateways status in the health page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #28942 from ricardoasmarques/validate-iscsi-controls

mgr/dashboard: Validate iSCSI controls min/max value

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28898 from ricardoasmarques/cephfs-client-evict

mgr/dashboard: Evict a CephFS client

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28081 from sebastian-philipp/kubejacker-doc-update

doc/dev/kubernetes: Update

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #29125 from bk201/wip-40827

mgr/dashboard: fix tox test failure

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Evict a CephFS client

Fixes: https://tracker.ceph.com/issues/24892
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Show iSCSI gateways status in the health page

Fixes: https://tracker.ceph.com/issues/39028
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: fix tox test failure

`api/task` now also returns tasks in progress module.
Mock related function for tests.

Fixes: https://tracker.ceph.com/issues/40827
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #29067 into master

* refs/pull/29067/head:
qa/tasks: add exception in do_thrash()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks: add exception in do_thrash()

Fixes the bug during DaemonWatchdog testing:

Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-daemonwatchdog-testing13/qa/tasks/daemonwatchdog.py", line 38, in _run
    self.watch()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-daemonwatchdog-testing13/qa/tasks/daemonwatchdog.py", line 112, in watch
    if thrasher.e is not None:
AttributeError: MonitorThrasher instance has no attribute 'e'

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge PR #29011 into master

* refs/pull/29011/head:
doc/releases/nautilus: move note to 14.2.2
doc/releases/nautilus: typo
doc/releases/nautilus.rst: pool stats fix was merged for 14.2.2 after all
doc: release notes for nautilus 14.2.2

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #28692 into master

* refs/pull/28692/head:
test_volume_client: add positive test for ceph_volume_client method
test_volume_client: rename test_put_object_versioned()
test_volume_client: rewrite test_put_object_versioned
test_volume_client: use sudo_write_file() form teuthology
test_volume_client: make test_object_versioned py3 compatible

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22788 from adamyanova/wip-s3java

A task to run S3 Java tests against RGW

Merge pull request #29048 from rjfd/wip-dashboard-progress-tasks

mgr/dashboard: integrate progress mgr module events into dashboard tasks list

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #28006 from Devp00l/wip-39482

mgr/dashboard: Watch for pool pg's increase and decrease

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #29092 into master

* refs/pull/29092/head:
osd/ECTransaction,ReplicatedBackend: create op is new in octopus

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28935 from votdev/issue_40248

mgr/dashboard: Allow users to change their password on the UI

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #28928 from p-na/wip-pna-e2e-pools

mgr/dashboard: Write E2E tests for pool creation, deletion and verification

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Rafael Quintero <rquinter@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Increase col size on mgr module edit form

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Disable event propagation in the helper icon

Fixes: http://tracker.ceph.com/issues/40715
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28882 from rjfd/wip-mgr-perf-counter-timestamp

mgr: return perf_counters data timestamps in nanosecs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc/releases/nautilus: move note to 14.2.2

Oops! This change was merged in 404ed645d42471b675f11fb7d11f46c006248152,
which was not part of 14.2.1 after all.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases/nautilus: typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29069 into master

* refs/pull/29069/head:
qa/tasks/ceph_manager: make upmap thrasher behave when no pools/pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29095 into master

* refs/pull/29095/head:
script/ptl-tool: update for python3

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #28900 into master

* refs/pull/28900/head:
doc: Update troubleshooting.rst
doc: drop kernel-client-troubleshooting.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: update for python3

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29071 into master

* refs/pull/29071/head:
selinux: Allow ceph to read udev db

Reviewed-by: Sage Weil <sage@redhat.com>

osd/ECTransaction,ReplicatedBackend: create op is new in octopus

Not nautilus!

Fixes: https://tracker.ceph.com/issues/40755
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #28987 into master

* refs/pull/28987/head:
Revert "mgr/dashboard: Add nolockdep option to e2e-script"
common: refactor handling of lockdep's group name in debug locks.
common: shared_mutex_debug doesn't use empty group name for lockdep.
common, crypto: give names to OpenSSL's mutexes.
common: tiny_vector – CPU-friendly container for mutexes & co.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29051 into master

* refs/pull/29051/head:
mon/OSDMonitor: tolerate duplicate MRemoveSnaps messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28468 from jmolmo/ansible_orchestrator

mgr/ansible: RGW service

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr: return perf_counters data timestamps in nanosecs

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend: move ellipsis to before progress in task execution description

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: test_pool: fix pool unit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add rudimentary pool create/exist/delete test

Fixes: http://tracker.ceph.com/issues/38093
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge branch 'master' into ansible_orchestrator

crimson/net: apply strict checks when handle addresses

* Apply strict checks at each step of addresses exchange, make sure we
  fail early when notice inconsistent addresses sent from peer;
* Throw error::bad_peer_address for address validation from peer;
* Stop learning my address before sending it out for peer as identity;
* Fix potential racing when try to learn my address by submit to core 0;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: tests: fix tests by mocking progress service

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend/task.service: format task title for progress tasks

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: add progress module tasks to list of dashboard tasks

Fixes: https://tracker.ceph.com/issues/38202
Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: progress mgr module service

This service module implements helper methods to retrieve
tasks information from the progress mgr module.

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/progress: persist started_at and finished_at timestamps

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/progress: export more event properties to json

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add SSO guard service.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Validate iSCSI controls min/max value

Fixes: https://tracker.ceph.com/issues/38018
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Allow users to change their password on the UI

Fixes: https://tracker.ceph.com/issues/40248
Signed-off-by: Volker Theile <vtheile@suse.com>

crimson/net: fix crimson-osd msgr policies

Also add checks to make sure we have a valid target address with
lossless policy.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: use target_addr to store the address to reconnect

With lossless policy, we will use target_addr to reconnect to peer,
which would be more useful when we switch to entity_addrvec_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: remove duplicated logging when connected

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: set connection policy for v1 protocol

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: use set_ephemeral_port() for non-bound port

Ephemeral port is different from the bound port when connecting, and
different from the peer_addr port when accepting.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #28959 from tchaikov/wip-crimson-op-write

crimson/osd: differentiate write from writefull

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/releases/nautilus.rst: pool stats fix was merged for 14.2.2 after all

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28861 from smanjara/wip-multi-meta

rgw/multisite: Don't allow certain radosgw-admin commands to run on non-master zone

Merge pull request #28809 from joscollin/wip-fix-rgw-warning

rgw: Silence warning: control reaches end of non-void function

Merge PR #28003 into master

* refs/pull/28003/head:
client: do not return EEXIST for mkdirs
pybind/mgr/volumes: print exceptions in purge thread
pybind/mgr/volumes: refactor trash readdir
pybind/mgr/volumes: use existing client provided recursive mkdir
pybind/mgr/volumes: cleanup fs removal
pybind/mgr/subvolumes: use bytes for paths
pybind/mgr/volumes: remove unused property
test: add basic purge queue validation test
mgr / volumes: schedule purge job for volumes on init
mgr / volumes: purge queue for async subvolume delete
mgr / volumes: maintain connection pool for fs volumes
mgr / volumes: wrap rmtree() call within try..except block
mgr / volumes: use negative error codes everywhere
test: cleanup removing all subvolumes before removing subvolume group

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

selinux: Allow ceph to read udev db

We are using libudev and reading the udev db files because of that. We
need to allow ceph to access these files in the SELinux policy.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge PR #29030 into master

* refs/pull/29030/head:
qa/tasks/ceph_deploy: assume systemd and simplify shutdown wonkiness

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #28994 from smithfarm/wip-40645

build/ops: run "npm ci" with a one-hour timeout

Reviewed-by: Tiago Melo <tmelo@suse.com>

qa/tasks/ceph_deploy: assume systemd and simplify shutdown wonkiness

The 'service' command was succeeding but not actually stopping ceph.  But
we can safely assume systemd now so get rid of the weirdness here.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #28839 into master

* refs/pull/28839/head:
osd: support osd_repair_during_recovery

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #28907 into master

* refs/pull/28907/head:
mgr/progress: added the time an event has been in progress
mgr/progress: added the time an event has been in progress
mgr/progress: added the time an event has been in progress
mgr/progress: added the time an event has been in progress

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29027 into master

* refs/pull/29027/head:
common/options: Set concurrent bluestore rocksdb compactions to 2

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tasks/ceph_manager: make upmap thrasher behave when no pools/pgs

Fixes: https://tracker.ceph.com/issues/40635
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: pool: make _get and _wait_for_pgs class methods

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #28820 from trociny/wip-journal-fetch-corrupt-end

journal: return error after first corruption detected

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

client: do not return EEXIST for mkdirs

Behavior should be similar to `mkdir -p`.

Introduced-by: 26905ca82776e8119a347a9a2e8d65c272c7ed62
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28633 from p-na/wip-pna-refactor-unit-tests

mgr/dashboard: Reduce code duplication through TableActionComponent UnitTests

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28999 from rafaelquint/bucketstest3

mgr/dashboard: RGW bucket E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Reduce code duplication through TableActionComponent Unit Tests

Fixes: http://tracker.ceph.com/issues/40399
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

crimson/net: printable accept address

entity_addr_t::TYPE_NONE is not printable, change to TYPE_ANY.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: unify loggings for message exchange

* remove duplicated information in message logs,
  because connection prefix already contains information about:
  "[my_name(my_logic_name) my_addr >> peer_name peer_addr]"
* unify message logs for both v1 and v2;
* continue to use familiar identifiers to highlight message logs:
  - received messages identified by "<== #seq === msg (type)";
  - sent messages identified by "--> #seq === msg (type)";

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: clean up loggings in v2 protocol

Defind log levels in V2 Protocol:
* error level, something error that cause connection to terminate:
  - fatal errors;
  - bugs;
* warn level: something unusual that identifies connection fault or replacement:
  - unstable network;
  - incompatible peer;
  - auth failure;
  - connection race;
  - connection reset;
* info level, something very important to show connection lifecycle,
  which doesn't happen very frequently;
* debug level, important logs for debugging, including:
  - all the messages sent/received (-->/<==);
  - all the frames exchanged (WRITE/GOT);
  - important fields updated (UPDATE);
  - connection state transitions (TRIGGER);
* trace level, trivial logs showing:
  - the exact bytes being sent/received (SEND/RECV(bytes));
  - detailed information of sub-frames;
  - integrity checks;
  - etc.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: cleanup, rename _peer_addr to _my_addr_from_peer

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: abort with unhandled exceptions from dispatcher

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: cleanup, fix spelling error

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: cleanup, return void from unexpected_tag()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: E2E Tests for Create, Edit, Delete, and Invalid Input on RGW Buckets

Fixes: https://tracker.ceph.com/issues/40372
Fixes: https://tracker.ceph.com/issues/40484
Fixes: https://tracker.ceph.com/issues/40259
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

Merge pull request #28879 from Devp00l/wip-40580

mgr/dashboard: Fix the table mouseenter event handling test

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28916 from p-na/wip-pna-fix-tag

mgr/dashboard: Add linter for unclosed HTML tags

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

crimson/net: unify the usage of entity_type_t and entity_name_t

* use standard entity_type_t(uint8_t) in favor of peer_type_t(int);
* use standard ceph_entity_type_name() to print where possible;
* use peer_name to encapsulate peer_type and peer_id in Connection.h;
* clean related interfaces in Messenger and Connection;
* print peer_name in Connection logging prefix;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: Add missing closing tag for container

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #27463 from rjfd/wip-dashboard-ssl-cert

mgr/dashboard: commands to set SSL certificate and key

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28213 from sebastian-philipp/orchestrator_cache

mgr/orchestrator: Add cache for Inventory and Services

Reviewed-by: Tim Serong <tserong@suse.com>

pybind/mgr/volumes: print exceptions in purge thread

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/mgr/volumes: refactor trash readdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/mgr/volumes: use existing client provided recursive mkdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/mgr/volumes: cleanup fs removal

In Nautilus, the simplest sequence is:

    fs fail name
    fs rm name --yes-i-really-mean-it

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/mgr/subvolumes: use bytes for paths

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28700 from batrick/i39620

doc/cephfs: improve add/remove MDS section

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #29033 from neha-ojha/wip-add-yt

doc/dev/developer_guide/index.rst: add youtube references

Reviewed-by: Noah Watkins <nwatkins@redhat.com>

Merge pull request #28649 from votdev/issue_40427

mgr/dashboard: Pool form uses different loading spinner

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mon/OSDMonitor: tolerate duplicate MRemoveSnaps messages

Fixes: http://tracker.ceph.com/issues/40774
Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/dashboard: update SSL configuration instructions

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: added CLI commands to set SSL certificate and key

Fixes: https://tracker.ceph.com/issues/39123
Signed-off-by: Ricardo Dias <rdias@suse.com>