* APIVersion:
* Moved to a separate file
* Added doctests
* Added sentinel values:
* DEFAULT = 1.0
* EXPERIMENTAL = 0.1
* NONE = 0.0
* Added to_mime_type() helper method
* Controllers.__init__:
* Added type hints
* Replaced string versions with APIVersions
* Feedback controller:
* Replaced with EXPERIMENTAL (probably it should be NONE)
Fixes: https://tracker.ceph.com/issues/52480 Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Conflicts:
src/pybind/mgr/dashboard/controllers/__init__.py
- Remove the current changes and keep the incoming new changes
src/pybind/mgr/dashboard/controllers/crush_rule.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/docs.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/feedback.py
- Deleted the file since feedback module isn't backported to pacific
src/pybind/mgr/dashboard/controllers/host.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/openapi.yaml
- Generated a new openapi yaml file
src/pybind/mgr/dashboard/tests/__init__.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_docs.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_host.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_tools.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_versioning.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/crush_rule.py
- Removed the MethodMap decorator which updates the version of the
enpoint to 2.0 because those changes which caused that version
updating were not backported to pacific
mgr/dashboard: make modified API endpoints backward compatible
Fixes: https://tracker.ceph.com/issues/52480 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing APIVersion class to handle versioning for API-endpints and making
them backward compatible.
Cherry-pick notes:
- src/test/rgw/bucket_notification/test_bn.py changes manually applied to src/test/rgw/rgw_multi/tests_ps.py for Pacific
- conflicts in rgw_op.cc due to rename of RGWObject to Object after Pacific
myoungwon oh [Tue, 17 Aug 2021 12:51:29 +0000 (21:51 +0900)]
test: allowing >= the real refcount to avoid false alarm
Current dedup allow to contain multiple same sources using
multiset, which results in inconsistent situation as follow
(during set_chunk, but not confined in set_chunk).
1. User issues set_chunk
2. OSD receives the set_chunk, and sends increment message
to an object in the low tier (INPROGRESS).
3. OSD map is changed (841 → 843)
3.5. on_change() is called
4. the set_chunk op is reenqueued by requeue_op()
5. OSD handles the duplicated set_chunk, but it is not able to
know the set_chunk is duplicated because it does not log on the disk yet.
6. OSD issues increment message again to the object
in the low tier. (increment operation is executed twice)
To fix this, this commit allows >= the real refcount in test cases
fixes: https://tracker.ceph.com/issues/51000
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 126df96)
Recently a new osd config has been added in 6ca32bde2e1d0dd58df168126582a570ac09aad6 and this is getting Modified.
So on our dashboard e2e config check which checks for the Modified
filter, this is also coming on the entry. So we need to increase the
count.
In /etc/sysconfig/ceph we allow operators to define if ceph daemons
should be restarted on upgrade: CEPH_AUTO_RESTART_ON_UPGRADE.
But the post selinux scripts will stop ceph.target regardless if this
is set to `no`, leading to operators adding various hacks to prevent
these unexpected or inconvenient daemon restarts. By now, if users
are using rpms directly, they are likely orchestrating their own
daemon restarts so should not rely on the rpm itself to do this.
Fixes: https://tracker.ceph.com/issues/21672 Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 092a6e3e83e9ef8e37cb6f1033c345dcb5224cfc)
Merge pull request #43002 from ktdreyer/pacific-52472-rm-virtualenv
pacific: *: s/virtualenv/python -m venv/
Reviewed-by: Kefu Chai <kchai@redhat.com> Reviewed-by: Samuel Just <sjust@redhat.com> Reviewed-by: Laura Paduano <lpaduano@suse.com> Reviewed-by: Xiubo Li <xiubli@redhat.com>
Clang complains:
```
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/ClusterWatcher.cc:175:19: error: cannot use parentheses when declaring variable with deduced class template specialization type
std::scoped_lock(m_lock);
^
```
fixes: https://github.com/ceph/ceph/pull/42751 Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 3f952b9c9b69a361441952b279a39b164881ce12)
Fixes: https://tracker.ceph.com/issues/52613 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 221fdb858be083a981697cdb59c19f0659be3f1d)
qa/tasks: Set default caps for 'osd' type in generate_caps()
Assign the default caps for osds to be the same as what the AuthMonitor
sets for a new osd. See AuthMonitor::validate_osd_new() which sets the
following caps for a new osd:
When an actual real world cluster is deployed, the above caps are applied.
Unless the user modifies the defaults, a cluster will operate with the
above caps. Therefore, it makes sense to use the defaults when testing
Ceph so that issues if any due to the default settings may be caught and
fixed.
Therefore, the caps for the 'osd' type is reset to the default in
generate_caps(). The caps for 'mgr' already reflects the system defaults.
The caps for 'mds' type is not changed in this commit and will be
investigated and changed if necessary later.
mon/MonCap: Update osd profile to allow cmd to set iops capacity on mon db
The default mon caps for osds is set to "allow profile osd", which allows
only "rw" capability. Osds with mclock scheduler enabled store their max
iops capacity on the mon config store. This can be achieved by executing
the "config set" command. However, since the osd(s) by default do not have
the execute permission, the command fails with "Permission denied" error.
Therefore, modify the default osd profile to allow running the "config set"
command with restriction to only set keys with name matching either (regex)
"osd_mclock_max_capacity_iops_hdd" or "osd_mclock_max_capacity_iops_ssd"
so that the osd has the permission to update the mon config store with the
desired information.
doc: Update mclock-config-ref doc steps to override osd max iops capacity.
Update the steps in the mclock config reference document to manually
override an OSDs max IOPS capacity. Provide information on the alternative
ways to override the osd_mclock_max_capacity_iops_[hdd,ssd] options for
an OSD.
osd: Add config option to skip running the OSD benchmark on start-up.
Introduce a new dev config option "osd_mclock_skip_benchmark" that
when set skips running the OSD benchmark on start-up. By default
this option is disabled. This is useful in the following scenarios:
- Dev/CI testing,
- Configurations that don't need QoS.
If the option is enabled, the default OSD iops capacity is read from
osd_mclock_max_capacity_iops_[hdd,ssd].
Conflicts:
src/common/options/osd.yaml.in
- Removed non-existent file: src/common/options/osd.yaml.in since the
switch to yaml for config options is not available in pacific yet.
Xiubo Li [Mon, 2 Aug 2021 03:53:37 +0000 (11:53 +0800)]
mds: switch mds_lock to fair mutex
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().
In most case this works well, but in corner case in the Finisher
thread in mds daemon, which may call more than one complete()s
once the mdlog flushing succeeds, after the mdlog flushing is done
it will call the queued complete callbacks and the Finisher thread
could always successfully acquire the mds_lock in successive
complete callbakcs even there may have other threads already being
stuck waiting the mds_lock. This will make the other threads starve
and if they are client's requests, it will cause several or even
tens of seconds long delay for user's operations.
This will switch the mds_lock to fair mutex and it could make sure
that the all the mds_lock waiters are in FIFO order and the Finisher
thread won't hold the mds_lock that long.
At the same time, if the finisher thread has many completes needed
to run the fair mutex could guarantee that the finisher won't be
scheduled out due to fair mutex unlock() if no any other mds_lock
waiter queued.
for better readability, as we don't need to care about the details of
the executor. and the returned futures are blocked in their dtor as
> these actions will not block for the shared state to become ready,
> except that it may block if all of the following are true: the
> shared state was created by a call to std::async, the shared state
> is not yet ready, and this was the last reference to the shared state.
Sage Weil [Mon, 19 Jul 2021 17:46:38 +0000 (13:46 -0400)]
mgr/nfs: use bucket owner creds for rgw bucket export
The bucket owner can always read/write to the bucket, so use those creds
for the export. This is less complicated than setting up a dedicated
user anyway.
projects / ceph.git / log