Jason Dillaman [Tue, 14 Jul 2020 22:49:30 +0000 (18:49 -0400)]
crush/CrushWrapper: rebuild reverse maps after rebuilding crush map
The Objecter will crash when localized reads are enabled and two threads
attempt to rebuild the (invalidated) reverse maps concurrently. This
should address the issue for the Objecter use-case without the need to
add additional locking.
Fixes: https://tracker.ceph.com/issues/44311 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8b866794f5b3674c5e3ad9adceb5e3230d55a0e0)
David Turner [Tue, 10 Nov 2020 19:31:35 +0000 (14:31 -0500)]
rpm,deb: change sudoers file mode to 440
change sudoers file mode to 440 to match recommended defaults.
From the sudoers man page.
> the default file mode is 0440 (read‐able by owner and group, writable
by none).
> The default mode may be changed via the “sudoers_mode” option to the
sudoers
> Plugin line in the sudo.conf(5) file.
Fixes: https://tracker.ceph.com/issues/48169 Signed-off-by: David Turner <drakonstein@gmail.com>
(cherry picked from commit 1de14c7f9bcaf94152c62b50c37f4a44b6445f58)
Conflicts:
debian/rules
- ignored extra lines introduced post-nautilus
Tatjana Dehler [Tue, 12 Jan 2021 09:28:35 +0000 (10:28 +0100)]
mgr/dashboard: add `--ssl` to `ng serve`
Add the `--ssl` option to the `ng serve` command in order to
reflect the changes to the cookies (https://tracker.ceph.com/issues/44591)
and be able to log into the dashboard by using the development server.
Fixes: https://tracker.ceph.com/issues/48847 Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 3971cdb0a64c077c9d136135b745778d875e0dae)
Conflicts:
src/pybind/mgr/dashboard/frontend/package.json
- Adopted the master branch changes.
sys_siglist is deprecated with glibc 2.32. A new thread-safe and
async-signal safe sigdescr_np() function is provided, so use it if
available.
Fixes: https://tracker.ceph.com/issues/47187 Signed-off-by: David Disseldorp <ddiss@suse.de>
(cherry picked from commit b9b6faf66ae67648626470cb4fc3f0850ac4d842)
Conflicts:
CMakeLists.txt
cmake/modules/CephChecks.cmake
- CephChecks.cmake file does not exist in nautilus; manually cherry-picked the
change in that file to top-level CMakeLists.txt
Jeff Layton [Fri, 20 Nov 2020 23:43:31 +0000 (18:43 -0500)]
client: when STATX_NLINK is requested request FsLs caps
We currently determine the getattr request mask prior to doing the
lookup, so we don't necessarily know whether the terminal entry is
a directory or something else.
Directory nlink count is governed by the dirstats, and those are
under Fs caps. Request both Fs and Ls if the request mask includes
STATX_NLINK.
Fixes: https://tracker.ceph.com/issues/48313 Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit e2ea2214cbb1ff4bd7332bb28db4f3477efb37df)
Jeff Layton [Fri, 20 Nov 2020 15:39:58 +0000 (10:39 -0500)]
client: ensure we take Fs caps when querying dirstat vxattrs
Dirstats values are only updated when Fs caps are held, so ensure that
we request those caps in the _getattr call when querying for them.
Drop the old XATTR_NAME_CEPH helper and rename XATTR_NAME_CEPH2 to
XATTR_NAME_CEPH. Add a new VXATTR_DIRSTAT field and add that to all of
the dirstats vxattrs.
Fixes: https://tracker.ceph.com/issues/48313 Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 3db582b1416dc170a0e25c957166d7bdc58e4c67)
Conflicts:
src/client/Client.cc
- keep hidden because we still need it in nautilus
- drop some code because we did not introduce the
following features: ceph.mirror.info and rsnaps
Nizamudeen A [Fri, 8 Jan 2021 07:37:27 +0000 (13:07 +0530)]
mgr/dashboard: Fix for datatable item not showing details after getting selected
**Regression in nautilus**
Datatable items are not showing the details even if an item in the list is selected. This is happening because of this backport (https://github.com/ceph/ceph/pull/37756/files) which backports the line this.selection.selected = $event['selected']; but this feature was not implemented in the nautilus branch originally.
Fixes: https://tracker.ceph.com/issues/48796 Signed-off-by: Nizamudeen A <nia@redhat.com>
Jason Dillaman [Thu, 19 Nov 2020 20:59:34 +0000 (15:59 -0500)]
pybind/mgr/rbd_support: delay creation of progress event
Create the progress module event upon receipt of the first
progress callback from the librbd API. This will help to ensure
that all prereqs have been validated for retryable errors like
scheduling an image to be removed while it still has attached
cloned children.
Fixes: https://tracker.ceph.com/issues/48296 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c0069b00e4748974d4bf5cfa1bdab68d6f043abb)
Conflicts:
src/pybind/mgr/rbd_support/task.py
- file does not exist in nautilus: applied the changes manually
Ilya Dryomov [Thu, 3 Dec 2020 10:24:32 +0000 (11:24 +0100)]
qa: krbd_stable_pages_required.sh: move to stable_writes attribute
bdi/stable_pages_required attribute was deprecated in 5.10 and now
always returns 0. The replacement is queue/stable_writes. (It is
also writeable, so we can simplify these test cases somewhat in the
future.)
Alfonso Martínez [Tue, 15 Dec 2020 08:28:46 +0000 (09:28 +0100)]
mgr/dashboard: CLI commands: read passwords from file
Fixes: https://tracker.ceph.com/issues/48355 Signed-off-by: Alfonso Martínez <almartin@redhat.com> Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 5d7ee7c1f0ad971fd0079f917e2b44cdef1d6f9f)
- Remove cephadm files and related code (does not apply to nautilus).
- Remove code related to non-existing functionality in nautilus.
- Adapt code to be py2 compatible.
- Resolve conflicts related to code divergence.
Matt Benjamin [Fri, 8 Jan 2021 18:23:56 +0000 (13:23 -0500)]
rgw_file: return common_prefixes in lexical order
Since inception RGWReaddirRequest has sent all leaf objects first
(in lexical order), then common_prefixes (in lexical order). In
hindsight, an overall listing could trivially be returned out of
lexical order, which can cause continued listing of large, mixed
directories to fail.
RCA by Dan Gryniewicz.
Fixes: https://tracker.ceph.com/issues/48410 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e561e98e5cca2678854e01c990f95e474022b7ed)
Conflicts:
src/rgw/services/svc_notify.cc
- robust_notify() call has different argument list in nautilus
src/rgw/services/svc_sys_obj_cache.cc
- distribute_cache() call has different argument list in nautilus
Paul Emmerich [Tue, 11 Jun 2019 12:10:55 +0000 (14:10 +0200)]
rgw: fix bucket limit check fill_status warnings
all variables in the calculation where ints, so the warning would
only ever trigger if objects_per_shard == limit_objects and the
percentage for > 100% used a granularity of 100%
Fixes: http://tracker.ceph.com/issues/40255 Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 694076419e6f20d1ed6c95b4b1ff94f134a10d4d)
which was intentionally omitted from the nautilus cherry-pick because this
package (python3-python3-saml) was not available in SLE-15-SP1/SES6. In
the meantime, the package has been made available there, so the line can
now be backported to nautilus.
Kefu Chai [Tue, 24 Nov 2020 06:12:54 +0000 (14:12 +0800)]
cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users
see also
https://www.boost.org/doc/libs/1_74_0/doc/html/boost_asio/std_executors.html#boost_asio.std_executors.polymorphic_i_o_executor
we could use `asio::any_io_executor` later on though for better
performance.
also, define CMP0093, so FindBoost reports Boost_VERSION in x.y.z
format. it is simpler to use `VERSION_GREATER_EQUAL` to compare its
version with 1.74 instead of its C macro version ("107000").
Conflicts:
CMakeLists.txt
src/librbd/CMakeLists.txt: change ""Boost_VERSION
VERSION_GREATER_EQUAL "1.74" to "Boost_VERSION VERSION_GREATER
1.73" as "VERSION_GREATER_EQUAL" was introduced in a newer version of
CMake, which is not available in ubuntu/xenial.
src/rgw/CMakeLists.txt: add the same macro definition applied to
librbd. without which, src/rgw/rgw_sync_module_aws.cc
still fails to compile with boost 1.74
Tim Serong [Fri, 22 Nov 2019 09:25:19 +0000 (20:25 +1100)]
mgr/PyModule: correctly remove config options
Previously, incorrect parameters were being passed to "config rm",
causing it to do nothing. This commit also ensures the correct
error message is shown for both the set and remove failure cases.
I've also moved the update of the in-memory config map to *after*
the value is persisted, to ensure the config map actually reflects
what's stored.
Kotresh HR [Tue, 4 Aug 2020 07:57:53 +0000 (13:27 +0530)]
mgr/volumes: Make number of cloner threads configurable
The number of cloner threads is set to 4 and it can't be
configured. This patch makes the number of cloner threads
configurable via the mgr config option "max_concurrent_clones".
On an increase in number of cloner threads, it will just
spawn the difference of threads between existing number of
cloner threads and the new configuration. It will not cancel
the running cloner threads.
On decrease in number of cloner threads, the cases are as follows.
1. If all cloner threads are waiting for the job:
In this case, all threads are notified and required number
threads are terminated.
2. If all the cloner threads are processing a job:
In this case, the condition is validated for each thread after
the current job is finished and the thread is termianted if the
condition for required number of cloner threads is not satisified.
3. If few cloner threads are processing and others are waiting:
The threads which are waiting are notified to validate the
number of threads required. If terminating those doesn't satisfy the
required number of threads, the remaining threads are terminated
upon completion of existing job.
Conflicts:
src/pybind/mgr/volumes/fs/volume.py
- keep connection_pool because nautilus does not introduce
the CephfsClient
src/pybind/mgr/volumes/module.py
- Remove nfs export related things as nautilus doesn't contains NFSExport, FSExport class
qa/tasks/cephfs/test_volumes.py
- nautilus expects particular 'mgr.id', hence used 'mgr.x' instead of 'mgr'
Conflicts:
- src/pybind/mgr/dashboard/module.py
Limiting TLS version seems to work only with CherryPy >= 8.9.1 (Ubuntu 18.04 or SUSE Leap 15.1) based on tests in Teuthology. It failed on Ubuntu 16.04 (CherryPy 3.5.0) and RHEL 7.9 (CherryPy 3.2.2).
- qa/tasks/mgr/test_dashboard.py
Tests must be removed because they will not work in Teuthology because of outdated cherrypy versions on most platforms. Only Ubuntu 18.04 (and downstream SUSE Leap 15.1) will work, but it's not possible to limit QA runs to that specific platforms.
Volker Theile [Tue, 15 Dec 2020 09:40:48 +0000 (10:40 +0100)]
mgr/dashboard: Drop invalid RGW client instances, improve logging
* Log RGW settings for easier problem tracking.
* Drop RGW client instances that do not exist anymore. This happens when a RGW service is removed via the orchestrator. The Dashboard tries to access the RGW client using the previous settings which leads to an error which might be confusing. Without dropping the error message 'RGW REST API failed request ...' is displayed instead of the correct one 'No RGW found'. Dropping the RGW client instance will produce correct error messages the next time the backend tries to establish a new connection.
Igor Fedotov [Mon, 5 Oct 2020 14:49:12 +0000 (17:49 +0300)]
os/bluestore: fix segfault on out-of-bound offset provided to claim_free_to_right() call
Hybrid allocator might provide such an offset when final extent is marked as free by HybridAllocator::_add_to_tree().
Hence provides start+size point out to the end of the controled space.
Fixes: https://tracker.ceph.com/issues/47751 Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0792382f4c51ec612d596efb13bf63fea92f3a2d)
Matthew Oliver [Thu, 14 May 2020 01:31:56 +0000 (11:31 +1000)]
rgw: S3 Put Bucket Policy should return 204 on success
Currently RGW returns a 200 on a successful PUT on a bucket policy but
the S3 api extects a 204, which makes sense as it's a success without
any contect returned:
The sample response in the example above is a 204.
This patch checks the op_ret in `RGWPutBucketPolicy::send_response()`
and on a success we turn it to a 204 (or STATUS_NO_CONTENT).
Fixes: https://tracker.ceph.com/issues/45467 Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 7fddff78c98ef9a3c73581b892c7dfa04b64cdb3)
Kotresh HR [Tue, 1 Dec 2020 10:44:17 +0000 (16:14 +0530)]
tasks/cephfs/test_volume_client: Add tests for authorize/deauthorize
1. Add testcase for authorizing auth_id which is not added by
ceph_volume_client
2. Add testcase to test 'allow_existing_id' option
3. Add testcase for deauthorizing auth_id which has got it's caps
updated out of band
Optionally allow authorizing auth-ids not created by ceph_volume_client
via the option 'allow_existing_id'. This can help existing deployers
of manila to disallow/allow authorization of pre-created auth IDs
via a manila driver config that sets 'allow_existing_id' to False/True.
Kotresh HR [Thu, 26 Nov 2020 09:18:16 +0000 (14:48 +0530)]
pybind/ceph_volume_client: Preserve existing caps while authorize/deauthorize auth-id
Authorize/Deauthorize used to overwrite the caps of auth-id which would
end up deleting existing caps. This patch fixes the same by retaining
the existing caps by appending or deleting the new caps as needed.
This patch disallow the ceph_volume_client to authorize the auth_id
which is not created by ceph_volume_client. Those auth_ids could be
created by other means for other use cases which should not be modified
by ceph_volume_client.
Fixes: https://tracker.ceph.com/issues/48555 Signed-off-by: Ramana Raja <rraja@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 3a85d2d04028a323952a31d18cdbefb710be2e2b)
Neha Ojha [Thu, 3 Dec 2020 19:18:04 +0000 (19:18 +0000)]
messages/MMonCommand, MMonCommandAck: don't log values for "config set" and "config-key set"
This acts like a big hammer to avoid adding sensitive information, like passwords
into mon/mgr/cluster logs when using "config set" and "config-key set" to set keys
whose values should be secure.
projects / ceph.git / log