* refs/pull/39408/head:
test: add test for validating cephfs-mirror daemon service status
pybind/mirroring: interface to fetch mirror daemon status
cephfs-mirror: register mirror daemon with service manager
cephfs-mirror: remove unneeded json_spirit header
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/39687/head:
tools/cephfs-shell: continue file listing even on error
tools/cephfs-shell: refactoring of code related listing
tools/cephfs-shell: fix listing of symbolic links
Reviewed-by: Rishabh Dave <ridave@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Tue, 9 Mar 2021 18:15:20 +0000 (12:15 -0600)]
mgr/cephadm: do not prime service cache on reconfig
Ceph daemon reconfig does not need any daemon state refresh since we don't
do a restart--we just rewrite the ceph.conf. This also avoids priming
our cache with a 'starting' state when the daemon wasn't touched.
Fixes: https://tracker.ceph.com/issues/49675 Signed-off-by: Sage Weil <sage@newdream.net>
Kefu Chai [Thu, 4 Mar 2021 10:25:33 +0000 (18:25 +0800)]
cmake: silence "You are in 'detached HEAD' state" warning
git warns us when the repo is in 'detached HEAD' state, like:
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in
this state without impacting any branches by switching back to a branch.
...
Turn off this advice by setting config variable advice.detachedHead to
false
Nizamudeen A [Tue, 2 Feb 2021 12:26:13 +0000 (17:56 +0530)]
mgr/dashboard: Host Maintenance Feature
In Cluster -> Hosts, I've added additional button to put the selected host on maintenance or exit out of the maintenance mode. Also for some hosts the ok-to-stop tests may trigger some warnings which requires a --force command to pass along with the maintenance enter command to enter a host into maintenance. In UI this is achieved using a confirmation Modal. In addition to this if the check error is It is NOT safe to stop the host then the host wont be able to put into maintenance mode.
Fixes: https://tracker.ceph.com/issues/49101 Signed-off-by: Nizamudeen A <nia@redhat.com>
for the stated purpose of hiding the %_smp_mflags macro in a higher-level macro.
But, on SUSE, the higher-level macro (%make_build) expands to:
make -O %{_smp_mflags}
The addition of the -O flag makes the build considerably slower and increases
the memory requirement. The exact reason for this is unknown - possibly it's due
to a bug in make, although the same slowness was observed with ninja as well.
In any event, this is a deal-breaker when building in the OBS, because the build
infrastructure there is optimized for builds that do not require huge amounts of
memory and we would rather have a fast build with mixed up compiler messages
than a very slow one with synced compiler messages.
Kefu Chai [Sat, 6 Mar 2021 16:32:42 +0000 (00:32 +0800)]
.github: correct the regex in mileston workflow
also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.
Kefu Chai [Sat, 6 Mar 2021 07:43:33 +0000 (15:43 +0800)]
cmake: make the linkage to pmem::pmemobj public
tools/ceph-dencoder/rbd_types.cc includes Types.h which in turn includes
libpmemobj.h via librbd/cache/pwl/Types.h. and ceph-dencoder pulls in the
rbd_type.cc's linked libraries by linking against rbd_types. but before
this change, rbd_types links against pmem::pmemobj as a PRIVATE library.
so, if we want to pull in rbd_types linkage we should always link
rbd_types as a PUBLIC library. as rbd_types include libpmemobj.h in its
header file.
Kefu Chai [Sat, 6 Mar 2021 04:22:39 +0000 (12:22 +0800)]
cmake: link libpmemobj against libpmem
libpmemobj should link against libpmem, but, in CMake, imported library
does not allow PRIVATE linkage. so pmem::pmem is added to the list of
INTERFACE_LINK_LIBRARIES.
Kefu Chai [Fri, 5 Mar 2021 06:04:23 +0000 (14:04 +0800)]
cmake: support COMPONENTS param in Findpmem.cmake
add two components: pmem and pmemobj to this package. so we can find
them and link against them in a more intuitive way.
before this change the COMPONENTS parameter passed to
find_package(pmem ...)
is dropped on the floor and ignored.
after this change, it is checked and taken into consideration.
also, in this change, the exposed variables are renamed from
PMEM_* to pmem_*
to be consistent with the package name. it's encouraged to be consistent
with the package name when it comes to the INCLUDE_DIR and LIBRARIES
variable names.
Marcus Watts [Tue, 2 Mar 2021 04:10:35 +0000 (23:10 -0500)]
rgw/kms/vault - PendingReleaseNotes pointer
The "new" transit logic requires configure changes to be
effective. Here is a pointer to the revised
documentation for those who already have data encrypted
using the previous version.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Tue, 26 Jan 2021 01:49:16 +0000 (20:49 -0500)]
rgw/kms/vault - s3tests for both old and new test logic.
Test both "old" and "new" transit logic with s3tests. Does not test
migration - that will need to be done separately. I've added
a "flavor" parameter so the test logic can tell the difference
between the "old" engine and the "new" engine. The vault
keys creation logic now has options to determine whether
the keys created are exportable (needed for the old transit
engine), or not (should be the case going forward with the
new transit engine.)
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Fri, 8 Jan 2021 22:49:20 +0000 (17:49 -0500)]
rgw/kms/vault - rework unit test logic for new transit logic.
The "new" transit logic is organized quite differently
than the old logic, so the existing unit test logic was
very broken. Additionally, it's possible to test the
input arguments and send_request() has more of them now,
so add logic to verify most of those arguments are correct.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Tue, 5 Jan 2021 02:22:07 +0000 (21:22 -0500)]
rgw/kms/vault - document configuration for new transit logic
Using the new transit logic requires slightly different configuration.
additionally there is some backwards compatibility support, which
also needed documentation.
The existing description of how to configure hashicorp vault
to work with ceph was also incomplete. I've fleshed that out a bit,
including considerably more information on how to use configure
and use the vault secret agent with ceph.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Tue, 5 Jan 2021 02:21:33 +0000 (21:21 -0500)]
rgw/kms/vault - new transit logic - fix compat logic
Teuthology passes in a vault uri that ends in /export/encryption-key/
So: we need to handle (and ignore) trailing slashes when deciding
to enable compatibility support.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Tue, 8 Dec 2020 23:09:04 +0000 (18:09 -0500)]
rgw/kms/vault - "compat" option
For transit engine:
"compat" option: 0=new only, 1=old & new, 2=old only.
This is just the option parsing itself: not the actual logic
for make_key | reconstitute_key.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Mon, 7 Dec 2020 22:55:22 +0000 (17:55 -0500)]
rgw/kms/vault - encryption context - first part
This includes the logic to process the user provided
encryption context, turn it into "canonical json", and
to add in a default arn if it's not present.
Also present here is the start of logic to distinguish
between "prepare_encrypt" and "prepare_decrypt" at a lower
level; as "make_key" and "reconstitute_key" these will be
the functions that separately create a new datakey using
the vault transit operation, and to retrieve a previously
stored datakey.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Rishabh Dave [Wed, 3 Mar 2021 11:44:22 +0000 (17:14 +0530)]
qa/cephfs: move is_blocklisted() to filesystem.CephCluster
Using self.fs.mon_manager in mount.py can lead to a crash since self.fs
can be None. Move is_blocklisted() to tasks.filesystem.CephCluster where
it can get access to mon_manager without depending on objects
representing Ceph FSs.
Fixes: https://tracker.ceph.com/issues/49511 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Marcus Watts [Mon, 7 Dec 2020 22:53:05 +0000 (17:53 -0500)]
rgw/kms/vault - define attribute to store encryption context
For rgw sse:kms use, the aws s3 standard provides an attribute
to store the base-64 encoded canonical json "encryption context".
This should be used to vary the per-object keys used for the
actual object encryption.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
Marcus Watts [Mon, 7 Dec 2020 22:48:31 +0000 (17:48 -0500)]
rgw/kms/vault - share get/set attr between rgw_crypt.cc and rgw_kms.cc
In order to pass down and manage "attrs" from crypt logic to kms
logic, it's necessary to share the functions that can get and
set strings in that structure. Eventually, I plan to have
the various engines store and retrieve a per-object "datakey" that
is encrypted (wrapped) by the named kms key.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
* refs/pull/38819/head:
win32*.sh: use ninja instead of make
doc: add ceph-dokan documentation
cephfs: Add ceph-dokan, providing Windows support
cephfs: add ceph_may_delete function
cephfs: disable inode emulation on Windows
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Marcus Watts [Mon, 7 Dec 2020 22:28:59 +0000 (17:28 -0500)]
rgw/kms/vault - relax configuration parsing for rgw_crypt_vault_secret_engine
To better manage forwards and backwards compatibility when using vault
transit for rgw object encryption (sse:kms); it is desirable to provide
parameters to control how this works. It was more attractive to overload
the existing rgw_crypt_vault_secret_engine parameter for this purpose
than to invent one or more all-new parameters.
Additionally, the enum support in the configuration parser looks like
it ought to have helpful syntax checking functionality. This is not so;
failure to provide a supported enum results in silently replacing that
with the default option, resulting in confusing and non-obvious behavior
that is not at all helpful.
This change removes the enum constraint on rgw_crypt_vault_secret_engine,
allowing for more useful messages from the rgw code, and the possibility
to also provide additional information on the same line.
Fixes: http://tracker.ceph.com/issues/48746 Signed-off-by: Marcus Watts <mwatts@redhat.com>
projects / ceph.git / log