Ernesto Puerta [Wed, 13 Mar 2024 13:06:10 +0000 (14:06 +0100)]
mgr/dashboard: fix NVMeoF API
* Update NVMe-oF gRPC Proto to 1.0.0
* Error handling,
* Missing PATCH for certain namespace ops (resize, set QoS, set balance
groups),
* Stop bypassing gRPC payloads and validate those in the back-end,
* Fix incorrect HTTP 1.1 semantics for some POST/DELETE and URIs.
* Catch errors/exceptions.
* Clean-up EndpointDoc Params
* Run Black linter.
* Remove most of NVMeoFClient glue code between gRPC and controller.
* Fix namespace delete endpoint by exposing trsvcid
* nvmeof io_stats support
Nizamudeen A [Tue, 5 Mar 2024 10:39:25 +0000 (16:09 +0530)]
mgr/dashboard: fix nvmeof api documentation
From Aviv:
POST /api/nvmeof/hosts - the description of the command it wrong IMO. It is not about creating a host. It is about allowing a host X to access subsystem Y.
GET /api/nvmeof/hosts/{subsystem_nqn} - also the description is not accurate. The command lists all hosts that are allowed to access this subsystem.
DELETE /api/nvmeof/hosts/{subsystem_nqn}/{host_nqn} - again the description should be changed as above.
POST /api/nvmeof/namespace - bad formatting of the description
GET /api/nvmeof/subsystem - the description is wrong, should say - "List all NVMeoF subsystems". And it shouldn't get any param.
POST /api/nvmeof/subsystem - few issues here. The serial_number, and max_namespaces are optional (we need to mention that). Also it is missing the --enable-ha argument that is also optional.
Some commands are missing: log_level, connection.i
Patrick Donnelly [Fri, 22 Mar 2024 15:36:40 +0000 (11:36 -0400)]
Merge PR #56202 into squid
* refs/pull/56202/head:
mds/quiesce-db: keep the db thread alive until shutdown
mds/quiesce-db: incorporate review comments
mds/quiesce: declare QuiesceDbPeerListing and QuiesceDbPeerAck
mds/quiesce: resolve the quiesce cluster at the mds monitor
include/types: add an I/O helper for std::unordered_map
messages: avoid using mutable members in MMDSQuiesce*
mds/quiesce-db: incorporate review comments
doc/cephfs/fs-volumes: doc fixes and updates
pybind/mgr: correct type hints for `get_quiesce_leader_info`
mds/quiesce: only use ACTIVE daemons for the quiesce cluster
mds,messages: quiesce db inter-rank messaging
mds/quiesce: MDSRankQuiesce - integration of the quiesce db manager
doc/cephfs/fs-volumes: Add info about the quiesce command
doc: fixes for local dev builds
mgr/volumes: support for `fs subvolume quiesce`
mgr/volumes: use `volume_exception_to_retval` as a decorator
pybind/mgr: add a `one-shot` parameter to send_command
mds/quiesce: QuiesceAgent implementation and unit tests
mds/quiesce: QuiesceDb.h and QuiesceDbManager with tests
common/Timer.cc: improve debug messages from the timer_thread
mds: MDSRank.cc: return status from `send_message_mds`
encoding: add emplace variants for map dencoders
common/Cond: make C_SaferCond private members protected to facilitate inheritance
qa/tasks/cephfs: give the tests more time to run heavy fs workloads
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Ivo Almeida [Wed, 21 Feb 2024 13:02:19 +0000 (13:02 +0000)]
mgr/dashboard: fix retention add for subvolume
- Added parameters for subvolume and subvolume group when adding a new
snap schedule.
- Added call to remove retention policies when removing a snap schedule
in case it is the last one with same path
Fixes: https://tracker.ceph.com/issues/64524 Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 80e1207f4b536fe6edbc81e61cbf951e135eba54)
Ramana Raja [Thu, 29 Feb 2024 17:12:19 +0000 (12:12 -0500)]
qa/suites: add diff-continuous and compare-mirror-image tests
... to rbd and krbd suites respectively.
This allows the compare-mirror-image tests introduced in ea3a567
to be run against various kernel branches, e.g., testing branch.
And allows diff_continuous test in rbd_suite to run against distro
kernel.
After some tests, it turns out that depending on the hardware,
the header 'Location' which is returned by the server after logged can be different.
I could notice the following:
the endpoint passed down to util.query() is wrong:
is passes the full url (scheme://addr:port/path) where it should only
pass the path. The cause is that RedFishClient.login() basically stores
the value of the Location header in `self.location`.
The consequence of this is that it makes the client unable to properly logout.
Matt Benjamin [Tue, 24 Oct 2023 21:59:34 +0000 (17:59 -0400)]
rgwlc: implement Filter size limit checks
i.e., this implements ObjectSizeGreaterThan and ObjectSizeLessThan
for supported expiration and transition actions.
Fixes: https://tracker.ceph.com/issues/63304 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c9b19be15362c63526e92f4d968f942a40cab150)
Matt Benjamin [Mon, 23 Oct 2023 18:57:33 +0000 (14:57 -0400)]
rgwlc: implement NewerNoncurrentVersions
Per AWS doc, this value controls "how many noncurrent versions
Amazon S3 will retain." [1] We understand this to mean, retain
NewerNoncurrentVersions of any object, regardless of expiration.
Nizamudeen A [Thu, 7 Mar 2024 08:43:54 +0000 (14:13 +0530)]
mgr/dashboard: disable applitools e2e
Temporarily disabling this so the CI could turn green. Meanwhile I'll
research for a proper way to handle the applitools e2es which I'll track
on https://tracker.ceph.com/issues/64783
Ilya Dryomov [Wed, 28 Feb 2024 13:20:16 +0000 (14:20 +0100)]
librbd: don't clip expanded diff on truncate in ObjectListSnapsRequest
If the diff was expanded due to LIST_SNAPS_FLAG_WHOLE_OBJECT, clipping
it when handling a truncate is wrong -- when subtracting that interval,
we either split the expanded extent into two or chop off a piece of it.
However the point of LIST_SNAPS_FLAG_WHOLE_OBJECT is to report a single
extent covering the entire object.
Ilya Dryomov [Sun, 18 Feb 2024 10:46:15 +0000 (11:46 +0100)]
librados/snap_set_diff: ignore truncates above size at start
Because currently calc_snap_set_diff() only ever appends to the running
diff, an excessive (either too large or completely bogus) zero extent
is reported in cases where an object is first expanded (with a snapshot
taken at that point) and then truncated but still above the size of the
object as of the starting snapshot.
Leonid Usov [Thu, 30 Nov 2023 14:42:22 +0000 (16:42 +0200)]
mds/quiesce: MDSRankQuiesce - integration of the quiesce db manager
* create an instance of the QuiesceDbManager in the rank
* update membership with a new mdsmap
* add an admin socket command for sending requests to the manager
Leonid Usov [Thu, 4 Jan 2024 17:51:32 +0000 (19:51 +0200)]
mgr/volumes: use `volume_exception_to_retval` as a decorator
When used as a decorator, it saves one indented try-catch block inside the decorated method.
This can be applied to most of the methods in the file, subject to a separate refactoring commit
Leonid Usov [Sun, 26 Nov 2023 11:29:11 +0000 (13:29 +0200)]
mds/quiesce: QuiesceAgent implementation and unit tests
QuiesceAgent is the layer that converts updates from the QuiesceDb
into calls to the QuiesceProtocol APIs, and then sends async acks
back to the db manager following the quiesce protocol events.
Leonid Usov [Tue, 31 Oct 2023 12:46:55 +0000 (14:46 +0200)]
mds/quiesce: QuiesceDb.h and QuiesceDbManager with tests
Quiesce DB is one of the components of the "Consistent Snapshots" epic.
The solution is discussed in a slide deck available for viewing to @redhat users:
https://docs.google.com/presentation/d/1wE3-e9AAme7Q3qmeshUSthJoQGw7-fKTrtS9PsdAIVo/edit?usp=sharing
This commit is focusing on the replicated quiesce database maintained by the MDS rank cluster.
One of the major goals was to design the component in a way that can be easily tested
outside of the MDS infrastructure, which is why the communication layer
has been asbtracted out by introducing just two communication callbacks
that will need to be implemented by the infrastructure.
The most of the component code is delivered in a single coherent commit, along with the uint tests.
Other commits will be dedicated to integration with the MDS infrastructure and other changes
that can't be attributed to the core quiesce db code or its tests.
The quiesce db component is composed of the following major parts/actors:
* QuiesceDbManager is the main actor, implementing both the leader and the replica roles.
Normally, there will be an instance of the manager per MDS rank, although given the
decoupling of the infrastructure and the manager, one can run any number of instances
on a single node, which is how test are working.
* The manager interfaces to the infrastructure via two main APIs with the infrastructure
that provides communication and cluster configuration (actor 2) and the quiesce db
client that is responsible for the quiescing of the roots (actor 3)
** ClusterMembership is how manager is configured to be part of a (virtual) cluster.
This structure will deliver information about other peers, the leader and provide
two communication APIs: send_listing_to for db replication from the leader to replicas
and send_ack for reporting quiesce success from the agents.
** Client Interface consists of a QuisceMap notify callback and a dedicated manager
method to submit asynchronous acks following the agent (rank) quiesce progress.
The API of the quiesce db is described in the slide deck mentioned above. The full scope
of capabilities are encapsulated in a single QuiesceDbRequest structure. This should
simplify the implementation of other components that will have to propagate the functionality
to the administrator user of the volumes plugin.
Venky Shankar [Mon, 4 Mar 2024 13:23:53 +0000 (18:53 +0530)]
mds: disable `defer_client_eviction_on_laggy_osds' by default
This config can result in a single client holding up mds to service
other clients since once a client is deferred from eviction due to
laggy OSD(s), a new clients cap acquire request can be possibly
blocked until the other laggy client resumes operation, i.e., when
the laggy OSD is considered non-laggy anymore.
Disable the config by default till the issue is fixed.
Jos Collin [Thu, 29 Feb 2024 10:50:03 +0000 (16:20 +0530)]
qa: enhance labeled perf counters test for cephfs-mirror
Implements checks for labeled perf counters in the appropriate tests.
This patch verifies snaps_synced, snaps_renamed, snaps_deleted and sync_failures metrics are
updated correctly based on the tests.
Fixes: https://tracker.ceph.com/issues/64486 Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 7b7e5d4221389ce077780450d78a4a1e6a3d0a43)
J. Eric Ivancich [Mon, 11 Mar 2024 21:19:40 +0000 (17:19 -0400)]
rgw: rgw-restore-bucket-index -- sort uses specified temp dir
The sort command sometimes makes use of temporary files. When the user
specifies a directory to be used for temp files, have the sort command
use that same directory.
Matt Benjamin [Mon, 19 Feb 2024 14:01:48 +0000 (09:01 -0500)]
rgw_lc: replace strftime w/fmt and chrono:calendar
It's reliably claimed that std::strftime is not
mt-safe, and this would be a likely root cause of
intermittent scrambled expiration header output cases
that have been reported.
Fixes: https://tracker.ceph.com/issues/63973 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 96a5cbb9aa47ed1a2b51e720475ebc08012cb4cc)
projects / ceph.git / log