Vallari Agrawal [Mon, 13 Jan 2025 11:05:27 +0000 (16:35 +0530)]
mon: do not show nvmeof in 'ceph versions' output
NVMeoF gateway version is independent of ceph version
so 'ceph version' shows wrong nvmeof version in output
(i.e. instead of gateway version, it shows Ceph version).
Hence, remove nvmeof in 'ceph versions' output.
To check for gateway version, use 'gw info' command.
Leonid Chernin [Sun, 20 Oct 2024 06:41:12 +0000 (09:41 +0300)]
Revert "mon/nvmeofgw*: fix HA usecase when gateway has no listeners -"
Temporary revert since it causes host issues
This reverts commit 4e8c7cd7f441e7253e188f3479869afc146689cb.
qa/tasks/ceph: provide configuration for setting configs via mon
These configs may be set using:
ceph:
cluster-config:
entity:
foo: bar
same as the current:
ceph:
config:
entity:
foo: bar
The configs will be set in parallel using the `ceph config set` command.
The main benefit here is to avoid using the ceph.conf to set configs which
cannot be overriden using subsequent `ceph config` command. The only way to
override is to change the ceph.conf in the test (yuck) or the admin socket
(which gets reset when the daemon restarts).
Finally, we can now exploit the `ceph config reset` command will let us
trivially rollback config changes after a test completes. That is exposed
as the `ctx.config_epoch` variable.
Leonid Chernin [Wed, 4 Sep 2024 08:54:59 +0000 (08:54 +0000)]
mon/NVMeofGw*:
1. fix blocklist bug - blockist was not called
2. originally monitor only bloklisted specific ana groups but since we allow
the changing of ns ana grp on the fly for the sake of ns load balance,
it is not good enough and we need to blocklist all the cluster contexts
of the failing gateway Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
(cherry picked from commit 936d3af670f491ecc0898ff0dd6b114b684db79d)
Leonid Chernin [Sun, 1 Sep 2024 13:17:45 +0000 (13:17 +0000)]
mon/NVMeofGw*: Fix issue when ana group of deleted GW was not serviced.
Introduced GW Deleting state Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
Resolves: rhbz#2310380
(cherry picked from commit d4f961a9c94b189c44b72242d731d1b92d8aeef5)
Adam King [Wed, 28 Aug 2024 17:46:59 +0000 (13:46 -0400)]
mgr/orchestrator: allow passing group to apply/add nvmeof commands
We no longer require the group when applying an nvmeof spec
but we still want to allow the commands to take a group
parameter (and this will at least make a group name
required when creating a new service on the command line)
It was decided by the nvmeof team to stick with defaulting to
an empty string rather than forcing the users onto other
non-empty names when they upgrade
It was decided by the nvmeof team to stick with defaulting to
an empty string rather than forcing the users onto other
non-empty names when they upgrade
Adam King [Tue, 20 Aug 2024 14:51:09 +0000 (10:51 -0400)]
cephadm: mount nvmeof certs into container
https://github.com/ceph/ceph/commit/2946b195edfc0b6a5cfcc661079e351601707ecb
incorrectly removed this line and since then these certs are
not being properly mounted into the container. This commit
adds the line back
Samuel Just [Wed, 14 Aug 2024 19:40:50 +0000 (12:40 -0700)]
include/ceph_features: add NVMEOFHA feature bit
Normally, we'd just use the SERVER_SQUID or SERVER_T flags instead of
using an extra feature bit. However, the nvmeof ha monitor paxos
service has had a more complex development journey. There are users
interested in using the nvmeof ha feature in squid, but it didn't make
the cutoff for backporting it. There's an upstream nvmeof-squid branch
in the ceph.git repository with the patches backported for anyone
interested in building it.
However, that means that users of our normal stable releases will see
the feature added to the monitor one release after anyone who chooses to
use the nvmeof-squid branch. We could disallow upgrades from
nvmeof-squid to T, but by adding a feature bit here we make such a
restriction unnecessary.
Adam King [Wed, 31 Jul 2024 17:30:02 +0000 (13:30 -0400)]
mgr/cephadm: make nvme-gw adds be able to handle multiple services/groups
Before this was grabbing the service spec for the first daemon
description in the list. This meant every daemon would be added
with the pool/group of whatever that spec happened to specify.
This patch grabs the spec, and therefore also the pool/group
individually for each nvmeof daemon
Adam King [Thu, 25 Jul 2024 17:36:07 +0000 (13:36 -0400)]
mgr/cephadm: migrate nvmeof specs without group field
As we have added the group field as a requirement for new
nvmeof specs and check for it in spec validation, we need
a migration to populate this field for specs we find that
don't have it.
Leonid Chernin [Tue, 17 Oct 2023 13:25:07 +0000 (13:25 +0000)]
mon: add NVMe-oF gateway monitor and HA
- gateway submodule
Fixes: https://tracker.ceph.com/issues/64777
This PR adds high availability support for the nvmeof Ceph service. High availability means that even in the case that a certain GW is down, there will be another available path for the initiator to be able to continue the IO through another GW. High availability is achieved by running nvmeof service consisting of at least 2 nvmeof GWs in the Ceph cluster. Every GW will be seen by the host (initiator) as a separate path to the nvme namespaces (volumes).
The implementation consists of the following main modules:
- NVMeofGWMon - a PaxosService. It is a monitor that tracks the status of the nvmeof running services, and take actions in case that services fail, and in case services restored.
- NVMeofGwMonitorClient – It is an agent that is running as a part of each nvmeof GW. It is sending beacons to the monitor to signal that the GW is alive. As a part of the beacon, the client also sends information about the service. This information is used by the monitor to take decisions and perform some operations.
- MNVMeofGwBeacon – It is a structure used by the client and the monitor to send/recv the beacons.
- MNVMeofGwMap – The map is tracking the nvmeof GWs status. It also defines what should be the new role of every GW. So in the events of GWs go down or GWs restored, the map will reflect the new role of each GW resulted by these events. The map is distributed to the NVMeofGwMonitorClient on each GW, and it knows to update the GW with the required changes.
It is also adding 3 new mon commands:
- nvme-gw create
- nvme-gw delete
- nvme-gw show
The commands are used by the ceph adm to update the monitor that a new GW is deployed. The monitor will update the map accordingly and will start tracking this GW until it is deleted.
Signed-off-by: Leonid Chernin <lechernin@gmail.com> Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
(cherry picked from commit 5843c6b04bacf9a7c981fca5d874ab3400f855db)
projects / ceph-ci.git / log