doc/man: remove docs about support for unix domain sockets
doc/man: support for unix domain sockets is not implemented, hence we
removed documentation about it.
(Note: the changes in this commit were the work of Rok Jaklič in
https://github.com/ceph/ceph/pull/48537. This pull request has been
raised because that pull request was for some mysterious reason causing
merge conflicts that were never resolved.)
Co-authored-by: Rok Jaklič rjaklic@gmail.com Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fa40b7ef560fc60a107dad1604650e0bcf27e77e)
tgfree [Fri, 3 Mar 2023 03:17:59 +0000 (11:17 +0800)]
doc/radosgw: multisite - edit "migrating a single-site"
Add the import step of the section "Migrating a Single-Site Deployment to Multi-Site"
in doc/radosgw/multisite.rst. The field "name" and "api_name" should be matched.
John Mulligan [Wed, 24 May 2023 17:42:26 +0000 (13:42 -0400)]
doc: make instructions to get an updated cephadm common
As discussed in person and over the ceph orch weekly, we want all users
to use a recent supported version of cephadm. Previously, the
instructions only had those downloading cephadm with curl using the
"add-repo" and "install" commands to get a up-to-date cephadm build.
According to ADK we've seen cases of users get "old" distro packages
in the past. Change the instructions so that the "update cephadm" steps
are common after acquiring a "bootstrap copy" of cephadm.
Starting with reef, cephadm is a compiled (zipapp) python application.
The cephadm script has been renamed and thus the old curl-based
download instructions will no loner work. While cephadm still has
no dependencies outside the Python stdlib, this will be changed in
future versions so it is no longer appropriate to just download the
source file of cephadm and run it either.
This change updates the `Install cephadm` section of the doc to explain
how to acquire a "compiled" version of cephadm as well as:
* moving and tweaking the note that the two installation methods are
distinct
* adding a new note linking to instructions on building cephadm
* moving the distribution-specific installations before the curl-based
installation to subtly hint that we prefer you to get it using
packages if you can
* Noting cephadm's minimal required python verision and how to run it
with a particular python version.
Adam King [Tue, 1 Aug 2023 21:43:36 +0000 (17:43 -0400)]
mgr/cephadm: filter hosts that can't support VIP for ingress
Keepalive daemons need the host to have an interface
on which they can set up their VIP. If a host
does not have any interface that can work, we should
filter it out
Adam King [Tue, 1 Aug 2023 20:32:06 +0000 (16:32 -0400)]
mgr/cephadm: select IPs/interface based on VIP for keepalive conf
We need to make sure the keepalive conf sets
the unicast src and peer IPs to be the ones
in the same subnet as the VIP we're setting up,
as well as specify the correct interface. Otherwise,
the keepalive daemons don't speak to each other
properly and all end up going into MASTER state.
Luis Domingues [Tue, 20 Jun 2023 07:07:57 +0000 (09:07 +0200)]
cephadm: Add parameter to set keepalived's virtual router id
When deploying more than 1 ingress, two instances of keepalived
can be generated with the same virtual_router_id. This commit
adds posibility to change the virtual_router_id of keepalived
from the spec file.
Luis Domingues [Tue, 13 Jun 2023 07:59:35 +0000 (09:59 +0200)]
cephadm: Split multicast interface and unicast_ip in keepalived.conf
When deploying keepalived, cephadm set interface and unicast_src_ip
in keepalived.conf. However, having both options are set, but
if unicast_src_ip is not in the interface set by 'interface',
the instrances of keepalived will not properly commuicate.
This commit makes both options exclusive, and add an option to set
either one or the other. Default is set to 'interface', as it seems
multicast is the default way to deploy keepalived.
Adam King [Wed, 23 Aug 2023 21:24:13 +0000 (17:24 -0400)]
cephadm: fixups for unit tests
These fixes really should be part of the commits
that brought in/changed the tests, but it's
difficult to find the exact commits as part of
this batch backport so I'm doing these minor fixups
in their own commit
John Mulligan [Wed, 12 Jul 2023 14:18:49 +0000 (10:18 -0400)]
pybind/mgr/cephadm: extend haproxy protocol test case
Extend that haproxy protocol test case so that for host1 we exercise the
new logic for gathering known-good IPs for the host and add those to
the ganesha config.
John Mulligan [Tue, 11 Jul 2023 21:07:25 +0000 (17:07 -0400)]
pybind/mgr/cephadm: add all known good IPs for hosts to haproxy list
Fixes: https://tracker.ceph.com/issues/61852
It was discovered that when the primary IP address of hosts managed by
cephadm are not all on the same subnet, then there's a chance that
ganesha will reject haproxy protocol connections.
Expand the list to all valid addresses for the hosts under management
such that we will not reject proxy protocol messages from the hosts
under cephadm management. This method was chosen over more sophisticated
possible approaches because we do not want to complicate the workflow
for the people setting up haxproxy+ganesha and do not want to miss any
possible subtle edge cases by trying to come up with the "perfect" list
of addresses.
John Mulligan [Tue, 11 Jul 2023 20:29:44 +0000 (16:29 -0400)]
pybind/mgr/cephadm: move haproxy client addrs to func
Move the list comprehension that gets the list of IP addresses for the
haproxy_hosts list to a separate function. This is in preparation for
making the logic more complex in a later commit.
Adam King [Fri, 18 Aug 2023 16:32:38 +0000 (12:32 -0400)]
cephadm: have attempt_bind raise up OSErrors
Before it was always converting the OSError to
our self-defined "Error" class. This causes an issue
with the port_in_use function that has special handling
for OSError when the errno is EADDRNOTAVAIL or
EAFNOSUPPORT. Since the error being raised was no
longer an OSError it wasn't being caught and checked
properly in port_in_use.
This has the additional property of being necessary
to check port availability for haproxy on its VIP. If
we fail deployment when EADDRNOTAVAIL is raised, it becomes
difficult to deploy the ingress service. If we deploy
haproxy first it fails because the VIP isn't available
yet (since keepalive isn't up) and it fails saying the port
it wants to bind to is unavailable (specifically EADDRNOTAVAIL).
If we try to deploy keepalive first it fails because it
needs to know the location of the haproxy daemons in
order to build its config file. This has worked in the past
by just having the haproxy fail to bind at first and then
fix itself once the keepalive daemon is deployed. That
no longer works if the haproxy daemon fails to deploy
because cephadm is reporting the port it needs is
unavailable. Since EADDRNOTAVAIL when deploying
haproxy likely means the VIP is not up rather than
something else is taking up the port it needs, fixing
the handling of this allows ingress deployment to
work while also allowing multiple haproxy daemons
on the same host to use the same frontend port
bound to different VIPs.
Adam King [Fri, 11 Aug 2023 18:00:31 +0000 (14:00 -0400)]
mgr/cephadm: only check haproxy frontend_port conflicts on VIP
If we know what IP the frontend_port will be binding
to, we can pass that down through the port_ips mapping
so cephadm will only check if that port on that specific
VIP if in use. This allows multiple haproxy daemons
to be bound to the same port on different VIPs on the
same host.
Note that you still must use a different monitor port
for the two different ingress services as that port
is bound to on the actual IP of the host. Only the
frontend port can be the same for haproxies on the
same host as long as the VIP is different.
Adam King [Fri, 11 Aug 2023 17:35:10 +0000 (13:35 -0400)]
cephadm: support for specifying IP a port will be bound to on
This is mostly for checking for port conflicts.
Currently, we just check if the port is bound to
on any IP on the host. This mechanism should allow
certain daemon types to specify a port -> IP mapping
that will be passed to the cephadm binary. That
mapping will then be used by cephadm to only
check for the port being bound to on that specific
IP rather than any IP on the host. The end result
is we could have daemons bound to the same
port on different IPs on the same node.
It's expected that daemon types will set this
up as part of their prepare_create or generate_config
functions where they may have more info about the
specific IPs and ports they need.
John Mulligan [Tue, 21 Mar 2023 19:40:17 +0000 (15:40 -0400)]
mgr/cephadm: add test explicitly validating proxy protocol
Adds a test case that specifically ensures that the proper values
appear in the configuration files for both nfs and haproxy when
the proxy protocol is enabled.
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/cephadm: minor reorganizing of nfs service test
This change makes some formatting and organizational changes to one of
the nfs + ingress test cases. This is in preparation for a future change
that will parametrize this function allowing it to test haproxy config
when proxy protocol is enabled and when not.
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/nfs: add support for selecting between haproxy ingress modes
When the haproxy-standard mode is selected, the haproxy ingress will
behave as in previous versions of ceph. When the haproxy-protocol mode
is selected, we need to enable protocol support on the ingress service
and nfs service.
While were at it, make a few organization tweaks to improve readability.
Fixes: https://tracker.ceph.com/issues/58933 Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 322c109f6bd6fa9c4449b4f938d907e1bc75629d)
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/cephadm: add enable_haproxy_protocol to ingress and nfs service specs
Add a new boolean field, enable_haproxy_protocol, to both the ingress
service spec and the nfs service spec. The ingress service spec needs
the field to tell haproxy to enable proxy protocol (v2) support.
The nfs service spec needs the field to allow the nfs server to accept
proxy protocol messages.
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/cephadm: add optional HAProxy_Hosts field to ganesha conf template
When NFS-Ganesha is used behind a HAProxy with proxy protocol support
enabled ganesha requires that HAProxy_Hosts is set to a list of hosts
that are permitted to use the proxy protocol. This change adds support
for specifying HAProxy_Hosts in the ganesha configuration file template.
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/cephadm: add support for default server options to template
Add support for default server-options within the haproxy backend
section. This change will later be used to enable haproxy proxy protocol
(v2) support.
Adam King [Wed, 2 Aug 2023 19:53:10 +0000 (15:53 -0400)]
mgr/cephadm: rename "igw_id" to "nvmeof_gw_id"
"igw_id" was leftover from the nvmeof implementation
being taken heavily from the iscsi implementation. "igw"
means nothing in this context, so we can change the name.
Adam King [Fri, 28 Jul 2023 20:59:42 +0000 (16:59 -0400)]
mgr/cephadm: make nvme-of auth keyring more restricted
Rather than giving full admin privileges,
try to be a bit more strict by limiting it
to profile rbd mon caps and full OSD
privileges for rbd tagged pools. I also wanted
to include an OSD cap like
allow all pool="*" object_prefix "nvmeof.state"
but this caused a failure in the nvme-of daemon
RADOS permission error (Failed to operate write op for oid nvmeof.None.state)
Adam King [Fri, 28 Jul 2023 17:24:06 +0000 (13:24 -0400)]
mgr/cephadm: still remove daemons in error state if ok-to-stop fails
The ok-to-stop function works for certain daemons
by checking if there are at least a certain number
(typically 1) daemon(s) that are actually running
and saying it's not ok-to-stop if if that won't
be true after the removals. This case breaks down
when all the daemons are in error state, making
it so cephadm will refuse to remove a set of
daemons that aren't even working because they're
not "ok to stop". Since ok-to-stop works in a
yes or no fashion, something like this where we
want to be willing to remove a certain subset
(or potentially all currently deployed) daemons
it's easier to keep this logic as part of applying
the service
Adam King [Thu, 27 Jul 2023 19:47:51 +0000 (15:47 -0400)]
cephadm: mount keyring deployed for nvmeof daemon
Before, we were just using the client.admin keyring
as a temporary workaround while we figured out
how to get the keyring to work. We should swap
over to using the keyring we actually generated
for the nvmeof daemon.
Adam King [Thu, 27 Jul 2023 19:44:47 +0000 (15:44 -0400)]
mgr/cephadm: add rados_id to nvmeof conf
This is going to be used as the rados_id
to be set when connecting to the cluster using
the keyring we generate for the nvmeof daemon.
The python librados library defaults the name
to "client.admin" and so if we don't provide
a name or rados_id, we'll only be able to
use nvmeof with the "client.admin" keyring
John Mulligan [Mon, 26 Jun 2023 16:08:34 +0000 (12:08 -0400)]
docs/cephadm: document new extra_{container,entrypoint}_args behavior
Document the new behaviors of extra_container_args and
extra_entrypoint_args. Document that current (previous in code terms)
behavior of splitting strings on spaces and document the recently added
support for ArgumentSpec objects and how they are used.
John Mulligan [Mon, 29 May 2023 20:52:31 +0000 (16:52 -0400)]
mgr/cephadm: rework how _process_ls_output builds a DaemonDescription
Previously, the _process_ls_output function would create an
DaemonDescription and then assign values to attributes one-by-one. This
prevents the class from doing any data type/value conversions unless we
resort to more complex approaches like descriptors or `__setattr__` for
example. Update the _process_ls_output function to build a
DaemonDescription using keyword args so that we have the opportunity
later to convert data passed to it in `__init__`.
John Mulligan [Wed, 17 May 2023 17:50:19 +0000 (13:50 -0400)]
python-common: replace types for extra_*_args with ArgSpec based types
Use the ArgSpec, ArgumentList, and GeneralArgList types throughout
the deployment module to support extra_container_args and
extra_entrypoint_args. The GeneralArgList type supports all possible
input forms while ArgumentList reflects the internal representation
of the argument lists.
John Mulligan [Wed, 17 May 2023 17:49:23 +0000 (13:49 -0400)]
python-common: add an ArgumentSpec type for handling args
Add a new ArgumentSpec type to the deployment package. The
ArgumentSpec serves a dual purpose:
* to allow more advanced configuration of extra arguments
* to keep backwards compatibility with string-based args
The previous versions of cephadm supported supplying extra container and
entrypoint arguments for fine-tuning of services and creating custom
containers. However, this mode assumed that spaces in an argument
always meant that the argument should be split into two parts:
"--foo bar" becomes `["--foo", "bar"]`. In some cases there's a good
reason to keep spaces as in "--title=My Little Cluster". When
an argument is expressed as a single string the ArgumentSpec is
designed to retain the existing behavior. When an argument is
expressed as a JSON object then you can explicitly express if
you want the argument split on spaces or not (not split is the default).
The alternative was to keep using strings but add some level of
shell-style quoting. This was discussed but deemed complex and
difficult to read in YAML. Round tripping that data is also challenging.
The JSON object approach also allows for future fields to be added
providing for possible extensibility.
John Mulligan [Thu, 25 May 2023 23:17:02 +0000 (19:17 -0400)]
mgr/orchestrator: reformat ceph.deployment.service_spec import line
Quoting https://peps.python.org/pep-0008/#maximum-line-length
```
The preferred way of wrapping long lines is by using Python’s
implied line continuation inside parentheses, brackets and braces. Long
lines can be broken over multiple lines by wrapping expressions in
parentheses. These should be used in preference to using a backslash for
line continuation.
```
I also think it is much more readable and is similar to the style
applied by automated code formatting tools like `black`.
John Mulligan [Thu, 25 May 2023 22:54:05 +0000 (18:54 -0400)]
mgr/cephadm: reformat ceph.deployment.service_spec import line
Quoting https://peps.python.org/pep-0008/#maximum-line-length
```
The preferred way of wrapping long lines is by using Python’s
implied line continuation inside parentheses, brackets and braces. Long
lines can be broken over multiple lines by wrapping expressions in
parentheses. These should be used in preference to using a backslash for
line continuation.
```
I also think it is much more readable and is similar to the style
applied by automated code formatting tools like `black`.
John Mulligan [Fri, 26 May 2023 15:40:51 +0000 (11:40 -0400)]
python-common: reformat ceph.deployment.service_spec import line
Quoting https://peps.python.org/pep-0008/#maximum-line-length
```
The preferred way of wrapping long lines is by using Python’s
implied line continuation inside parentheses, brackets and braces. Long
lines can be broken over multiple lines by wrapping expressions in
parentheses. These should be used in preference to using a backslash for
line continuation.
```
I also think it is much more readable and is similar to the style
applied by automated code formatting tools like `black`.
John Mulligan [Wed, 17 May 2023 18:30:18 +0000 (14:30 -0400)]
python-common: add tests for service spec extra args handling
There were no existing tests for how the service spec(s) handled the
extra_container_args and extra_entrypoint_args values. Add a short
parametrized test function to assert some basic properties of how the
arguments are currently handled. In particular, it asserts that the
values can appear at the top-level of the YAML and under spec.
This is in preparation for adding a more sophisticated argument
type in the future.
John Mulligan [Mon, 19 Jun 2023 16:54:24 +0000 (12:54 -0400)]
cephadm: dont set ctx.image if json deploy image is unset
If no image has been provided in the "deploy from" json do not set
ctx.image to it (empty-string or None) as we may have had a valid
value passed on the --image CLI option.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 15 Jun 2023 19:54:09 +0000 (15:54 -0400)]
cephadm: remove call to get_parm from fetch_configs
Stop using get_parm in fetch_configs. Doing so makes clear that
the two if-branches in fetch_configs are symmetric, in the
handling of custom_config_files and symmetric with the behavior
of fetch_custom_config_files.
It also reveals that get_parm only has one remaining caller making it
simpler to remove get_parm in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 May 2023 20:23:13 +0000 (16:23 -0400)]
cephadm: combine deploy functions some more
During review it was pointed out that much of command_deploy and
command_deploy_from were still common. Combine the bulk of
command_deploy and command_deploy_from into a new _common_deploy.
The old _common_deploy is renamed to _dispatch_deploy but kept
as a separate function.
The reason for keeping _dispatch_deploy as a function is that
it limits the scope of variables. It helps knowing exactly what
is needed and what is not needed from _common_deploy.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 1 Jun 2023 17:27:37 +0000 (13:27 -0400)]
cephadm: drop deploy_arguments key from deploy config JSON
Drop the now unnecessary deploy_arguments subsection from the JSON
object that the new `_orch deploy` takes. The intermediate code that
used it in the ceph mgr module has been replaced with `params`.
We still want to share some code with the old deploy, especially the
default values for various things that get attached to `ctx`. In
some cases this is particularly important as not all code checks
that `ctx` has an attribute first. Create an `ArgumentFacade` type
to reuse the `add_argument` calls for deploy, and instead store the
default key-value pairs for `ctx`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 20 May 2023 17:42:02 +0000 (13:42 -0400)]
cephadm: split command_deploy_from into mockable parts
Split up command_deploy_from so that the part that reads the JSON
from an input source is moved into read_configuration_source and
the part that applied the configuration data to the ctx is moved
into apply_deploy_config_to_ctx. This will make these sections
reusable and, more importantly, make it easier to mock/monkeypatch
these functions in tests.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 18 May 2023 20:14:24 +0000 (16:14 -0400)]
cephadm: make the old command_deploy deprecated with decorator
Add a deprecated command decorator. Decorate the old command_deploy with
it. By default it just logs if a deprecated command was used, but
custom builds of cephadm can set the NO_DEPRECATED var to True
and fail if a deprecated command is used.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 18 May 2023 19:10:16 +0000 (15:10 -0400)]
mgr/cephadm: add exchange types and use them for deploy
Add an exchange module for defining the "exchange protocol" that the
mgr module uses for communicating with the cephadm binary. The
exchange module classes use data descriptors to define supported
fields in the resulting data and will be serialized to JSON for
communicating with cephadm.
The cephadm binary does not use these types because it currently
doesn't import anything outside of the standard library. A suggested
future change would be to move 'exhange.py' to python-common somewhere
and have the module be imported by both the mgr and cephadm and used
for both serialization and deserialization.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 16 May 2023 17:49:28 +0000 (13:49 -0400)]
mgr/cephadm: block future use of extra_args
Convert `extra_args` into a read-only property so that
current code continues to work but anything that tries
to set any `extra_args` now or in the future will be
intentionally broken.
This is mainly to find issues when running unit and teuthology tests.
This patch could be dropped before the final merge if desired.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 16 May 2023 17:48:58 +0000 (13:48 -0400)]
mgr/cephadm: convert extra_args to daemon_params dict
In serve.py we were creating various command line arguments
(and their values) to pass to cephadm. Now that we support
a single unified JSON-based input we can stop passing
these as CLI parameters and just pass it as a singular
JSONified dict.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 11 May 2023 15:07:53 +0000 (11:07 -0400)]
mgr/cephadm: deploy using deploy-from command
The new `deploy-from` command aims to be a cleaner, clearer way to pass
input to cephadm for service deployment. Make a minimal set of change to
use that new command.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 24 May 2023 18:11:52 +0000 (14:11 -0400)]
mgr/cephadm: allow passing a list as a compound command to _run_cephadm
As we're adding "namespaces" to the cephadm command it's easier to work
with a compound list of terms as the command. This also avoids an issue
where the function call injects args between the command and the "args"
passed to _run_cephadm.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 15 Jun 2023 19:34:15 +0000 (15:34 -0400)]
cephadm: update code to use fetch_custom_config_files
Update the code to use fetch_custom_config_files rather than
get_custom_config_files. Like other fetch_ functions recently added
to cephadm we don't LBYL for checking values on ctx, but rather
call the fetch function on ctx and then check if the result contains
actionable data.
get_custom_config_files gets removed as no remaining callers exist.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 15 Jun 2023 19:34:09 +0000 (15:34 -0400)]
cephadm: add fetch_custom_config_files function
This function works similarly to get_custom_config_files but doesn't
require the use of "ctx.config_json" and will prefer sourcing the
data from `config_blobs`.
This function also returns the list of dicts rather than return a dict
with exactly one key which maps to said list of dicts. There appeared
to be no benefit to that when the call sites of get_custom_config_files
was examined.
This function is `fetch_configs` evil twin.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 26 May 2023 19:06:40 +0000 (15:06 -0400)]
cephadm: add fetch_tcp_ports function for getting processed ports
The cephadm cli can pass ports as either a string containing space
separated port numbers or, in the newer method, a (JSON) list of ints.
The fetch_tcp_ports function abstracts away the handling of
these input forms.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 11 May 2023 15:05:07 +0000 (11:05 -0400)]
cephadm: add deploy-from command
Add the `deploy-from` command. This command is like `deploy` in
functionality but instead of sourcing input from a mixture of various
CLI arguments, CLI arguments that take JSON strings, and JSON data
provided on stdin - this command aims to take configuration from a
single unified JSON object provided on the commands stdin.
This change strives to be as reasonably backwards compatible with
`deploy` as possible and this leads to some awkward hoop-jumping,
including: needing to nest CLI arguments inside the JSON, because
that is what the manager constructs; duplicate sources of the fsid
(CLI and JSON); the strange behavior of CephadmContext when setting
values; and a few others I'm probably forgetting.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 11 May 2023 14:30:51 +0000 (10:30 -0400)]
cephadm: add fetch_configs function for reading configuration items from ctx
The CephadmContext holds much of cephadm's state, including values
from cli options and (soon) configuration file input. Add a
`fetch_configs` function that reads processed configuration items from `config_blobs`
(a dict mapping config name to arbitrary configuration content) or
falls back to using the `get_parm` function on `ctx.config_json`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 11 May 2023 14:30:12 +0000 (10:30 -0400)]
cephadm: add fetch_meta function for reading metadata properties from ctx
The CephadmContext holds much of cephadm's state, including values from
cli options and (soon) configuration file input. Add a `fetch_meta`
function that reads processed metadata from `meta_properties` or parses
JSON stored on `meta_json`. Parsed JSON is saved to `meta_properties`.
Future code should always use fetch_meta for reading, and only ever
set `meta_properties` as a python dict, not round trip JSON through
meta_json.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 11 May 2023 14:25:38 +0000 (10:25 -0400)]
cephadm: split default_image decorator into two functions
Keep default_image as a decorator for functions that will only
ever need to update an image passed by the CLI. For other future
functions that want to execute code prior to assigning an image
from CLI parameters add `update_default_image` which takes a
CephadmContext and updates it and can be used by the caller
at an arbitrary point in the code flow.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 1 Jun 2023 15:16:13 +0000 (11:16 -0400)]
cephadm: change tests to assert using mock call not func.args
The Python docs [1] note that:
Changed in version 3.8: Added args and kwargs properties.
In order to run tests on python 3.6 (example: `tox -e py36`) we
change the tests to do the asserts using the older call(...)
comparison style, with mock.ANY used for args we don't care about.
Adam King [Thu, 13 Apr 2023 17:54:00 +0000 (13:54 -0400)]
cephadm: open ports in firewall when adopting monitoring stack daemons
Otherwise we risk the prometheus/alertmanager/grafana
not functioning properly after adoption due to the necessary
port in the firewall not being open.
Adam King [Thu, 13 Apr 2023 17:05:11 +0000 (13:05 -0400)]
cephadm: still try to open ports in firewall on redeploy/reconfig
Prior to this patch we were discarding the provided
ports on reconfig and redeploy in order to not fail
thinking there was a port conflict with the instance
of the daemon we were about to reconfig/redeploy. However,
it's still desirable for us to make sure the firewall ports
are open when we do a reconfig/redpeloy, so this refactors
the port handling approach to have it do that but
still avoid checking for port conflicts. It also include
an update of the type signature of deploy_daemon
to the py3 style. That wasn't needed for the change
but since I was added an arugment there I thought we might
as well do it now.
John Mulligan [Tue, 6 Jun 2023 17:24:37 +0000 (13:24 -0400)]
cephadm: use 0o600 as the default mode for write_new
Add a constant DEFAULT_MODE of `0o600`, and make it the default of
the perms argument to write_new. This reduces a lot of code since
0o600 is the majority of the permissions used. Other cases can continue
to pass None to indicate no particular permissions are desired.
John Mulligan [Tue, 6 Jun 2023 17:16:29 +0000 (13:16 -0400)]
cephadm: convert SNMPGateway create_daemon_conf to use write_new
While it is not entirely clear why this pattern of using os.open and
posix open flags instead of `open` directly was used I determined (using
strace) that the only major difference between these open flags and
those used by `open` was the lack of O_TRUNC. Unlike some other cases
this function does not use an intermediate temporary file. This means
that if the file being written already exists and the data being written
is smaller then the remaining data will not be over-written.
I looked over the context that this function is used in and decided that
this behavior must not be intentional. Thus it should be safe
to convert this function to `write_new`.
projects / ceph.git / log