Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit de448fafe0fe7d6889f3f75ac37f870cde00cf1c) Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Jason Dillaman [Wed, 15 Apr 2020 20:27:07 +0000 (16:27 -0400)]
qa/workunits/rbd: retry the addition of a mirror pool peer
We might race with the remote rbd-mirror daemon creating a
tx-only peer when adding a new peer. Therefore, delete the
tx-only peer and attempt to re-create it.
Fixes: https://tracker.ceph.com/issues/44938 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4311f597a98b6870d7895e6403fb32356bfbe9)
mgr/rook: Fix wrong data output when listing devices
Information used to build the <device ls> output is not the right one.
Rook provides in the devices configmap the item <cephVolumeData> which contains the information returned by <ceph-volume>.
This is the right source information to build the devices list.
Michael Fritch [Wed, 8 Apr 2020 20:42:12 +0000 (14:42 -0600)]
cephadm: show error when no command is specified
Traceback (most recent call last):
File "./cephadm", line 4452, in <module>
if args.func != command_check_host:
AttributeError: 'Namespace' object has no attribute 'func'
Matthew Oliver [Thu, 2 Apr 2020 23:22:02 +0000 (23:22 +0000)]
tox: Fix the tox.ini's to support older versions of tox
The src/cephadm/tox.ini and src/pybind/mgr/tox.ini both don't run
on older versions of tox.
When using tox 2.9.1 both fail for different reasons.
`src/cephadm/tox.ini` fails because `skipsdist=true` only works if it's
directly under the `[tox]` section.
`src/pybind/mgr/tox.ini` fails because older versions of tox can't find
the requirements.txt because they don't like whitespace between the `-r`
and `requirements.txt`.
This patch changes the tox.ini's to be backwards compatible for those
who happen to be running slightly older version of tox.
Matthew Oliver [Thu, 5 Mar 2020 02:55:47 +0000 (13:55 +1100)]
cephadm: ceph-iscsi first draft
This if the first draft of the ceph-iscsi in cephadm.
There are a few gotchas when running `rbd-target-api` in a container:
1. We need both the ceph.conf and iscsi-gateway.cfg, so needed to
ability to pass extra config. This latter is based off the spec, so now
the daemon config func api allows you to return a dict of configs:
{ 'config': '<str>' # will be appended to the ceph.conf
'<conf name>': 'str', # Will be dumped in datadir/<conf name>
...
}
It will be up to cephadm to know to bind mount it to the right location.
The first 'config' isn't used by this patch, but makes it possible for
specs or config funcs to append anything? maybe it's overkill.
2. We need the kernel's configfs in the container so we can configure
LIO. There is a chicken and egg problem, configfs isn't mounted on the
host to bind mount when the container starts. So now a check is added to
the `unit.run` and cleanup in the `unit.poststop` scripts for
daemon_type iscsi.
3. rbd-target-api is python and hardcodes a few things, like logging
through `/dev/log` which happens to be a domain socket. So `/dev/log`
also needed to be bind mounted into the continer.
4. The daemon expects the keyring to be in `/etc/ceph` so this needed to
be specifically bind mounted to the correct location too.
As this currently stands this is deploying and starting the api on port
5000, so seems to be "working", also gateway.conf does exist in the
pool. I have yet to set up an iscsi device, but will test that next.
The `rbd-target-api` daemon expects the ssl key and cert to be named a
certain name in the contianer. So SSL isn't working yet. However, I do
hav a PR in ceph-iscsi to look in the mon config-key store for them[0].
dashboard: Resolve FQDN / hostname mismatch in hosts overview panel
In the AVG Disk Utilization panel, the result is calculated
by combining the output of node_disk_io_time_seconds_total
with the output of ceph_disk_occupation. However, the
first vector encodes the instance label with the full FQDN
while the ceph label only contains the hostname:port. In
order for these to match correctly, the domain name and port
has to be stripped from the labels.
When moving to LVM-based ceph-volume setups, several
grafana dashboards stopped working. The problem is that
(device, instance) no longer results in unique labels
which causes errors like:
"many-to-many matching not allowed: matching labels must be unique on one side"
The references to `$osd_hosts` etc. were encoded as
`[[osd_hosts]]` in the PromQL expression divisor, and
the panel always displayed N/A as the result of the
query.
Replacing the `[[...]]` with `$...` makes the expression
work again.
Robin H. Johnson [Fri, 27 Mar 2020 19:48:13 +0000 (20:48 +0100)]
rgw: reject control characters in response-header actions
S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.
Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com> Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com> Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)
Ilya Dryomov [Fri, 6 Mar 2020 19:16:45 +0000 (20:16 +0100)]
msg/async/crypto_onwire: fix endianness of nonce_t
As a AES-GCM IV, nonce_t is implicitly shared between server and
client. Currently, if their endianness doesn't match, they are unable
to communicate in secure mode because each gets its own idea of what
the next nonce should be after the counter is incremented.
Several RFCs state that the nonce counter should be BE, but since we
use LE for everything on-disk and on-wire, make it LE.
The secure mode uses AES-128-GCM with 96-bit nonces consisting of a
32-bit counter followed by a 64-bit salt. The counter is incremented
after processing each frame, the salt is fixed for the duration of
the session. Both are initialized from the session key generated
during session negotiation, so the counter starts with essentially
a random value. It is allowed to wrap, and, after 2**32 frames, it
repeats, resulting in nonce reuse (the actual sequence numbers that
the messenger works with are 64-bit, so the session continues on).
Because of how GCM works, this completely breaks both confidentiality
and integrity aspects of the secure mode. A single nonce reuse reveals
the XOR of two plaintexts and almost completely reveals the subkey
used for producing authentication tags. After a few nonces get used
twice, all confidentiality and integrity goes out the window and the
attacker can potentially encrypt-authenticate plaintext of their
choice.
We can't easily change the nonce format to extend the counter to
64 bits (and possibly XOR it with a longer salt). Instead, just
remember the initial nonce and cut the session before it repeats,
forcing renegotiation.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com> Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com> Reviewed-by: Sage Weil <sage@redhat.com>
Conflicts:
src/msg/async/ProtocolV2.h [ context: commit ed3ec4c01d17
("msg: Build target 'common' without using namespace in
headers") not in octopus ]
> mgr_util: add CephfsClient implementation
>
> This pulls parts of the VolumesClient implementation into mgr_util to
> make the CephFS specific pieces available to other mgr modules. To
> reduce code duplication the VolumeClient now extends the CephfsClient
> class to add the volume specific methods.
> Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Sage Weil [Thu, 2 Apr 2020 23:36:39 +0000 (18:36 -0500)]
cephadm: create /var/run/ceph dir via unit.run, not unit file
The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.
Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.
Daniel-Pivonka [Fri, 27 Mar 2020 18:38:41 +0000 (14:38 -0400)]
mgr/cephadm: add useful error if python3 is not on remote host
show 'Error ENOENT: New host example (example) failed check: ["Can't communicate with
remote host, possibly because python3 is not installed there"]' instead of traceback
with OSError: cannot send(already closed?) when adding host if python3 is not on host
Sage Weil [Fri, 27 Mar 2020 15:39:09 +0000 (10:39 -0500)]
qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes
Revert part of 96220c0c0574eb5b896023e1552f528bef9e1ca5 so that we still
distribute a *final* ceph.conf and admin keyring to all nodes, right after
all of the mons are up.
projects / ceph.git / log