Nizamudeen A [Tue, 15 Jun 2021 08:47:58 +0000 (14:17 +0530)]
mgr/dashboard: Fix 500 error while exiting out of maintenance
When you add a host in maintenance mode and then exit the maintenance
mode, a 500 server error will popup which will interrupt the whole
exit maintenance process and leave the host in an unknown/offline state.
It happened when I was setting the status of the host through the
HostSpec(). With this change, I am using the enter_maintenance api of
the orch to enable the maintenance.
Conflicts:
src/pybind/mgr/dashboard/controllers/host.py
- Removed the addr and label variables since its not backported yet to
pacific
- Removed Versioning since for pacific this endpoint is not yet
versioned to 0.1
src/pybind/mgr/dashboard/services/orchestrator.py
- Removed the addr and label variables since its not backported yet to
pacific
src/pybind/mgr/dashboard/frontend/src/app/shared/api/host.service.ts
- Didn't included this file because the changes in there are irrelevant
in pacific
src/pybind/mgr/dashboard/openapi.yaml
- Regenerated the openapi file
Kefu Chai [Fri, 4 Jun 2021 03:25:12 +0000 (11:25 +0800)]
debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-rook anymore
per https://www.debian.org/doc/debian-policy/ch-relationships.html
> Recommends
> This declares a strong, but not absolute, dependency.
>
> The Recommends field should list packages that would be found together
> with this one in all but unusual installations.
ceph-mgr-modules-core provides a set of ceph-mgr modules which are
always enabeld. but the rook module enables ceph-mgr to install and
configure a Ceph cluster using Rook. this module is very useful but
it does not have such a strong connection with ceph-mgr-modules-core.
we can always install it separately for using better intergration with
Rook.
Tatjana Dehler [Thu, 27 May 2021 09:46:50 +0000 (11:46 +0200)]
mgr/dashboard: show partially deleted RBDs
An RBD might be partially deleted if the deletion
process has been started but was interrupted. In
this case return the RBD as part of the RBD list
and mark it as partially deleted.
Fixes: https://tracker.ceph.com/issues/48603 Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit d83c277ac1861df31d2a39d16e20c7bebbea676e)
Conflicts:
src/pybind/mgr/dashboard/services/rbd.py
Fixed minor imports conflict.
Kamoltat [Thu, 13 May 2021 17:38:10 +0000 (17:38 +0000)]
pybind/mg/progress: Disregard unreported pgs
The global recovery event progress calculations only
takes into account pgs with `reported_epoch < start_epoch_of_event`
but sometimes the pgs doesn't get move before or after the creation
of the global recovery event, therefore this might result in a bug
where the global event gets stuck forever unless there is another
event that specifically makes the pgs that get stuck moves and updates
its `reported_epoch`.
Therefore, we decided to disregard pgs that are in active+clean state
but has `reported_epoch < start_epoch_of_event`.
rgw: radosgw_admin remove bucket not purging past 1,000 objects
The zipper refactoring combines use of old is_truncated variable and
newer structure that manages is_truncated. This prevented advancing
past the first page of 1,000 entries.
Cory Snyder [Fri, 28 May 2021 19:08:49 +0000 (15:08 -0400)]
mgr/DaemonServer.cc: prevent integer underflow that is triggered by large increases to pg_num/pgp_num
This fixes a scenario where mgrs continually crash while attempting to apply large increases to pg_num/pgp_num. The max step size (estmax) for each incremental update to the pgp_num is calculated as a percentage of the pg_num, which permits the possibility for the max step size (estmax) to be greater than the current pgp_num when the increase is large; this causes an integer underflow when the max step size is subtracted from the pgp_num in order to calculate the next step size with std::clamp. The integer underflow causes hi < lo in args passed to std::clamp, which causes a failed assertion, SIGABRT, and ultimately crashing mgr.
Zac Dover [Wed, 2 Jun 2021 14:11:59 +0000 (00:11 +1000)]
doc/cephadm: enriching "setting a limit"
This PR improves the flow and syntax of the
sentences in the "Setting a Limit" section of the
"Service Management" chapter of the cephadm documentation.
(This section is about setting limits to the number of
daemons created, and about limiting the creation of daemons
to a specified number of hosts.)
Zac Dover [Sun, 9 May 2021 15:06:02 +0000 (01:06 +1000)]
doc/cephadm: rewrite "deploy. mon. w/cephadm" 2/2
This is the second half of PR#41241.
This PR creates a sub-subsection that explains
a configuration step that is sometimes necessary
when configuring monitoring in Ceph clusters. (The
configuration step involves DNS zones.)
Zac Dover [Sun, 9 May 2021 14:39:47 +0000 (00:39 +1000)]
doc/cephadm: rewrite "deploy. mon. w/cephadm"
This PR corrects various minor usage errors in
the subsection "Deploying Monitoring with cephadm"
of the "monitoring" section of the cephadm guide.
It also adds prompts, and cleans up the list rst a
bit.
This is the first of two PRs planned for this section.
The second PR will create a sub-subsection that explains
an additional installation step that is necessary when
a Ceph cluster and a user accessing that cluster are in
different DNS zones. But that's another story, for another
PR.
mgr/dashboard: simplify object locking fields in 'Bucket Creation' form
- UI: The object locking retention period must be expressed in days.
Years value from API (as API keeps param parity with RGW API) are converted to days.
- API: add more validation and increase test coverage.
Fixes: https://tracker.ceph.com/issues/49885 Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 09394a19913b8159a7608a594e29886eaae7570e)
After mkfs the store may not yet contain monmap:last_committed but
might be respawning after setting mon_sync:temp_newer_monmap.
Load that stashed map before falling back to the mkfs:monmap.
Fixes: https://tracker.ceph.com/issues/50230 Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit cc0b4c77753962717da8a280a585990f7eec3c7b)
rgw: completion of multipart upload leaves delete marker
The multipart upload meta object is deleted when the multipart upload
is completed. When the bucket is versioned, it needs to be deleted
from the bucket index rather than go through versioning delete process
that adds a delete marker to the bucket index rather than simply
removing it from the bucket index.
Jonas Jelten [Mon, 15 Mar 2021 22:21:07 +0000 (23:21 +0100)]
os/bluestore: strip trailing slash for directory listings
Calls to BlueRocksEnv::GetChildren may contain a trailing / in the
queried directory, which is stripped away with this patch.
If it's not stripped, the directory entry is not found in BlueFS:
```
10 bluefs readdir db/
20 bluefs readdir dir db/ not found
3 rocksdb: [db/db_impl/db_impl_open.cc:1785] Persisting Option File error: OK
```
Kefu Chai [Tue, 16 Mar 2021 01:38:41 +0000 (09:38 +0800)]
os/bluestore: use string_view in BlueFS
this improves the performance of lookup. when it comes to mutation
operations, std::map<> always require an instance of string, but
it only require a single instance of string for each mutation operation
in general, so this does not incurs performance regression due to
multiple copies of the same string_view object.
Kefu Chai [Tue, 16 Mar 2021 01:13:00 +0000 (09:13 +0800)]
os/bluestore: move BlueRocksEnv::split() to .cc
this helper is only used by the functions in the .cc file, and it does
not reference BlueRocksEnv member variable or methods. so move it to
an anonymous namespace.
Sage Weil [Mon, 7 Jun 2021 18:44:27 +0000 (13:44 -0500)]
Merge PR #41684 into pacific
* refs/pull/41684/head:
cephadm: improve is_container_running()
mgr/cephadm:fix alerts sent to wrong URL
mgr/cephadm: Warn about OSDs to be deleted manually when deleting an OSD service
doc: add ceph-nfs link
mgr: Fix orch osd rm stop help message
mgr/cephadm/inventory: do not try to resolve current mgr host
pybind/mgr/mgr_module: make get_mgr_ip() return mgr's IP from mgrmap
mgr/restful: use get_mgr_ip() instead of hostname
doc/cephadm: s/the the/the
doc/cephadm: enriching "daemon status"
doc/cephadm: enriching "Service Specification"
mgr/cephadm: Don't call _check_host without hosts
cephadm: stop passing --no-hosts to podman
mgr/nfs: use host.addr for backend IP where possible
mgr/cephadm: convert host addr if non-IP to IP
mgr/dashboard,prometheus: new method of getting mgr IP
doc/cephadm: remove any reference to the use of DNS or /etc/hosts
mgr/cephadm: use known host addr
mgr/cephadm: resolve IP at 'orch host add' time
doc/cephadm: enrich "service status"
doc/cephadm: recommend redeploying monitoring stack daemon after changing image
cephadm: clean-up error message
cephadm: raise an error when `--config` file is not found
cephadm: manage cephadm log with logrotated
mgr/cephadm: progress item for service apply
mgr: expose ceph.conf path to modules
common/config: track the path to the conf file we loaded
qa/tasks/cephfs/test_nfs: fix info test
doc/cephfs/fs-nfs-exports: document --ingress --virtual-ip
mgr/nfs: move ingress vs virtual_ip check to cluster interface
PendingReleaseNotes: clarify deprecated
PendingReleaseNotes: note breaking CLI changes
doc/cephadm/nfs: document nfs+ingress
qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
mgr/nfs: take --ingress argument to 'nfs cluster create'
mgr/cephadm: adjust debug output for device refresh
mgr/cephadm: ingress: fix log msg
mgr/cephadm: fix logging of config/placement errors
common/options: enable nfs module for new clusters
cephadm: --stop-signal=SIGTERM
mgr/orchestrator: default nfs pool, namespaces
mgr/cephadm: nfs: create pool if it doesn't yet exist
doc/cephadm/nfs: update
mgr/nfs: change 'nfs cluster info'
mgr/nfs: take optional virtual_ip for deploying ingress
mgr/nfs: remove 'nfs cluster update'
mgr/nfs: factor out ganesha pool creation
mgr/nfs: delete -> rm for CLI
mgr/nfs: add some type annotations
python-common: fix IngressSpec yaml dump
mgr/cephadm: ingress: remove eth0 default
qa/tasks/cephadm: allow mounting volumes in shell
cephadm: add -v arg to shell
qa/tasks/vip: add 'vip.exec' task
mgr/orchestrator: add --port arg to 'orch apply nfs'
mgr/cephadm: nfs: add purge
mgr/cephadm: ingress: support nfs
mgr/cephadm: do not reconfigure daemons on deleted services
mgr/cephadm: nfs: shell out to rados tool for conf creation
mgr/cephadm: nfs: add rank to grace file from mgr module
mgr/cephadm: nfs: bind ganesha to appropriate ip:port
mgr/cephadm: enable ranked daemons for nfs
mgr/cephadm: support creation of daemons with ranks
mgr/cephadm: make _plan show removed daemon names
mgr/cephadm/schedule: assign/map ranks
mgr/cephadm: add rank[_generation] properties
mgr/cephadm/inventory: store optional rank_map along with specs
mgr/cephadm: include service_name is generated DaemonDescription
mgr/orchestrator: include service_name in DaemonDescription dump
mgr/cephadm/inventory: fix deleted check
mgr/cephadm: simplify
mgr/cephadm/schedule: make placement shuffle deterministic
mgr/cephadm: document CephadmService flags
doc/cephadm: fix prompts in service-management.rst
qa/tasks/cephadm: Include bootstrap registry options for downstream
qa/suites/rados: include rook test in rados
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com> Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Paul Cuzner [Wed, 2 Jun 2021 23:34:19 +0000 (11:34 +1200)]
mgr/cephadm:fix alerts sent to wrong URL
The path_prefix in prometheus.yml was specifying an
endpoint prefix, which was invalid. This resulted in 404
errors when trying to send alerts to alertmanager and
blocked alerts being sent on to the ceph-dashboard API
receiver. This fix remves this prefix.
Fixes: https://tracker.ceph.com/issues/51073 Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 9d408a70c7d01fd7c94f9b814af916396d7cbf1f)
Sage Weil [Thu, 3 Jun 2021 14:29:00 +0000 (10:29 -0400)]
mgr/cephadm/inventory: do not try to resolve current mgr host
The CNI configuration may set up a private network for the container, which
is mapped to the hostname in /etc/hosts. For example, my test box sets
up 10.88.0.0/24 because I was using crio + kubeadm on this host earlier
(at least I think that's why):
In any case, we should never trust a lookup of our own hostname from inside
a container!
This isn't quite sufficient, though: if this is a single-host cluster, then
we fall back to using get_mgr_ip(). That value may be distorted by the
public_network option on the mgr, but we don't have any other good
options here, and single-node clusters are unlikely to have complex
network configs.
Sage Weil [Wed, 2 Jun 2021 02:31:11 +0000 (22:31 -0400)]
pybind/mgr/mgr_module: make get_mgr_ip() return mgr's IP from mgrmap
The previous approach was convoluted: we tried to do a DNS lookup on the
hostname, which would fail if /etc/hosts had an entry. Which, with podman,
it does. And the IP it has will vary in all sorts of weird ways. For
example, CNI on my host means that I get a dynamic address in 10.88.0.0/24.
Avoid all of that nonsense and use the IP that is in the mgrmap. There
may be multiple IPs (v2 + v1, or maybe even IPv4 + v6 in the future); in
that case, use the first one.
In 7f047005fc72e1f37a45cde2d742bb2eb1e62881, we made the pg removal code
much more efficient. But it started marking the pgmeta object as an unexpected
onode, which in reality is expected to be removed after all the other objects.
This behavior is very easily reproducible in a vstart cluster:
ceph osd pool create test 1 1
rados -p test bench 10 write --no-cleanup
ceph osd pool delete test test --yes-i-really-really-mean-it
Before this patch:
"do_delete_work additional unexpected onode list (new onodes has appeared
since PG removal started[#2:00000000::::head#]" seen in the OSD logs.
After this patch:
"do_delete_work removing pgmeta object #2:00000000::::head#" is seen.
Related to:https://tracker.ceph.com/issues/50466 Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 0e917f1b1e18ca9e48b3f91110d3a46b086f7d83)
Aashish Sharma [Tue, 1 Jun 2021 05:09:24 +0000 (10:39 +0530)]
mgr/dashboard: API Version changes do not apply to pre-defined methods (list, create etc.)
Methods like list(), create(), get() etc doesn't get applied the version.Also for the endpoints that get the version changed, the docs and the request header has still the version v1.0+ in them. So with the version reduced it gives 415 error when trying to make the request. This PR fixes this issue.
Aashish Sharma [Wed, 19 May 2021 05:57:13 +0000 (11:27 +0530)]
mgr/dashboard: API Version changes doesnt appy to pre-defined methods(list, create etc.)
Methods like list(), create(), get() etc doesn't get applied the version.Also for the endpoints that get the version changed, the docs and the request header has still the version v1.0+ in them. So with the version reduced it gives 415 error when trying to make the request. This PR fixes this issue.
Conflicts:
src/pybind/mgr/dashboard/controllers/__init__.py(resolved conflicts due to https://github.com/ceph/ceph/pull/40063)
src/pybind/mgr/dashboard/tests/test_docs.py(removed unused import DEFAULT_VERSION)
projects / ceph.git / log