Nathan Cutler [Tue, 27 Oct 2020 20:40:38 +0000 (21:40 +0100)]
doc/PendingReleaseNotes: clean up for 15.2.6
This commit drops release notes that have already been published and
organizes the remaining release notes under a heading so it is clear
they are targeting the 15.2.6 release.
Conflicts:
src/test/mon/MonMap.cc
- do not attempt to introduce boost::intrusive_ptr into Nautilus
- monmap.build_initial takes bare cct in nautilus (master: cct.get())
Conflicts:
src/test/mon/MonMap.cc
- do not attempt to introduce boost::intrusive_ptr into nautilus
- monmap.build_initial takes bare cct in nautilus (master: cct.get())
Jason Dillaman [Tue, 13 Oct 2020 01:34:25 +0000 (21:34 -0400)]
librbd: update AioCompletion return value before evaluating pending count
If the pending count is decremented before the return value is updated,
there is a possibility of two ASIO threads concurrently decrementing the
pending count down from 2 -> 1 -> 0. In the second thread (the one that
performs the final decrement from 1 -> 0), it can finalize the completion
before the first thread has had a chance to update the return value.
Fixes: https://tracker.ceph.com/issues/47847 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 94f3bce53c39017028ce44a80697f55af2a82e68)
Jason Dillaman [Fri, 16 Oct 2020 15:25:39 +0000 (11:25 -0400)]
journal: possible race condition between flush and append callback
When notifying the journal recorder of an overflow or if the object
close request has completed due to no more in-flight IO, it was
possible for a race between a flush request and the processing of
an append completion to attempt to kick off duplicate notifications.
Since the overflowed and closed callbacks are properly protected from
duplicates, use a counter instead of a boolean to track possible
in-flight handler callbacks.
Fixes: https://tracker.ceph.com/issues/47880 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 458ab997fe77ea78803a34c6c9715225aa3413ba)
KeyCount should return object count + common prefix count.
see S3 example: https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#API_ListObjectsV2_Example_5
rgw/gc: fix for incrementing the perf counter 'gc_retire_object'
in the new gc queue code for omap offload, when gc objects from queue
are deleted. This was missed out initially.
rgw/gc: fixing the condition when marker for a queue is
always reset to empty which causes RGWGC::list to get stuck in
a loop, which ultimately is broken out of when the queue's truncated
flag is false.
1. Check for entries size also while evaluating whether objects cache for
a gc object should be marked as 'transitioned' in case of cls_rgw_gc_list.
When there are no entries, we get back a return value of 0, and the
object cache is not marked as 'transitioned'.
2. Also for the last gc object, we need to check whether the queue is still
under process and set the correct flag.
Missing the two conditions above causes the GC::list to loop continously
over the same gc object.
qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap
Previously it still could race when unmap_device returned success
because the device was not found in `rbd-nbd list-mapped` (the nbd
device was removed) but the test failed because the process was still
found in the ps table.
rgw: radosgw-admin should paginate internally when listing bucket
Currently `radosgw-admin bucket list ...`, when listing a bucket, asks
for the value of "--max-entries" internally. To list a large bucket
entirely the user would have to set "--max-entries" to a large value
(e.g., 10000000). Internally this doesn't paginate, so it will try to
produce the entire list at once. This can consume a lot of memory, and
there are known cases where this induces an out-of-memory crash.
So now we'll set a maximum pagination size of 10,000. So even with
large values of "--max-entries" it will still be able to produce the
full listing without stressing memory, because it will ask for at most
10,000 entries at a time.
Or Friedmann [Thu, 23 Jul 2020 15:36:07 +0000 (18:36 +0300)]
rgw: fix expiration header returned even if there is only one tag in the object the same as the rule
Expiration header returned even if there is only one tag in the object the same as the rule
Signed-off-by: Or Friedmann <ofriedma@redhat.com> Reported-by: Avi Mor <avmor@redhat.com> Fixes: https://tracker.ceph.com/issues/46614
(cherry picked from commit bf7c7e59f390afb53cb1e30a440ab26bb093c11c)
rgw: rgw-orphan-list should use "plain" formatted `rados ls` output
The previous version that used "json-pretty" output for `rados ls`
added complications due to json's escaping of special characters. So
this version returns to the "plain" output for `rados ls` but deals
with entries (oids) that might have namespaces and/or locators as
well.
rgw: allow rgw-orphan-list to note when rados objects are in namespace
Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.
However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.
Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/shared/smart-list/smart-list.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/shared/smart-list/smart-list.component.ts
- Use ngx-bootstrap tabset for tabs.
Patrick Donnelly [Tue, 13 Oct 2020 17:09:41 +0000 (10:09 -0700)]
qa: set rados op timeouts for mds/ceph-fuse
Now that the osdc Objecter obeys updates to these configs, let's use
them to avoid having them block forever on operations that may never
complete (or should complete in a timely manner).
Fixes: https://tracker.ceph.com/issues/47734 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d060c9a768c6974d3b68e4d408edf78bba9e0e85)
Have the Objecter track the rados_(mon|osd)_op_timeout configs so that
it can be configured at runtime/startup. This is useful for the
MDS/ceph-fuse so that we can avoid waiting forever for a response from
the Monitors that will never come (statfs on a deleted file system's
pools).
Also: make these configs take a time value rather than double. This is
simpler to deal with in the code and allows time units to be used (e.g.
"5m" for 5 minutes).
Fixes: https://tracker.ceph.com/issues/47734 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a8a23747aa081d938c9b277ab42507dd506bf6c2)
Jan Fajerski [Tue, 20 Oct 2020 13:05:50 +0000 (15:05 +0200)]
Merge PR #37520 into octopus
* refs/pull/37520/head:
ceph-volume: don't exit before empty report can be printed
PendingReleaseNotes: add note about batch refactor
pybind/mgr/orchestrator: fix preview for new c-v batch output
ceph-volume batch: return valid empty json reports
python-common: do not skip unavailable devices
ceph-volume: pass filter_for_batch as keyword argument
ceph-volume batch: fix very_fast_allocation plan and add tests
ceph-volume: batch: call the right prepare method
python-common drive-groups: allow strings for wal and db size
ceph-volume inventory: add option to filter unwanted devices
ceph-volume: address review comments
ceph-volume: batch: fix size retrieval for lvs
ceph-volume: include encryption in batch report
ceph-volume lvm batch: use namedtuple instead of tuple
ceph-volume: address review comments, mostly tidying, clarification
idempotency must result in the same outcome
ceph-volume: batch test should pass --journal-devices with filestore
ceph-volume: make --journal optional, add --journal-slots
doc: update ceph-volume lvm batch docs
py-common/deployment: drop workarounds for old batch
ceph-volume batch: add deprecation warning for auto behaviour
ceph-volume batch: add ceph.conf mocking to pass tests
ceph-volume batch: use disk.Size for size args
ceph-volume batch: Fix osd_ids passing and improve plan formatting
ceph-volume batch: track rel_size in percent, more tests
ceph-volume batch: improve backwards compatibility
ceph-volume: batch - enable legacy auto behaviour
ceph-volume: batch - major refactor
ceph-volume: Device - available_lvm if 10 extents are free.
ceph-volume: Device - add vg_free property
ceph-volume: prepare/create - size args as Size class
ceph-volume: disk.Size - add cast to bool
ceph-volume: api/lvm - add VolumeGroup.free_percent property
ceph-volume: util.device - add vg_free_percent property
ceph-volume: api/lvm - query LV units in bytes
ceph-volume: lvm/common - refactor common arg specification
ceph-volume: batch: fix argument help message
Sebastian Krah [Fri, 2 Oct 2020 09:07:40 +0000 (11:07 +0200)]
mgr/dashboard: Show warning when replicated size is 1
Issues a warning when creating a replicated pool and the replicated size
is 1. It won't stop the user from creating the pool, but will give the
user a hint that no replication will be created.
Fixes: https://tracker.ceph.com/issues/42404 Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit af1a7dbfe2b0e2e40650d64d4c3247f4961743c2)
Conflicts:
src/pybind/mgr/dashboard/frontend/src/styles/defaults/_bootstrap-defaults.scss
- The file does not exist in octopus, moving a slightly changed version of the diff
to src/pybind/mgr/dashboard/frontend/src/styles.scss instead
Conflicts:
doc/mgr/dashboard.rst
Fixed conflict because https://github.com/ceph/ceph/commit/d6369ac159a759b00769c0af34131ecba422e8d5
has not been backported to octopus.
mgr/dashboard: Enable per RBD graphs in details of RBD
Fixes: https://tracker.ceph.com/issues/47433 Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3d3230b99bf075020256bdc7cf082fd54fc61992)
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.ts
Fixed conflicts because https://github.com/ceph/ceph/pull/35290,
https://github.com/ceph/ceph/pull/35311 and https://github.com/ceph/ceph/pull/35574
have not been backported to octopus.
The "ceph fs volume create" command takes a placement string,
just like "ceph nfs cluster create" does. Align the documentation
so the placement string is explained in the same manner in both
places.
mgr/dashboard: fix error when typing existing paths in the Ganesha form
- The `CephFS.ls_dir()` implementation in the backend had changed, the code in the
UI endpoint `/ui-api/nfs-ganesha/lsdir` needs to adapt.
- Add fs_name as resource_id in `/ui-api/nfs-ganesha/lsdir/<fs_name>` to distinguish FS.
- Add more checks and unit tests.
Conflicts:
src/pybind/mgr/dashboard/controllers/nfsganesha.py
src/pybind/mgr/dashboard/tests/test_ganesha.py
- Conflict due to import order changed in the master
projects / ceph.git / log