qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap
Previously it still could race when unmap_device returned success
because the device was not found in `rbd-nbd list-mapped` (the nbd
device was removed) but the test failed because the process was still
found in the ps table.
rgw: radosgw-admin should paginate internally when listing bucket
Currently `radosgw-admin bucket list ...`, when listing a bucket, asks
for the value of "--max-entries" internally. To list a large bucket
entirely the user would have to set "--max-entries" to a large value
(e.g., 10000000). Internally this doesn't paginate, so it will try to
produce the entire list at once. This can consume a lot of memory, and
there are known cases where this induces an out-of-memory crash.
So now we'll set a maximum pagination size of 10,000. So even with
large values of "--max-entries" it will still be able to produce the
full listing without stressing memory, because it will ask for at most
10,000 entries at a time.
Or Friedmann [Thu, 23 Jul 2020 15:36:07 +0000 (18:36 +0300)]
rgw: fix expiration header returned even if there is only one tag in the object the same as the rule
Expiration header returned even if there is only one tag in the object the same as the rule
Signed-off-by: Or Friedmann <ofriedma@redhat.com> Reported-by: Avi Mor <avmor@redhat.com> Fixes: https://tracker.ceph.com/issues/46614
(cherry picked from commit bf7c7e59f390afb53cb1e30a440ab26bb093c11c)
rgw: rgw-orphan-list should use "plain" formatted `rados ls` output
The previous version that used "json-pretty" output for `rados ls`
added complications due to json's escaping of special characters. So
this version returns to the "plain" output for `rados ls` but deals
with entries (oids) that might have namespaces and/or locators as
well.
rgw: allow rgw-orphan-list to note when rados objects are in namespace
Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.
However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.
Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.
Jan Fajerski [Tue, 20 Oct 2020 13:05:50 +0000 (15:05 +0200)]
Merge PR #37520 into octopus
* refs/pull/37520/head:
ceph-volume: don't exit before empty report can be printed
PendingReleaseNotes: add note about batch refactor
pybind/mgr/orchestrator: fix preview for new c-v batch output
ceph-volume batch: return valid empty json reports
python-common: do not skip unavailable devices
ceph-volume: pass filter_for_batch as keyword argument
ceph-volume batch: fix very_fast_allocation plan and add tests
ceph-volume: batch: call the right prepare method
python-common drive-groups: allow strings for wal and db size
ceph-volume inventory: add option to filter unwanted devices
ceph-volume: address review comments
ceph-volume: batch: fix size retrieval for lvs
ceph-volume: include encryption in batch report
ceph-volume lvm batch: use namedtuple instead of tuple
ceph-volume: address review comments, mostly tidying, clarification
idempotency must result in the same outcome
ceph-volume: batch test should pass --journal-devices with filestore
ceph-volume: make --journal optional, add --journal-slots
doc: update ceph-volume lvm batch docs
py-common/deployment: drop workarounds for old batch
ceph-volume batch: add deprecation warning for auto behaviour
ceph-volume batch: add ceph.conf mocking to pass tests
ceph-volume batch: use disk.Size for size args
ceph-volume batch: Fix osd_ids passing and improve plan formatting
ceph-volume batch: track rel_size in percent, more tests
ceph-volume batch: improve backwards compatibility
ceph-volume: batch - enable legacy auto behaviour
ceph-volume: batch - major refactor
ceph-volume: Device - available_lvm if 10 extents are free.
ceph-volume: Device - add vg_free property
ceph-volume: prepare/create - size args as Size class
ceph-volume: disk.Size - add cast to bool
ceph-volume: api/lvm - add VolumeGroup.free_percent property
ceph-volume: util.device - add vg_free_percent property
ceph-volume: api/lvm - query LV units in bytes
ceph-volume: lvm/common - refactor common arg specification
ceph-volume: batch: fix argument help message
Sebastian Krah [Fri, 2 Oct 2020 09:07:40 +0000 (11:07 +0200)]
mgr/dashboard: Show warning when replicated size is 1
Issues a warning when creating a replicated pool and the replicated size
is 1. It won't stop the user from creating the pool, but will give the
user a hint that no replication will be created.
Fixes: https://tracker.ceph.com/issues/42404 Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit af1a7dbfe2b0e2e40650d64d4c3247f4961743c2)
Conflicts:
src/pybind/mgr/dashboard/frontend/src/styles/defaults/_bootstrap-defaults.scss
- The file does not exist in octopus, moving a slightly changed version of the diff
to src/pybind/mgr/dashboard/frontend/src/styles.scss instead
Conflicts:
doc/mgr/dashboard.rst
Fixed conflict because https://github.com/ceph/ceph/commit/d6369ac159a759b00769c0af34131ecba422e8d5
has not been backported to octopus.
mgr/dashboard: Enable per RBD graphs in details of RBD
Fixes: https://tracker.ceph.com/issues/47433 Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3d3230b99bf075020256bdc7cf082fd54fc61992)
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.ts
Fixed conflicts because https://github.com/ceph/ceph/pull/35290,
https://github.com/ceph/ceph/pull/35311 and https://github.com/ceph/ceph/pull/35574
have not been backported to octopus.
The "ceph fs volume create" command takes a placement string,
just like "ceph nfs cluster create" does. Align the documentation
so the placement string is explained in the same manner in both
places.
mgr/dashboard: fix error when typing existing paths in the Ganesha form
- The `CephFS.ls_dir()` implementation in the backend had changed, the code in the
UI endpoint `/ui-api/nfs-ganesha/lsdir` needs to adapt.
- Add fs_name as resource_id in `/ui-api/nfs-ganesha/lsdir/<fs_name>` to distinguish FS.
- Add more checks and unit tests.
Conflicts:
src/pybind/mgr/dashboard/controllers/nfsganesha.py
src/pybind/mgr/dashboard/tests/test_ganesha.py
- Conflict due to import order changed in the master
Jason Dillaman [Mon, 21 Sep 2020 18:05:17 +0000 (14:05 -0400)]
librbd: skip cache initialization if data IoCtx is invalid
This resolves the potential for an assertion failure in the cache
constructors that occurs when the data IoCtx is not valid. Errors
are deferred until actual IOs are issued against the invalid data
pool.
Conflicts:
src/librbd/plugin/ParentCache.cc
- parent_cache_enabled is not split to above file.
We just add another data ioctx checking before
sending parent cache
Jason Dillaman [Mon, 21 Sep 2020 16:53:37 +0000 (12:53 -0400)]
osdc/ObjectCacher: overwrite might cause stray read request callbacks
In librbd, if readahead is active, there might be a pending read request
for the cache which is then (partially) overwritten by a write request.
This overwrite will cause bh splits and merges which can cause the
bh read callback to fail to invoke the pending read callbacks.
Fixes: https://tracker.ceph.com/issues/46822 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 94d43165ed7319d163640f38d154f8f80408eb14)
rgw: fix setting of namespace in ordered and unordered bucket listing
The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.
luo rixin [Tue, 1 Sep 2020 09:06:40 +0000 (17:06 +0800)]
rgw/rgw_file: Fix the incorrect lru object eviction
In func lookup_fh, when RGWFileHandle not be found in fh_cache, it
need to recycle an object and create an new RGWFileHandle. When there
are multi threads use lookup_fh to find and create RGWFileHandle concurrently,
it must to make sure evict lru object from the partiton of fh_cache which new
RGWFileHandle will be inserted to.
Fixes: https://tracker.ceph.com/issues/47235 Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit f2097338722d7f2526bb815da47695f2da17fcce)
rgw: modifying documentation to include explanation of
open id connect provider related REST APIs and removing
references to token introspection for validating incoming
web token.
projects / ceph.git / log