Venky Shankar [Fri, 9 Oct 2020 11:06:45 +0000 (07:06 -0400)]
tests/pybind/cephfs: cleanup xattrs before starting tests
Some xattr tests do not fully cleanup set xattrs. Subsequent
tests may expect xattrs to be absent during the test, such as
setting an xattr and then removing followed by checking if the
xattr list to be empty. This may fail if earlier tests do not
cleanup xattrs, especially for root.
So, cleanup xattrs on root before starting tests. Other directories
are removed anyway, so we do not have to bother about those.
Venky Shankar [Tue, 25 Aug 2020 01:48:53 +0000 (21:48 -0400)]
mds: restrict setting/removing certain xattrs in ceph namespace
Since all ceph.* xattrs need not be virtual (stored in inode
structure), restrict certain xattrs (ceph.mirror.info) to be
persisted in xattr_map. Other ceph.* xattrs which do not pass
the virtual xattr check are rejected.
Venky Shankar [Wed, 26 Aug 2020 12:55:51 +0000 (08:55 -0400)]
mds: introduce ceph.mirror.info virtual xattr
This is a compound xattr with the xattr value being fixed in
format. MDS stores this xattr as two (since the xattr value
right now just has two components or compounds) separate entries
in the xattr_map. This is done to avoid bloating the xattr value
if more "compounds" are added.
You may ask, why do it this way rather having the application
(cephfs-mirror daemon in this case) just set each xattr one
after the other? Well, we loose xattr consistency (from the
application point-of-view) -- an application crash (bug!) or
an ENOSPC in the server could leave M out of N xattrs (M < N)
on-disk.
With the compound xattr operation done on the server side,
journaling ensure that either all (N) xattrs are available or
none are available after recovering from a crash.
Venky Shankar [Tue, 25 Aug 2020 01:26:12 +0000 (21:26 -0400)]
mds: introduce is_ceph_vxattr() helper
Not all ceph.* xattrs are virtual -- virtual in the sense
that such xattrs have entries in the inode structure (inode_t)
rather than in xattr_map. There could be cases where an xattr
is in ceph namespace but does not necessarily need to be stored
for each inode -- so filter such xattrs individually rather than
treating all ceph.* xattrs as virtual.
mds: support sending empty perf metrics to ceph-manager
Right now, there are no per-mds metrics that are tracked and
sent by mds. However, such metrics will get added soon. So,
send empty performance metrics to ceph-manager for now.
Changcheng Liu [Wed, 23 Sep 2020 07:39:47 +0000 (15:39 +0800)]
mailmap: update Intel employee mail/org
1. "changcheng.liu@aliyun.com" need be classified into intel until now.
This reverts part of commit: df07e9f3
2. add "Yuan Lu <yuan.y.lu@intel.com>" in mailmap
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
luo rixin [Tue, 1 Sep 2020 09:06:40 +0000 (17:06 +0800)]
rgw/rgw_file: Fix the incorrect lru object eviction
In func lookup_fh, when RGWFileHandle not be found in fh_cache, it
need to recycle an object and create an new RGWFileHandle. When there
are multi threads use lookup_fh to find and create RGWFileHandle concurrently,
it must to make sure evict lru object from the partiton of fh_cache which new
RGWFileHandle will be inserted to.
Fixes: https://tracker.ceph.com/issues/47235 Signed-off-by: luo rixin <luorixin@huawei.com>
Jan Fajerski [Fri, 9 Oct 2020 09:22:01 +0000 (11:22 +0200)]
Merge PR #32778 into master
* refs/pull/32778/head:
msg: Use sockets instead of pipes for wakeup events on win32
common: Convert remote errors on win32
common,msg: Initialize Windows WSA and TLS
common,os,kv: Define aligned_free
common: avoid CLOCK_*_COARSE warnings on win32
global: Windows support
Jan Fajerski [Thu, 8 Oct 2020 06:45:26 +0000 (08:45 +0200)]
ceph-volume: don't exit before empty report can be printed
get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.
Fixes: https://tracker.ceph.com/issues/47760 Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Nizamudeen A [Wed, 7 Oct 2020 17:21:10 +0000 (22:51 +0530)]
mgr/dashboard: Add short descriptions to the telemetry report preview
A small info icon with a popover on the telemetry report preview form step2 for
Report ID and Report Preview.
Report ID: "A randomized UUID to identify a particular cluster over the course of several telemetry reports."
Report preview: "The actual telemetry data that will be submitted."
Fixes: https://tracker.ceph.com/issues/47610 Signed-off-by: Nizamudeen A <nia@redhat.com>
rgw: fix setting of namespace in ordered and unordered bucket listing
The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
rgw: allow rgw-orphan-list to note when rados objects are in namespace
Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.
However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.
Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
include: explicitly define all types needed for libc++ iterator
There is a serious difference between iterator in
libstdc++ (gcc) versus libc++ (clang)
This generates a rather big and vague set of warnings and notes.
The crux is the warnings about `requirement '!__is_forward_iterator....`
Extra information can also be found on:
https://www.fluentcpp.com/2018/05/08/std-iterator-deprecated/
```
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:142:15: error: no matching constructor for initialization of 'io::Extents' (aka 'vector<pair<unsigned long, unsigned long> >')
io::Extents image_extents{read_op.image_interval.begin(),
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:769:35: note: in instantiation of member function 'librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>::send_read' requested here
template class librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>;
^
/usr/include/c++/v1/vector:516:14: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
explicit vector(size_type __n, const allocator_type& __a);
^
/usr/include/c++/v1/vector:518:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
vector(size_type __n, const value_type& __x);
^
/usr/include/c++/v1/vector:559:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'const std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
vector(const vector& __x, const allocator_type& __a);
^
/usr/include/c++/v1/vector:568:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'initializer_list<std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::value_type>' (aka 'initializer_list<std::__1::pair<unsigned long, unsigned long> >') for 1st argument
vector(initializer_list<value_type> __il, const allocator_type& __a);
^
/usr/include/c++/v1/vector:579:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
vector(vector&& __x, const allocator_type& __a);
^
/usr/include/c++/v1/vector:521:9: note: candidate template ignored: requirement '!__is_forward_iterator<interval_set<unsigned long, std::map>::iterator>::value' was not satisfied [with _InputIterator = interval_set<unsigned long, std::map>::iterator]
vector(_InputIterator __first,
^
/usr/include/c++/v1/vector:536:9: note: candidate template ignored: requirement 'is_constructible<std::__1::pair<unsigned long, unsigned long>, unsigned long &>::value' was not satisfied [with _ForwardIterator = interval_set<unsigned long, std::map>::iterator]
vector(_ForwardIterator __first,
^
/usr/include/c++/v1/vector:502:40: note: candidate constructor not viable: requires single argument '__a', but 2 arguments were provided
_LIBCPP_INLINE_VISIBILITY explicit vector(const allocator_type& __a)
^
/usr/include/c++/v1/vector:514:14: note: candidate constructor not viable: requires single argument '__n', but 2 arguments were provided
explicit vector(size_type __n);
^
/usr/include/c++/v1/vector:558:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
vector(const vector& __x);
^
/usr/include/c++/v1/vector:565:5: note: candidate constructor not viable: requires single argument '__il', but 2 arguments were provided
vector(initializer_list<value_type> __il);
^
/usr/include/c++/v1/vector:571:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
vector(vector&& __x)
^
/usr/include/c++/v1/vector:519:5: note: candidate constructor not viable: requires 3 arguments, but 2 were provided
vector(size_type __n, const value_type& __x, const allocator_type& __a);
^
/usr/include/c++/v1/vector:496:5: note: candidate constructor not viable: requires 0 arguments, but 2 were provided
vector() _NOEXCEPT_(is_nothrow_default_constructible<allocator_type>::value)
^
/usr/include/c++/v1/vector:529:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
vector(_InputIterator __first, _InputIterator __last, const allocator_type& __a,
^
/usr/include/c++/v1/vector:543:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
vector(_ForwardIterator __first, _ForwardIterator __last, const allocator_type& __a,
^
1 error generated.
```
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Adam Kupczyk [Wed, 7 Oct 2020 10:34:04 +0000 (06:34 -0400)]
os/bluestore: fix memory accounting in TwoQBufferCacheShard
Variable buffer_bytes has been redefined in TwoQBufferCacheShard,
causing PriCache to see always 0 usage when 2q cache was selected,
as it looks at it through BufferCacheShard::_get_bytes().
Kefu Chai [Tue, 6 Oct 2020 11:09:18 +0000 (19:09 +0800)]
crimson/os: mark class with "final" dtor "final"
to silence warning like
scheduler.cc:145:30: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
~ClassedOpQueueScheduler() final {};
^
/home/jenkins-build/build/workspace/ceph-perf-crimson/ceph-pr/src/crimson/osd/scheduler/scheduler.cc:52:7: note: mark 'ClassedOpQueueScheduler<T>' as 'final' to silence this warning
class ClassedOpQueueScheduler : public Scheduler {
^
Kefu Chai [Tue, 6 Oct 2020 11:03:38 +0000 (19:03 +0800)]
crimson/os: mark dtor of interface class "virtual"
we use a smart_ptr in Seastore to manage the life cycle of
`TransactionManager` which is in turn a subclass
`ExtentCallbackInterface`, so should declare the dtor of latter as
virtual.
mgr/dashboard: fix error when typing existing paths in the Ganesha form
- The `CephFS.ls_dir()` implementation in the backend had changed, the code in the
UI endpoint `/ui-api/nfs-ganesha/lsdir` needs to adapt.
- Add fs_name as resource_id in `/ui-api/nfs-ganesha/lsdir/<fs_name>` to distinguish FS.
- Add more checks and unit tests.
Or Ozeri [Wed, 16 Sep 2020 08:42:45 +0000 (11:42 +0300)]
librbd: support assembling sparse results of multiple object extents
Currently, Striper supports assembling results representing a single object extent.
Recently, the object dispatch API was extended allowing to read multiple object extents per rados operation.
This commit enables the Striper to correctly un-sparsify the results of the new read extents API.
projects / ceph.git / log