Ramana Raja [Thu, 23 Jun 2016 10:36:53 +0000 (16:06 +0530)]
ceph_volume_client: modify locking of meta files
File locks are applied on meta files before updating the meta
file contents. These meta files would need to be cleaned up
sometime, which could lead to locks being held on unlinked meta
files. Prevent this by checking whether the file had been deleted
after lock was acquired on it.
Ramana Raja [Tue, 7 Jun 2016 19:12:18 +0000 (00:42 +0530)]
ceph_volume_client: recover from dirty auth and auth meta updates
Check dirty flag after locking something and call recover() if we are
opening something dirty (racing with another instance of the driver
restarting after failure) -- only required if someone running multiple
manila-share instances with Ceph loaded.
Ramana Raja [Tue, 21 Jun 2016 06:44:56 +0000 (12:14 +0530)]
ceph_volume_client: modify data layout in meta files
Notable changes to data layout in auth meta and volume meta files:
In the auth meta files, add a 'dirty' flag to track the status of auth
updates to a single volume.
In the volume meta file, make the 'dirty' flag track the status of
auth updates for a single ID.
Optimize the recovery of partial auth update changes to auth meta,
volume meta, and the Ceph backend, facilitated by changes in the
data layout in the meta files.
Store a two-way mapping between auth IDs and volumes.
Enables us to record some metadata on auth ids (which
openstack tenant created it) so that we can avoid exposing
keys to other tenants who try to use the same ceph
auth id.
Enables us to expose the list of which auth ids have access
to a volume, so that Manila's update_access() can be
implemented efficiently.
DNM: see TODOs inline.
Fixes: http://tracker.ceph.com/issues/15615 Signed-off-by: John Spray <john.spray@redhat.com>
John Spray [Mon, 7 Mar 2016 13:06:41 +0000 (13:06 +0000)]
pybind: enable integer flags to libcephfs open
The 'rw+' style flags are handy and convenient, but
they don't capture all possibilities. Change to
optionally accept an integer here for advance users
who want to specify arbitrary combinations of
flags.
Mark Nelson [Sun, 17 Jul 2016 11:39:41 +0000 (06:39 -0500)]
os/bluestore: revert preferred csum behavior
This passes "ceph_test_objectstore --gtest_filter=*/2".
This restores 4K random read performance to previous levels when objects
are were previously written out using large IOs (4MB in this case):
Somnath Roy [Sat, 9 Jul 2016 02:41:46 +0000 (22:41 -0400)]
Bluestore: Fixed a Bluestore crash
A bluestore race condition is been fixed by protecting txc structures
within _txc_state_poc with collection lock.
Mark's comments:
This fixes segfaults during random write tests with bluestore.
This passes "ceph_test_objectstore --gtest_filter=*/2".
This may introduce a small performance regresion, though there is enough
noise in the results to make it inconclusive.
Closes #10220
Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
test: run_seed_to_range.sh: check existance of a directory using [ -d "$dir" ]
sadly, sh evalutes `[ -d ]` to true. as it takes "-d" as a non-empty
string as true.
this fixes following failure
```
2016-07-12T23:22:02.839 INFO:teuthology.orchestra.run.mira084.stderr:cp:
missing destination file operand after β.β
2016-07-12T23:22:02.839 INFO:teuthology.orchestra.run.mira084.stderr:Try
'cp --help' for more information.
```
see
http://pulpito.ceph.com/kchai-2016-07-12_23:09:35-rados-wip-kefu-testing2---basic-mira/311334/
this fixes the test failure of
```
2016-07-12T23:29:40.935
INFO:tasks.workunit.client.0.mira101.stderr:/home/ubuntu/cephtest/workunit.client.0/cephtool/test.sh:
line 153: /CEPH_WATCH_9445: Permission denied
```
see
http://pulpito.ceph.com/kchai-2016-07-12_23:09:35-rados-wip-kefu-testing2---basic-mira/311333/
Fixes: http://tracker.ceph.com/issues/16665
Instead of drain_all_but() that specifies number of stacks to leave behind,
added drain_all_but_stack() that has a specific stack specified. This is needed
so that we don't call wakeup() through lease_cr->go_down() on a cr stack that
was already collected.
cmake: disable -fvar-tracking-assignments for ceph_dencoder.cc
there are too many variables involved in main() of ceph_dencoder.cc, but
-fvar-tracking-assignments is enabled by default, when the compiler
finds that the number of variables exceeds the limit
"max-vartrack-size", it gives up, and tries again disabling this option.
this change avoids the recompilation of ceph_dencoder.cc and also silences
the warning of
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/encoding/ceph_dencoder.cc:
In function βint main(int, const char**)β:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/encoding/ceph_dencoder.cc:289:5:
note: variable tracking size limit exceeded with
-fvar-tracking-assignments, retrying without
int main(int argc, const char **argv)
^
```
include/ceph_fs.h: guard `#define CEPH_SETATTR_*` with #ifndef
this silences the warnings like:
```
from
/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-11.0.0-513-gfe98b0b/src/java/native/libcephfs_jni.cc:33:
warning:
/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-11.0.0-513-gfe98b0b/src/include/ceph_fs.h:373:0:
"CEPH_SETATTR_MODE" redefined [enabled by default]
^
In file included from
/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-11.0.0-513-gfe98b0b/src/java/native/libcephfs_jni.cc:32:0:
/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-11.0.0-513-gfe98b0b/src/include/cephfs/libcephfs.h:106:0:
note: this is the location of the previous definition
^
```
Josh Durgin [Thu, 23 Jun 2016 00:54:33 +0000 (17:54 -0700)]
googletest: use ceph branch
Added the same functionality as the previous gtest/gmock branches -
.gitignore and not installing shared libs that we only want to
statically link to test programs.
the mock_open() does not support "for i in open()", and readline() and
readlines() support was added in py3.3 and py3.4 so for better backwards
compatibility, we should change the code being tested to use the plain
read() call.
and also use open(path) instead of open(path, 'rb') for simplicity,
otherwise we need to use bytestring for comparison and pass the same
parameters to mock.
this fixes the pep8 failure:
```
pep8 runtests: commands[0] | flake8 ceph_detect_init tests
tests/test_all.py:121:17: E128 continuation line under-indented for
visual indent
tests/test_all.py:125:17: E128 continuation line under-indented for
visual indent
tests/test_all.py:129:17: E128 continuation line under-indented for
visual indent
tests/test_all.py:133:17: E128 continuation line under-indented for
visual indent
```
Haomai Wang [Thu, 7 Jul 2016 11:12:46 +0000 (19:12 +0800)]
ceph_test_msgr: clean previous sent queue infos
1. A -> B
2. goto standby
3. B mark down
4. A reconnect to B
5. got reset session and dispatch remote reset
6. because remote reset is executed in DispatchQueue, it will be delayed
7. A -> B successfully and begin to send message
8. assert because we found the first message is missing but it's reasonble
Haomai Wang [Wed, 29 Jun 2016 09:14:16 +0000 (17:14 +0800)]
msg/async: make sure worker started before let msgr ready
When we create event thread, it need a little time to enter event loop(like
calling set_owner), if caller is going to call create_file_event before event
thread enter event loop, it will trigger assert.
projects / ceph.git / log