]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph-volume simple.activate remove str formatting from logging
Alfredo Deza [Wed, 8 Nov 2017 14:17:52 +0000 (09:17 -0500)]
ceph-volume simple.activate remove str formatting from logging

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 230faf04a185c0b95c8ad497ddd5501dc70c1409)

7 years agoceph-volume util.arg_validators always convert the path to an absolute path
Alfredo Deza [Wed, 8 Nov 2017 14:11:41 +0000 (09:11 -0500)]
ceph-volume util.arg_validators always convert the path to an absolute path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b5acf4d720b5395ae4ccbf007c94c99610e40089)

7 years agoceph-volume util.arg_validators require superuser privs to check paths
Alfredo Deza [Wed, 8 Nov 2017 14:10:47 +0000 (09:10 -0500)]
ceph-volume util.arg_validators require superuser privs to check paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 376dad74159532f901d0044f95330fc02f93bd25)

7 years agoceph-volume simple.scan skip directories to scan
Alfredo Deza [Wed, 8 Nov 2017 14:09:06 +0000 (09:09 -0500)]
ceph-volume simple.scan skip directories to scan

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6bc1e9ee0b8d7bc475f218ae3dd8c6c7fe5a983a)

7 years agoceph-volume tests.devices.simple add activate CLI unit tests
Alfredo Deza [Tue, 7 Nov 2017 20:08:13 +0000 (15:08 -0500)]
ceph-volume tests.devices.simple add activate CLI unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cde0d517563b537319d06c758b93e421cdabe59d)

7 years agoceph-volume tests.devices add help menu unit test
Alfredo Deza [Tue, 7 Nov 2017 20:07:44 +0000 (15:07 -0500)]
ceph-volume tests.devices add help menu unit test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dcc7c72196a327f9501045f025761b228b5379df)

7 years agodoc/ceph-volume systemd describe the interaction with 'simple' activation
Alfredo Deza [Tue, 7 Nov 2017 18:42:45 +0000 (13:42 -0500)]
doc/ceph-volume systemd describe the interaction with 'simple' activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7071eeaeb5d77c6ab6e7a24f1b630cb0cf8a8e7c)

7 years agodoc/ceph-volume index initial description of simple
Alfredo Deza [Tue, 7 Nov 2017 18:42:11 +0000 (13:42 -0500)]
doc/ceph-volume index initial description of simple

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f9ee51f9feadb6d0f91b76a2b2ba8a24f1523a77)

7 years agodoc/ceph-volume lvm add back a system.rst with just the lvm portion of it
Alfredo Deza [Tue, 7 Nov 2017 18:41:12 +0000 (13:41 -0500)]
doc/ceph-volume lvm add back a system.rst with just the lvm portion of it

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f872cd29b93e02b77172f7b382c4746af9ba845d)

7 years agodoc/ceph-volume make systemd.rst generic enough to be related to other sub-commands
Alfredo Deza [Tue, 7 Nov 2017 18:40:44 +0000 (13:40 -0500)]
doc/ceph-volume make systemd.rst generic enough to be related to other sub-commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cf4e32a0dc3ab4f55fe7857b51afbc63a638ec66)

7 years agodoc/ceph-volume move lvm/systemd.rst to top level, making it generic
Alfredo Deza [Tue, 7 Nov 2017 18:39:52 +0000 (13:39 -0500)]
doc/ceph-volume move lvm/systemd.rst to top level, making it generic

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0672dd608d4601a936d703ea5ba6edc14a161d8e)

7 years agodoc/ceph-volume simple.activate initial take on documenting activation
Alfredo Deza [Tue, 7 Nov 2017 16:52:17 +0000 (11:52 -0500)]
doc/ceph-volume simple.activate initial take on documenting activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 80c796bd1c43125f5678710b494b03c486b3fce6)

7 years agodoc/ceph-volume lvm update systemd anchor to reflect lvm parent
Alfredo Deza [Tue, 7 Nov 2017 13:35:57 +0000 (08:35 -0500)]
doc/ceph-volume lvm update systemd anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 53420a4756a5e4fa6b0f2aa74d6ad1e2919f1954)

7 years agodoc/ceph-volume lvm.activate update systemd anchor to reflect lvm parent
Alfredo Deza [Tue, 7 Nov 2017 13:35:43 +0000 (08:35 -0500)]
doc/ceph-volume lvm.activate update systemd anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7fc0e7ebcae06f28f6ed08c0db14075ada34e8a7)

7 years agodoc/ceph-volume lvm.systemd update anchor to reflect lvm parent
Alfredo Deza [Tue, 7 Nov 2017 12:56:35 +0000 (07:56 -0500)]
doc/ceph-volume lvm.systemd update anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 01a8b7e6a45f6fd85bf53d3cab15acc81d0bae1b)

7 years agodoc/ceph-volume document the scan functionality
Alfredo Deza [Mon, 6 Nov 2017 20:53:39 +0000 (15:53 -0500)]
doc/ceph-volume document the scan functionality

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b30a3eaf17a20a1273724b2cfcaf0a7b36eb08f6)

7 years agodoc/ceph-volume update the index to include the simple sub command and files
Alfredo Deza [Mon, 6 Nov 2017 19:17:20 +0000 (14:17 -0500)]
doc/ceph-volume update the index to include the simple sub command and files

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d7f85c64e4b409ea95e30ccb5a1388278843c478)

7 years agoceph-volume tests.simple add tests for trigger parsing from systemd
Alfredo Deza [Mon, 6 Nov 2017 16:15:28 +0000 (11:15 -0500)]
ceph-volume tests.simple add tests for trigger parsing from systemd

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0e2620234e82abf85274b3c08b39cdf76430ce48)

7 years agoceph-volume tests.simple create etc_path scan tests
Alfredo Deza [Mon, 6 Nov 2017 15:36:14 +0000 (10:36 -0500)]
ceph-volume tests.simple create etc_path scan tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 90ebc4e1ffa7b3273787d708bb2c05180530b5d1)

7 years agoceph-volume tests.simple add checks for get_contents in scan
Alfredo Deza [Mon, 6 Nov 2017 15:06:02 +0000 (10:06 -0500)]
ceph-volume tests.simple add checks for get_contents in scan

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 647bfe0883d2bb05b4ff7f99894075df3e73d2d5)

7 years agoceph-volume tests.util add binary verification tests
Alfredo Deza [Mon, 6 Nov 2017 14:43:58 +0000 (09:43 -0500)]
ceph-volume tests.util add binary verification tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 827aa57557561355dd4068e26801b5f1ed62f295)

7 years agoceph-volume tests create a fixture for temporary files
Alfredo Deza [Mon, 6 Nov 2017 14:42:53 +0000 (09:42 -0500)]
ceph-volume tests create a fixture for temporary files

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 23da3445718c01f71e2a5748fb4fc1c5fdc85ca5)

7 years agoceph-volume systemd.systemctl helpers to mask ceph-disk
Alfredo Deza [Fri, 3 Nov 2017 18:36:38 +0000 (14:36 -0400)]
ceph-volume systemd.systemctl helpers to mask ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3da23fa4caea8c9dc91da3eee5b5acc1860bf242)

7 years agoceph-volume devices include simple at the module import level
Alfredo Deza [Fri, 3 Nov 2017 18:36:13 +0000 (14:36 -0400)]
ceph-volume devices include simple at the module import level

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 027a09b3ea6bac416ff632a69a53393badd51708)

7 years agoceph-volume main add simple to the sub-commands
Alfredo Deza [Fri, 3 Nov 2017 18:35:45 +0000 (14:35 -0400)]
ceph-volume main add simple to the sub-commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 94579c77ffe4d5ba2d457ad3981a2b9ed232a90f)

7 years agoceph-volume util add an OSDPath argument validator
Alfredo Deza [Fri, 3 Nov 2017 18:35:03 +0000 (14:35 -0400)]
ceph-volume util add an OSDPath argument validator

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a826fdf4a2ed9ff704fb823d05d3cb706ba89e38)

7 years agoceph-volume simple.system add a helper to check if a file is a binary or not
Alfredo Deza [Fri, 3 Nov 2017 18:34:39 +0000 (14:34 -0400)]
ceph-volume simple.system add a helper to check if a file is a binary or not

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7bf8676045b4f966b60cdadf4d5af1c89fca69ae)

7 years agoceph-volume util.system add a context manager for temporary mounting/unmounting
Alfredo Deza [Fri, 3 Nov 2017 18:34:06 +0000 (14:34 -0400)]
ceph-volume util.system add a context manager for temporary mounting/unmounting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6394cdd41943e3cda6867ecc59ab835cb2151707)

7 years agoceph-volume simple.scan initial take on directory/device scanning
Alfredo Deza [Fri, 3 Nov 2017 18:32:42 +0000 (14:32 -0400)]
ceph-volume simple.scan initial take on directory/device scanning

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0e54b9be621cc0969a3bbc972d1870080ccdae77)

7 years agoceph-volume simple.trigger enable systemd triggering for simple setups
Alfredo Deza [Fri, 3 Nov 2017 18:27:23 +0000 (14:27 -0400)]
ceph-volume simple.trigger enable systemd triggering for simple setups

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cbc0c0f775e96a5b0ab5d5bc141acebe4cc48373)

7 years agoceph-volume simple.activate initial take on activation
Alfredo Deza [Fri, 3 Nov 2017 18:26:13 +0000 (14:26 -0400)]
ceph-volume simple.activate initial take on activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5a97374605408d18e9be29bcc051ae7b913d1675)

7 years agoceph-volume simple add __init__ for the module
Alfredo Deza [Fri, 3 Nov 2017 18:23:47 +0000 (14:23 -0400)]
ceph-volume simple add __init__ for the module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 285e5bdd28a42d98418ca90ad0cdec6bd87849db)

7 years agoceph-volume simple create the initial, main module
Alfredo Deza [Fri, 3 Nov 2017 18:23:29 +0000 (14:23 -0400)]
ceph-volume simple create the initial, main module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f050502425a1993ae8edfdb5785b24129cffeeba)

7 years agoMerge pull request #18801 from linuxbox2/luminous-rgw-readdir2
Abhishek L [Thu, 9 Nov 2017 17:08:27 +0000 (18:08 +0100)]
Merge pull request #18801 from linuxbox2/luminous-rgw-readdir2

rgw_file: implement variant offset readdir processing -- pullup 2 of 2

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoMerge pull request #18734 from shinobu-x/wip-21648-luminous
Kefu Chai [Thu, 9 Nov 2017 14:36:27 +0000 (22:36 +0800)]
Merge pull request #18734 from shinobu-x/wip-21648-luminous

luminous: mgr[zabbix] float division by zero

Reviewed-by: Wido den Hollander <wido@widodh.nl>
7 years agoMerge pull request #18851 from tchaikov/wip-18759-pr-luminous
Kefu Chai [Thu, 9 Nov 2017 14:35:44 +0000 (22:35 +0800)]
Merge pull request #18851 from tchaikov/wip-18759-pr-luminous

mon/LogMonitor: "log last" should return up to n entries

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18741 from shinobu-x/wip-22019-luminous
Kefu Chai [Thu, 9 Nov 2017 14:27:47 +0000 (22:27 +0800)]
Merge pull request #18741 from shinobu-x/wip-22019-luminous

luminous: tests: "ceph osd create" is not idempotent

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agomon/LogMonitor: "log last" should return up to n entries 18851/head
Kefu Chai [Mon, 6 Nov 2017 08:01:23 +0000 (16:01 +0800)]
mon/LogMonitor: "log last" should return up to n entries

limit the # of returned entries to "num", and backoff the start iterator
by one if it reaches the rend().

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit de14103f5201da69b360d9b01e249896d59376b3)

7 years agorgw_file: implement variant offset readdir processing 18801/head
Matt Benjamin [Mon, 16 Oct 2017 01:48:19 +0000 (21:48 -0400)]
rgw_file: implement variant offset readdir processing

Introduce new rgw_readdir2(...), which in which continues
from an arbitrary dirent name, which presumably has been
seen in a prior partial enumeration.

Add single-file unit test for READDIR cases, librgw_file_marker.cc.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit a2c34c597371809bd8fd40ee1fc6b8e6a5145609)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #18800 from linuxbox2/luminous-rgw_file-fsid-mount
Abhishek L [Thu, 9 Nov 2017 09:10:53 +0000 (10:10 +0100)]
Merge pull request #18800 from linuxbox2/luminous-rgw_file-fsid-mount

Luminous rgw file fsid mount -- pullup 1 of 2

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoMerge pull request #18767 from smithfarm/wip-22020-luminous
Abhishek L [Thu, 9 Nov 2017 09:04:08 +0000 (10:04 +0100)]
Merge pull request #18767 from smithfarm/wip-22020-luminous

luminous: multisite: race between sync of bucket and bucket instance metadata

Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18794 from theanalyst/wip-18709
Abhishek L [Thu, 9 Nov 2017 08:56:16 +0000 (09:56 +0100)]
Merge pull request #18794 from theanalyst/wip-18709

luminous: rgw: Stale bucket index entry remains after object deletion

Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18726 from shinobu-x/wip-luminous-22035
Kefu Chai [Wed, 8 Nov 2017 10:19:27 +0000 (18:19 +0800)]
Merge pull request #18726 from shinobu-x/wip-luminous-22035

luminous: Spurious ceph-mgr failovers during mon elections

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18738 from shinobu-x/wip-21547-luminous
Kefu Chai [Wed, 8 Nov 2017 10:18:39 +0000 (18:18 +0800)]
Merge pull request #18738 from shinobu-x/wip-21547-luminous

luminous: ceph-mgr gets process called "exe" after respawn

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18727 from shinobu-x/wip-luminous-22034
Kefu Chai [Wed, 8 Nov 2017 10:17:54 +0000 (18:17 +0800)]
Merge pull request #18727 from shinobu-x/wip-luminous-22034

luminous: key mismatch for mgr after upgrade from jewel to luminous(dev)

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18723 from shinobu-x/wip-luminous-22023
Kefu Chai [Wed, 8 Nov 2017 06:44:41 +0000 (14:44 +0800)]
Merge pull request #18723 from shinobu-x/wip-luminous-22023

luminous: osd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL

7 years agoMerge pull request #18737 from shinobu-x/wip-21549-luminous
Kefu Chai [Wed, 8 Nov 2017 06:38:10 +0000 (14:38 +0800)]
Merge pull request #18737 from shinobu-x/wip-21549-luminous

luminous: the dashboard uses absolute links for filesystems and clients

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18736 from shinobu-x/wip-21638-luminous
Kefu Chai [Wed, 8 Nov 2017 06:37:19 +0000 (14:37 +0800)]
Merge pull request #18736 from shinobu-x/wip-21638-luminous

luminous: dashboard OSD list has servers and osds in arbitrary order

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18728 from shinobu-x/wip-luminous-22032
Kefu Chai [Wed, 8 Nov 2017 06:36:40 +0000 (14:36 +0800)]
Merge pull request #18728 from shinobu-x/wip-luminous-22032

luminous: dashboard barfs on nulls where it expects numbers

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #18699 from shinobu-x/wip-luminous-21538
Kefu Chai [Wed, 8 Nov 2017 06:34:35 +0000 (14:34 +0800)]
Merge pull request #18699 from shinobu-x/wip-luminous-21538

luminous: upmap does not respect osd reweights

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18719 from tchaikov/wip-pr-18373-luminous
Kefu Chai [Wed, 8 Nov 2017 06:30:42 +0000 (14:30 +0800)]
Merge pull request #18719 from tchaikov/wip-pr-18373-luminous

luminous: osd,mon: add max-pg-per-osd limit

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agodoc: fix the desc of "osd max pg per osd hard ratio" 18719/head
Kefu Chai [Wed, 18 Oct 2017 08:07:50 +0000 (16:07 +0800)]
doc: fix the desc of "osd max pg per osd hard ratio"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d940f6220346ad21dfc4f351c57389e8430c0257)

7 years agomon: change "mon_pg_warn_min_per_osd" to uint64_t
Kefu Chai [Tue, 17 Oct 2017 04:17:09 +0000 (12:17 +0800)]
mon: change "mon_pg_warn_min_per_osd" to uint64_t

* this silences a warning from -Wsign-compare.
* also switch all its user to the new-style option

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c3cd0c13a1585388df0f17ab53c170e39c711615)

7 years agoMerge pull request #18702 from liewegas/wip-pr-18670-luminous
Sage Weil [Wed, 8 Nov 2017 04:27:15 +0000 (22:27 -0600)]
Merge pull request #18702 from liewegas/wip-pr-18670-luminous

luminous: qa/tasks/thrashosds: set min_in default to 4

7 years agoMerge pull request #18596 from smithfarm/wip-21955-luminous
Abhishek L [Tue, 7 Nov 2017 21:28:58 +0000 (22:28 +0100)]
Merge pull request #18596 from smithfarm/wip-21955-luminous

luminous: tests: add EC data pool to testing

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>
7 years agorgw_file: introduce rgw_mount2 with a bucket name parameter 18800/head
Gui Hecheng [Sat, 22 Jul 2017 05:12:06 +0000 (13:12 +0800)]
rgw_file: introduce rgw_mount2 with a bucket name parameter

This explicitly allow mount a specified bucket in librgw.
Originally, mounting a bucket is implemented like a sub-directory
mount in nfs-ganesha FSAL_RGW with lookup_path.
With this change, we allow root_fh to points exactly to the root
of a mounted fs instance, which is a bucket or "/" rather than
always let root_fh points to "/".

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit 860716e17e840df11a7e1d8b6ca0c1ee694b038b)

7 years agorgw_file: new fsid for fs instance
Gui Hecheng [Fri, 26 May 2017 09:45:29 +0000 (17:45 +0800)]
rgw_file: new fsid for fs instance

We use an in-memory fs_inst_counter for fsid currently, but it
inevitably cause nfs automount problem across a nfs-ganesha crash.
So here we relate the fsid to the s3 uid with a hash.
With this, fsid remains consistent across all fs instances and
across crashes.

We should be able to upgrade from old fsid to new fsid with
update_fh().

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit 178b547e83a2c0eeb024bbe4ca2ff8d8e7d50572)

7 years agoMerge pull request #18628 from batrick/i21953
Abhishek L [Tue, 7 Nov 2017 21:07:45 +0000 (22:07 +0100)]
Merge pull request #18628 from batrick/i21953

luminous: mds: sanitize mdsmap of removed pools

Reviewed-By: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18626 from dillaman/wip-21968
Abhishek L [Tue, 7 Nov 2017 17:33:21 +0000 (18:33 +0100)]
Merge pull request #18626 from dillaman/wip-21968

luminous: cls/journal: fixed possible infinite loop in expire_tags

Reviewed-By: Mykola Golub <trociny@gmail.com>
7 years agoMerge pull request #18688 from liewegas/wip-balancer-luminous
Abhishek L [Tue, 7 Nov 2017 17:32:20 +0000 (18:32 +0100)]
Merge pull request #18688 from liewegas/wip-balancer-luminous

luminous: mgr/balancer: backport crush-compat work from master

Reviewed-By: John Spray <jspray@redhat.com>
7 years agorgw: fix BZ 1500904, Stale bucket index entry remains after object deletion 18794/head
J. Eric Ivancich [Fri, 3 Nov 2017 13:15:13 +0000 (09:15 -0400)]
rgw: fix BZ 1500904, Stale bucket index entry remains after object deletion

We have a race condition:

 1. RGW client #1: requests an object be deleted.
 2. RGW client #1: sends a prepare op to bucket index OSD #1.
 3. OSD #1:        prepares the op, adding pending ops to the bucket dir entry
 4. RGW client #2: sends a list bucket to OSD #1
 5. RGW client #2: sees that there are pending operations on bucket
                   dir entry, and calls check_disk_state
 6. RGW client #2: check_disk_state sees that the object still exists, so it
                   sends CEPH_RGW_UPDATE to bucket index OSD (#1)
 7. RGW client #1: sends a delete object to object OSD (#2)
 8. OSD #2:        deletes the object
 9. RGW client #2: sends a complete op to bucket index OSD (#1)
10. OSD #1:        completes the op
11. OSD #1:        receives the CEPH_RGW_UPDATE and updates the bucket index
                   entry, thereby **RECREATING** it

Solution implemented:

At step #5 the object's dir entry exists. If we get to beginning of
step #11 and the object's dir entry no longer exists, we know that the
dir entry was just actively being modified, and ignore the
CEPH_RGW_UPDATE operation, thereby NOT recreating it.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit b33f529e79b74314a2030231e1308ee225717743)

7 years agoMerge pull request #18569 from kmroz/wip-21939-luminous
Abhishek L [Tue, 7 Nov 2017 17:21:41 +0000 (18:21 +0100)]
Merge pull request #18569 from kmroz/wip-21939-luminous

luminous: list bucket which enable versioning get wrong result when user marker

Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18591 from cbodley/wip-21938
Abhishek L [Tue, 7 Nov 2017 17:21:03 +0000 (18:21 +0100)]
Merge pull request #18591 from cbodley/wip-21938

luminous: rgw: fix error handling in ListBucketIndexesCR
(approved in irc & standups)

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoMerge pull request #18539 from linuxbox2/luminous-baixuyeyu-refcnt
Abhishek L [Tue, 7 Nov 2017 17:17:57 +0000 (18:17 +0100)]
Merge pull request #18539 from linuxbox2/luminous-baixuyeyu-refcnt

luminous: baixuyeyu refcnt

Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18516 from ceph/wip-bp-pr18193-luminous
Abhishek L [Tue, 7 Nov 2017 17:15:38 +0000 (18:15 +0100)]
Merge pull request #18516 from ceph/wip-bp-pr18193-luminous

qa/rgw: ignore errors from 'pool application enable'

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18599 from linuxbox2/lum-rgwfile-21940
Abhishek L [Tue, 7 Nov 2017 17:14:21 +0000 (18:14 +0100)]
Merge pull request #18599 from linuxbox2/lum-rgwfile-21940

rgw_file:  set s->obj_size from bytes_written

Reviewed-By: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18566 from kmroz/wip-21914-luminous
Abhishek L [Tue, 7 Nov 2017 12:07:09 +0000 (13:07 +0100)]
Merge pull request #18566 from kmroz/wip-21914-luminous

luminous: [rbd-mirror] peer cluster connections should filter out command line optionals

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>
7 years agoosd,mon: change "mon_max_pg_per_osd" to uint64_t
Kefu Chai [Tue, 17 Oct 2017 04:10:55 +0000 (12:10 +0800)]
osd,mon: change "mon_max_pg_per_osd" to uint64_t

as it should never be a negative number. if this option is disabled,
it's 0.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0d68197376b6cf48c6679d77ec9f28f279f3da68)

7 years agoosd: add max-pg-per-osd limit
Kefu Chai [Tue, 26 Sep 2017 07:54:14 +0000 (15:54 +0800)]
osd: add max-pg-per-osd limit

osd will refused to create new pgs, until its pg number is lower
than the max-pg-per-osd upper bound setting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c7df944c7f28232873ba681eedce72cdb062ea5)

7 years agoqa/tasks/ceph: add "create_rbd_pool" option
Kefu Chai [Sun, 15 Oct 2017 12:46:28 +0000 (20:46 +0800)]
qa/tasks/ceph: add "create_rbd_pool" option

True by default, but we can opt not to create the "rbd" pool.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ac1e13ce63a6f32ae9ab0db8196c8cc8100d94ed)

7 years agoosd: print osd as "osd.$osdid" in log message
Kefu Chai [Thu, 12 Oct 2017 12:07:46 +0000 (20:07 +0800)]
osd: print osd as "osd.$osdid" in log message

easier to read this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c6b71ebf6b7b7c54560f331d5f8e4e24f5a60633)

7 years agomon/OSDMonitor: bypass checks if pg_temp.forced
Kefu Chai [Tue, 26 Sep 2017 07:50:08 +0000 (15:50 +0800)]
mon/OSDMonitor: bypass checks if pg_temp.forced

originally, monitor ignores the pg_temp messages sent from replica osds,
but to re-trigger peering if replica osd's pg number drops down below
the setting, we need to bypass the checks.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit da6f7a810055a0dc9e364aca8e6efbfbc32881b3)

7 years agomessages/MOSDPGTemp: add `forced` field
Kefu Chai [Tue, 26 Sep 2017 07:49:11 +0000 (15:49 +0800)]
messages/MOSDPGTemp: add `forced` field

the monitor is supposed to bypass some checks if `forced` is true.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6335bf791003e9e33b2f738ab3da06f3c626537a)

7 years agoMerge pull request #18775 from tchaikov/wip-upgrade-luminous-x
Kefu Chai [Tue, 7 Nov 2017 09:48:34 +0000 (17:48 +0800)]
Merge pull request #18775 from tchaikov/wip-upgrade-luminous-x

qa: add 1-ceph-install to upgrade:luminous-x

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
7 years agoqa: add 1-ceph-install to upgrade:luminous-x 18775/head
Yuri Weinstein [Tue, 29 Aug 2017 16:50:51 +0000 (09:50 -0700)]
qa: add 1-ceph-install to upgrade:luminous-x

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit aff342627cf77e2faae6904e626d9fb2c897f155)

7 years agoqa/suites/rados/thrash/d-*/*balancer*: debug osd in mgr 18688/head
Sage Weil [Tue, 7 Nov 2017 03:17:22 +0000 (21:17 -0600)]
qa/suites/rados/thrash/d-*/*balancer*: debug osd in mgr

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agorgw: remove placement_rule from cls_user_bucket_entry 18767/head
Casey Bodley [Tue, 31 Oct 2017 20:56:01 +0000 (16:56 -0400)]
rgw: remove placement_rule from cls_user_bucket_entry

placement_rule is no longer needed in cls_user_bucket_entry, because the
only time that it's needed, we can read it from the bucket instance in
RGWRados::update_containers_stats()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8e62e3526643da67f5af7daa687120feed469785)

7 years agorgw: remove placement_rule from rgw_link_bucket()
Casey Bodley [Tue, 31 Oct 2017 20:29:31 +0000 (16:29 -0400)]
rgw: remove placement_rule from rgw_link_bucket()

with the fallback in RGWRados::update_containers_stats(), we no longer
need to pass placement_rule into the cls_user_bucket_entry

this removes the dependency between bucket entrypoint metadata and its
bucket instance metadata during multisite sync

Fixes: http://tracker.ceph.com/issues/21990
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit dd5d278a4f9b080234cbb77e448208803ec0cf93)

7 years agorgw: take placement_rule from bucket info in update_containers_stats
Casey Bodley [Tue, 31 Oct 2017 20:26:54 +0000 (16:26 -0400)]
rgw: take placement_rule from bucket info in update_containers_stats

in case the cls_user_bucket_entry doesn't contain a placement_rule, take
it from the bucket instance info

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 531195d80d5bac774d58b210307eb19b57d26345)

7 years agoqa/suites/rados/thrash: combine balancer with require-luminous
Sage Weil [Sat, 4 Nov 2017 14:32:51 +0000 (09:32 -0500)]
qa/suites/rados/thrash: combine balancer with require-luminous

We can only do the balancer stuff if we've done the mkfs-time
require luminous because otherwise the mgr commands aren't proxied via
the mon.

Only do balancer for thrash tests; not for basic, monthrash, verify.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa: stop testing deprecated "ceph osd create" 18741/head
Kefu Chai [Wed, 1 Nov 2017 02:45:09 +0000 (10:45 +0800)]
qa: stop testing deprecated "ceph osd create"

"ceph osd create" is not idempotent, and is considered deprecated.

Fixes: http://tracker.ceph.com/issues/21993
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 371d3f40911917704f4992c48236b4c60905bf4e)

7 years agomgr: set explicit thread name 18738/head
John Spray [Fri, 15 Sep 2017 16:23:39 +0000 (12:23 -0400)]
mgr: set explicit thread name

This gets used as our process name in some situations
when respawning.  This is the same as what commit 4f177bb6b
did for the MDS.

Fixes: http://tracker.ceph.com/issues/21404
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 44dce0a5664639b2ae0949cc434f1be1176d872d)

7 years agopybind/mgr/dashboard: updated favicon (old one was 404) 18737/head
Nick Erdmann [Tue, 22 Aug 2017 16:55:11 +0000 (16:55 +0000)]
pybind/mgr/dashboard: updated favicon (old one was 404)

Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit 7a54520f5c3d4db27fe7e2a12760dcd838ba89aa)

7 years agopybind/mgr/dashboard: clean up
Nick Erdmann [Tue, 22 Aug 2017 08:08:45 +0000 (08:08 +0000)]
pybind/mgr/dashboard: clean up

Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit f512ac3d7712850f274cd01bb589aa9fbc8c8bef)

7 years agopybind/mgr/dashboard: add url_prefix
Nick Erdmann [Mon, 21 Aug 2017 17:21:10 +0000 (17:21 +0000)]
pybind/mgr/dashboard: add url_prefix

This adds a configuration variable url_prefix to the dashboard that
that is prepended to all URLs so you can access the dashboard at
http://$IP:$PORT/$PREFIX/. This is necessary if you wish to use a
reverse http proxy that forwards to the dashboard under a sub-path.

Fixes: http://tracker.ceph.com/issues/20568
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit 6d285fff0df598d66f46d38bf0a6a2cc9c0dc62f)

7 years agomgr/dashboard: sort servers and OSDs in OSD list 18736/head
John Spray [Wed, 27 Sep 2017 14:13:48 +0000 (15:13 +0100)]
mgr/dashboard: sort servers and OSDs in OSD list

Fixes: http://tracker.ceph.com/issues/21572
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f409099fa25b73fe580ee4662aff51636f118ed2)

7 years agomgr/zabbix: log exceptions with backtrace 18734/head
John Spray [Sat, 23 Sep 2017 14:22:34 +0000 (15:22 +0100)]
mgr/zabbix: log exceptions with backtrace

Logging just the exception object gave a sometimes-enigmatic
single line.  Let's use the logger exception method so that
we get a backtrace.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 316dcc8f0b42b85503632d5472c45c9828882f2c)

7 years agomgr/zabbix: fix div by zero
John Spray [Sat, 23 Sep 2017 14:18:18 +0000 (15:18 +0100)]
mgr/zabbix: fix div by zero

Fixes: http://tracker.ceph.com/issues/21518
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 9c02738b4ea0303f5b3cbfc4748d6791007be834)

7 years agoMerge pull request #18673 from dzafman/wip-21833
Sage Weil [Sat, 4 Nov 2017 15:52:27 +0000 (10:52 -0500)]
Merge pull request #18673 from dzafman/wip-21833

osd: build_past_intervals_parallel: Ignore new partially created PGs

7 years agomgr/dashboard: handle null in format_number 18728/head
John Spray [Wed, 27 Sep 2017 13:33:58 +0000 (14:33 +0100)]
mgr/dashboard: handle null in format_number

Fixes: http://tracker.ceph.com/issues/21570
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f43859d4dca696ba4cac224e0c7e87e4b2aad40f)

7 years agomon: don't blow away bootstrap-mgr on upgrades 18727/head
John Spray [Thu, 19 Oct 2017 13:28:18 +0000 (09:28 -0400)]
mon: don't blow away bootstrap-mgr on upgrades

Fixes: http://tracker.ceph.com/issues/20950
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 57229ea2a4369518c7a16b7a09b045b7896f5a70)

7 years agomon: handle monitor lag when killing mgrs 18726/head
John Spray [Thu, 12 Oct 2017 10:57:50 +0000 (11:57 +0100)]
mon: handle monitor lag when killing mgrs

Fixes: http://tracker.ceph.com/issues/20629
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 3c3776b30a2da3f5d2ece15c57713c0ce191c778)

7 years agoosd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL 18723/head
Yao Zongyou [Tue, 31 Oct 2017 05:23:49 +0000 (13:23 +0800)]
osd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL

These two counters should be reported to mgr because the dashboard
plugin needing them to display osd detailed information.

Fixes: http://tracker.ceph.com/issues/21981
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit 40bb3c5f05b6f2a917a7e400f88677f41a9629a3)

7 years agopybind/mgr/mgr_module: fix calc_pg_upmaps
Sage Weil [Thu, 2 Nov 2017 21:18:03 +0000 (16:18 -0500)]
pybind/mgr/mgr_module: fix calc_pg_upmaps

This was fixed in the object cleanup.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ed442164d52c803916ca43a5ed76047089d3da66)

7 years agomgr/balancer: enable module by default
Sage Weil [Thu, 2 Nov 2017 21:11:26 +0000 (16:11 -0500)]
mgr/balancer: enable module by default

It will still be "off".

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 26710f0a9b176289237a52e5fa5894342ad163dc)

7 years agoqa/tasks/thrashosds: set min_in default to 4 18702/head
Sage Weil [Wed, 1 Nov 2017 13:31:31 +0000 (08:31 -0500)]
qa/tasks/thrashosds: set min_in default to 4

We have EC tests with k=2,m=2, so we need a min of 4.

Fixes: http://tracker.ceph.com/issues/21997
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d21809b14ea58dc1f44df844e407ebab5a315062)

7 years agoMerge pull request #18446 from theanalyst/wip-21854-luminous
Abhishek L [Fri, 3 Nov 2017 09:26:44 +0000 (10:26 +0100)]
Merge pull request #18446 from theanalyst/wip-21854-luminous

luminous: rgw_file: explicit NFSv3 open() emulation

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>
7 years agoosd: upmap should respect osd reweights 18699/head
Theofilos Mouratidis [Mon, 25 Sep 2017 14:17:47 +0000 (16:17 +0200)]
osd: upmap should respect osd reweights

Modify OSDMap::calc_pg_upmaps to take the osd reweight into account when
computing the size of each OSD.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
Fixes: http://tracker.ceph.com/issues/21538
(cherry picked from commit b8561cefa705d31954d47723125376a3c087f4f4)

7 years agotest/cli/osdmaptool: test upmap with an out osd
Theofilos Mouratidis [Mon, 25 Sep 2017 14:15:45 +0000 (16:15 +0200)]
test/cli/osdmaptool: test upmap with an out osd

Test upmap with one osd marked out.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit ff9e024a59d87eda6cbf3be9464ee7ae6f1140d7)

7 years agoosdmaptool: add --mark-out option
Theofilos Mouratidis [Mon, 25 Sep 2017 14:14:09 +0000 (16:14 +0200)]
osdmaptool: add --mark-out option

Add a --mark-down <id> option which sets the reweight for a given
osd id to 0.0.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit bd75a78187143f750f90fb937984eaa6333159d3)