From: Or Friedmann Date: Mon, 3 Jan 2022 16:32:16 +0000 (+0000) Subject: rgw: Add admin ops API for rate limiting X-Git-Tag: v17.1.0~99^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=fd084fd7fcf501596e76500b3009dbd29bb0e1bd;p=ceph.git rgw: Add admin ops API for rate limiting Add admin ops API for rate limiting and some bug fixes Signed-off-by: Or Friedmann --- diff --git a/PendingReleaseNotes b/PendingReleaseNotes index da2de5ed8d17..4f96dac6edbc 100644 --- a/PendingReleaseNotes +++ b/PendingReleaseNotes @@ -44,6 +44,13 @@ * MDS upgrades no longer require stopping all standby MDS daemons before upgrading the sole active MDS for a file system. +* RGW: RGW now supports rate limiting by user and/or by bucket. + With this feature it is possible to limit user and/or bucket, the total operations and/or + bytes per minute can be delivered. + This feature is allowing the admin to limit only READ operations and/or WRITE operations. + The rate limiting configuration could be applied on all users and all bucket by using + global configuration. + * RGW: `radosgw-admin realm delete` is now renamed to `radosgw-admin realm rm`. This is consistent with the help message. diff --git a/doc/radosgw/admin.rst b/doc/radosgw/admin.rst index fa00a5d03bca..731e8990f418 100644 --- a/doc/radosgw/admin.rst +++ b/doc/radosgw/admin.rst @@ -311,7 +311,7 @@ To add administrative capabilities to a user, execute the following:: You can add read, write or all capabilities to users, buckets, metadata and usage (utilization). For example:: - --caps="[users|buckets|metadata|usage|zone|amz-cache|info|bilog|mdlog|datalog|user-policy|oidc-provider|roles]=[*|read|write|read, write]" + --caps="[users|buckets|metadata|usage|zone|amz-cache|info|bilog|mdlog|datalog|user-policy|oidc-provider|roles|ratelimit]=[*|read|write|read, write]" For example:: @@ -513,7 +513,7 @@ After userA completes this 1GB operation, the RGW will block the user request fo - **Maximum Write Bytes:** The ``--max-write-bytes`` setting allows you to specify the maximum number of write bytes per minute per RGW. A 0 value disables this setting (which means unlimited access). -- **Rate Limit Scope:** The ``--ratelimit-scope `` option sets the scope for the rate limit. +- **Rate Limit Scope:** The ``--ratelimit-scope`` option sets the scope for the rate limit. The options are ``bucket`` , ``user`` and ``anonymous``. Bucket rate limit apply to buckets. The user rate limit applies to a user. Anonymous applies to an unauthenticated user. Anonymous scope is only available for global rate limit. @@ -542,7 +542,7 @@ Get User Rate Limit Get the current configured rate limit parameters For example:: - radosgw-admin ratelimit set --ratelimit-scope=user --uid= + radosgw-admin ratelimit get --ratelimit-scope=user --uid= For example:: @@ -621,7 +621,7 @@ view the global rate limit settings:: The global rate limit settings can be manipulated with the ``global ratelimit`` counterparts of the ``ratelimit set``, ``ratelimit enable``, and ``ratelimit disable`` -commands. ``Per user and per bucket ratelimit configuration is overriding the global configuration``:: +commands. Per user and per bucket ratelimit configuration is overriding the global configuration:: radosgw-admin global ratelimit set --ratelimit-scope bucket --max-read-ops=1024 radosgw-admin global ratelimit enable --ratelimit-scope bucket @@ -637,7 +637,7 @@ The global rate limit can configure rate limit scope for all unauthenticated use radosgw-admin global ratelimit enable --ratelimit-scope=anonymous .. note:: In a multisite configuration, where there is a realm and period - present, changes to the global quotas must be committed using ``period + present, changes to the global rate limit must be committed using ``period update --commit``. If there is no period present, the rados gateway(s) must be restarted for the changes to take effect. diff --git a/doc/radosgw/adminops.rst b/doc/radosgw/adminops.rst index 1752027e6e00..b3e0a11832b5 100644 --- a/doc/radosgw/adminops.rst +++ b/doc/radosgw/adminops.rst @@ -1977,6 +1977,124 @@ as mentioned in Set Bucket Quota section above. +Rate Limit +========== + +The Admin Operations API enables you to set and get ratelimit configurations on users and on bucket and global rate limit configurations. See `Rate Limit Management`_ for additional details. +Rate Limit includes the maximum number of operations and/or bytes per minute, separated by read and/or write, to a bucket and/or by a user and the maximum storage size in megabytes. + +To view rate limit, the user must have a ``ratelimit=read`` capability. To set, +modify or disable a ratelimit, the user must have ``ratelimit=write`` capability. +See the `Admin Guide`_ for details. + +Valid parameters for quotas include: + +- **Bucket:** The ``bucket`` option allows you to specify a rate limit for + a bucket. + +- **User:** The ``uid`` option allows you to specify a rate limit for a user. + +- **Maximum Read Bytes:** The ``max-read-bytes`` setting allows you to specify + the maximum number of read bytes per minute. A 0 value disables this setting. + +- **Maximum Write Bytes:** The ``max-write-bytes`` setting allows you to specify + the maximum number of write bytes per minute. A 0 value disables this setting. + +- **Maximum Read Ops:** The ``max-read-ops`` setting allows you to specify + the maximum number of read ops per minute. A 0 value disables this setting. + +- **Maximum Write Ops:** The ``max-write-ops`` setting allows you to specify + the maximum number of write ops per minute. A 0 value disables this setting. + +- **Global:** The ``global`` option allows you to specify a global rate limit. + The value should be either 'True' or 'False'. + +- **Rate Limit Scope:** The ``ratelimit-scope`` option sets the scope for the rate limit. + The options are ``bucket`` , ``user`` and ``anonymous``. + ``anonymous`` is only valid for setting global configuration + +- **Enable/Disable Rate Limit:** The ``enabled`` option specifies whether the + rate limit should be enabled. The value should be either 'True' or 'False'. + +Get User Rate Limit +~~~~~~~~~~~~~~~~~~~ + +To get a rate limit, the user must have ``ratelimit`` capability set with ``read`` +permission. :: + + GET /{admin}/ratelimit?ratelimit-scope=user&uid= + + +Set User Rate Limit +~~~~~~~~~~~~~~~~~~~ + +To set a rate limit, the user must have ``ratelimit`` capability set with ``write`` +permission. :: + + POST /{admin}/ratelimit?ratelimit-scope=user&uid=<[&max-read-bytes=][&max-write-bytes=][&max-read-ops=][&max-write-ops=][enabled=]> + + + +Get Bucket Rate Limit +~~~~~~~~~~~~~~~~~~~~~ + +To get a rate limit, the user must have ``users`` capability set with ``read`` +permission. :: + + GET /{admin}/ratelimit?bucket=&ratelimit-scope=bucket + + + +Set Rate Limit for an Individual Bucket +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To set a rate limit, the user must have ``ratelimit`` capability set with ``write`` +permission. :: + + POST /{admin}/ratelimit?bucket=&ratelimit-scope=bucket<[&max-read-bytes=][&max-write-bytes=][&max-read-ops=][&max-write-ops=]> + + + +Get Global Rate Limit +~~~~~~~~~~~~~~~~~~~~~ + +To get a global rate limit, the user must have ``ratelimit`` capability set with ``read`` +permission. :: + + GET /{admin}/ratelimit?global= + + + +Set Global User Rate Limit +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To set a rate limit, the user must have ``ratelimit`` capability set with ``write`` +permission. :: + + POST /{admin}/ratelimit?ratelimit-scope=user&global=<[&max-read-bytes=][&max-write-bytes=][&max-read-ops=][&max-write-ops=][enabled=]> + + + +Set Global Rate Limit Bucket +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To set a rate limit, the user must have ``ratelimit`` capability set with ``write`` +permission. :: + + POST /{admin}/ratelimit?ratelimit-scope=bucket&global=<[&max-read-bytes=][&max-write-bytes=][&max-read-ops=][&max-write-ops=]> + + + +Set Global Anonymous User Rate Limit +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To set a rate limit, the user must have ``ratelimit`` capability set with ``write`` +permission. :: + + POST /{admin}/ratelimit?ratelimit-scope=anon&global=<[&max-read-bytes=][&max-write-bytes=][&max-read-ops=][&max-write-ops=][enabled=]> + + + Standard Error Responses ======================== @@ -2029,6 +2147,7 @@ Binding libraries .. _Admin Guide: ../admin .. _Quota Management: ../admin#quota-management +.. _Rate Limit Management: ../admin#rate-limit-management .. _IrekFasikhov/go-rgwadmin: https://github.com/IrekFasikhov/go-rgwadmin .. _QuentinPerez/go-radosgw: https://github.com/QuentinPerez/go-radosgw .. _twonote/radosgw-admin4j: https://github.com/twonote/radosgw-admin4j diff --git a/qa/tasks/radosgw_admin_rest.py b/qa/tasks/radosgw_admin_rest.py index 4e40a98f9a6e..3de4d6bc9258 100644 --- a/qa/tasks/radosgw_admin_rest.py +++ b/qa/tasks/radosgw_admin_rest.py @@ -69,6 +69,8 @@ def rgwadmin_rest(connection, cmd, params=None, headers=None, raw=False): return 'usage', '' elif cmd[0] == 'info': return 'info', '' + elif cmd[0] == 'ratelimit': + return 'ratelimit', '' elif cmd[0] == 'zone' or cmd[0] in zone_sub_resources: if cmd[0] == 'zone': return 'zone', '' @@ -139,10 +141,11 @@ def task(ctx, config): admin_display_name = 'Ms. Admin User' admin_access_key = 'MH1WC2XQ1S8UISFDZC8W' admin_secret_key = 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG' - admin_caps = 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read' + admin_caps = 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' user1 = 'foo' user2 = 'fud' + ratelimit_user = 'ratelimit_user' subuser1 = 'foo:foo1' subuser2 = 'foo:foo2' display_name1 = 'Foo' @@ -733,3 +736,80 @@ def task(ctx, config): assert len(name) > 0 # fsid is a uuid, but I'm not going to try to parse it assert len(fsid) > 0 + + # TESTCASE 'ratelimit' 'user' 'info' 'succeeds' + (ret, out) = rgwadmin_rest(admin_conn, + ['user', 'create'], + {'uid' : ratelimit_user, + 'display-name' : display_name1, + 'email' : email, + 'access-key' : access_key, + 'secret-key' : secret_key, + 'max-buckets' : '1000' + }) + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'user', 'uid' : ratelimit_user}) + assert ret == 200 + + # TESTCASE 'ratelimit' 'user' 'info' 'not existing user' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'user', 'uid' : ratelimit_user + 'string'}) + assert ret == 404 + + # TESTCASE 'ratelimit' 'user' 'info' 'uid not specified' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'user'}) + assert ret == 400 + + # TESTCASE 'ratelimit' 'bucket' 'info' 'succeeds' + ratelimit_bucket = 'ratelimitbucket' + connection.create_bucket(ratelimit_bucket) + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'bucket', 'bucket' : ratelimit_bucket}) + assert ret == 200 + + # TESTCASE 'ratelimit' 'bucket' 'info' 'not existing bucket' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'bucket', 'bucket' : ratelimit_bucket + 'string'}) + assert ret == 404 + + # TESTCASE 'ratelimit' 'bucket' 'info' 'bucket not specified' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'bucket'}) + assert ret == 400 + + # TESTCASE 'ratelimit' 'global' 'info' 'succeeds' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'global' : 'true'}) + assert ret == 200 + + # TESTCASE 'ratelimit' 'user' 'modify' 'not existing user' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'user', 'uid' : ratelimit_user + 'string', 'enabled' : 'true'}) + assert ret == 404 + + # TESTCASE 'ratelimit' 'user' 'modify' 'uid not specified' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'user'}) + assert ret == 400 + + # TESTCASE 'ratelimit' 'bucket' 'modify' 'not existing bucket' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'bucket', 'bucket' : ratelimit_bucket + 'string', 'enabled' : 'true'}) + assert ret == 404 + + # TESTCASE 'ratelimit' 'bucket' 'modify' 'bucket not specified' 'fails' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'bucket', 'enabled' : 'true'}) + assert ret == 400 + + # TESTCASE 'ratelimit' 'user' 'modifiy' 'enabled' 'max-read-bytes = 2' 'succeeds' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'user', 'uid' : ratelimit_user, 'enabled' : 'true', 'max-read-bytes' : '2'}) + assert ret == 200 + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'user', 'uid' : ratelimit_user}) + assert ret == 200 + user_ratelimit = out['user_ratelimit'] + assert user_ratelimit['enabled'] == True + assert user_ratelimit['max_read_bytes'] == 2 + + # TESTCASE 'ratelimit' 'bucket' 'modifiy' 'enabled' 'max-write-bytes = 2' 'succeeds' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'bucket', 'bucket' : ratelimit_bucket, 'enabled' : 'true', 'max-write-bytes' : '2'}) + assert ret == 200 + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'info'], {'ratelimit-scope' : 'bucket', 'bucket' : ratelimit_bucket}) + assert ret == 200 + bucket_ratelimit = out['bucket_ratelimit'] + assert bucket_ratelimit['enabled'] == True + assert bucket_ratelimit['max_write_bytes'] == 2 + + # TESTCASE 'ratelimit' 'global' 'modify' 'anonymous' 'enabled' 'succeeds' + (ret, out) = rgwadmin_rest(admin_conn, ['ratelimit', 'modify'], {'ratelimit-scope' : 'bucket', 'global': 'true', 'enabled' : 'true'}) + assert ret == 200 \ No newline at end of file diff --git a/src/rgw/CMakeLists.txt b/src/rgw/CMakeLists.txt index d53dad434dff..b050beb1cd4f 100644 --- a/src/rgw/CMakeLists.txt +++ b/src/rgw/CMakeLists.txt @@ -269,6 +269,7 @@ set(rgw_a_srcs rgw_rest_usage.cc rgw_rest_info.cc rgw_rest_user.cc + rgw_rest_ratelimit.cc rgw_swift_auth.cc rgw_usage.cc rgw_opa.cc diff --git a/src/rgw/rgw_admin.cc b/src/rgw/rgw_admin.cc index f72e6f7204fd..063da4ff922d 100644 --- a/src/rgw/rgw_admin.cc +++ b/src/rgw/rgw_admin.cc @@ -1304,7 +1304,6 @@ bool set_ratelimit_info(RGWRateLimitInfo& ratelimit, OPT opt_cmd, int64_t max_re ratelimit.enabled = true; break; - case OPT::RATELIMIT_SET: case OPT::GLOBAL_RATELIMIT_SET: ratelimit_configured = false; @@ -6766,19 +6765,6 @@ int main(int argc, const char **argv) cerr << "failure: " << cpp_strerror(-r) << ": " << err << std::endl; return -r; } - RGWRateLimitInfo ratelimit_info; - std::unique_ptr bucket_sal; - auto iter = bucket->get_attrs().find(RGW_ATTR_RATELIMIT); - if(iter != bucket->get_attrs().end()) { - try { - std::cerr << "here" << std::endl; - bufferlist& bl = iter->second; - auto biter = bl.cbegin(); - decode(ratelimit_info, biter); - } catch (buffer::error& err) { - cerr << "ERROR: failed to decode rate limit" << std::endl; - } - } } if (opt_cmd == OPT::BUCKET_LINK) { diff --git a/src/rgw/rgw_common.cc b/src/rgw/rgw_common.cc index 5262460905c8..3aef32f51212 100644 --- a/src/rgw/rgw_common.cc +++ b/src/rgw/rgw_common.cc @@ -2015,7 +2015,8 @@ bool RGWUserCaps::is_valid_cap_type(const string& tp) "roles", "user-policy", "amz-cache", - "oidc-provider"}; + "oidc-provider", + "ratelimit"}; for (unsigned int i = 0; i < sizeof(cap_type) / sizeof(char *); ++i) { if (tp.compare(cap_type[i]) == 0) { diff --git a/src/rgw/rgw_main.cc b/src/rgw/rgw_main.cc index a932433e8c41..8d6555a2b7fa 100644 --- a/src/rgw/rgw_main.cc +++ b/src/rgw/rgw_main.cc @@ -32,6 +32,7 @@ #include "rgw_rest_config.h" #include "rgw_rest_realm.h" #include "rgw_rest_sts.h" +#include "rgw_rest_ratelimit.h" #include "rgw_swift_auth.h" #include "rgw_log.h" #include "rgw_tools.h" @@ -512,6 +513,7 @@ int radosgw_Main(int argc, const char **argv) admin_resource->register_resource("log", new RGWRESTMgr_Log); admin_resource->register_resource("config", new RGWRESTMgr_Config); admin_resource->register_resource("realm", new RGWRESTMgr_Realm); + admin_resource->register_resource("ratelimit", new RGWRESTMgr_Ratelimit); rest.register_resource(g_conf()->rgw_admin_entry, admin_resource); } diff --git a/src/rgw/rgw_rest_ratelimit.cc b/src/rgw/rgw_rest_ratelimit.cc new file mode 100644 index 000000000000..5820ab71abde --- /dev/null +++ b/src/rgw/rgw_rest_ratelimit.cc @@ -0,0 +1,349 @@ +// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*- +// vim: ts=8 sw=2 smarttab ft=cpp +#include "rgw_rest_ratelimit.h" +class RGWOp_Ratelimit_Info : public RGWRESTOp { +int check_caps(const RGWUserCaps& caps) override { + return caps.check_cap("ratelimit", RGW_CAP_READ); +} + + void execute(optional_yield y) override; + + const char* name() const override { return "get_ratelimit_info"; } +}; +void RGWOp_Ratelimit_Info::execute(optional_yield y) +{ + ldpp_dout(this, 20) << "" << dendl; + std::string uid_str; + std::string ratelimit_scope; + std::string bucket_name; + std::string tenant_name; + bool global = false; + RESTArgs::get_string(s, "uid", uid_str, &uid_str); + RESTArgs::get_string(s, "ratelimit-scope", ratelimit_scope, &ratelimit_scope); + RESTArgs::get_string(s, "bucket", bucket_name, &bucket_name); + RESTArgs::get_string(s, "tenant", tenant_name, &tenant_name); + // RESTArgs::get_bool default value to true even if global is empty + bool exists; + std::string sval = s->info.args.get("global", &exists); + if (exists) { + if (!boost::iequals(sval,"true") && !boost::iequals(sval,"false")) { + op_ret = -EINVAL; + ldpp_dout(this, 20) << "global is not equal to true or false" << dendl; + return; + } + } + RESTArgs::get_bool(s, "global", false, &global); + + if (ratelimit_scope == "bucket" && !bucket_name.empty() && !global) { + std::unique_ptr bucket; + int r = store->get_bucket(s, nullptr, tenant_name, bucket_name, &bucket, y); + if (r != 0) { + op_ret = r; + ldpp_dout(this, 0) << "Error on getting bucket info" << dendl; + return; + } + RGWRateLimitInfo ratelimit_info; + auto iter = bucket->get_attrs().find(RGW_ATTR_RATELIMIT); + if (iter != bucket->get_attrs().end()) { + try { + bufferlist& bl = iter->second; + auto biter = bl.cbegin(); + decode(ratelimit_info, biter); + } catch (buffer::error& err) { + ldpp_dout(this, 0) << "Error on decoding ratelimit info from bucket" << dendl; + op_ret = -EIO; + return; + } + } + flusher.start(0); + s->formatter->open_object_section("bucket_ratelimit"); + encode_json("bucket_ratelimit", ratelimit_info, s->formatter); + s->formatter->close_section(); + flusher.flush(); + return; + } + if (ratelimit_scope == "user" && !uid_str.empty() && !global) { + RGWRateLimitInfo ratelimit_info; + rgw_user user(uid_str); + std::unique_ptr user_sal; + user_sal = store->get_user(user); + if (!rgw::sal::User::empty(user_sal)) { + op_ret = user_sal->load_user(this, y); + if (op_ret) { + ldpp_dout(this, 0) << "Cannot load user info" << dendl; + return; + } + } else { + ldpp_dout(this, 0) << "User does not exist" << dendl; + op_ret = -ENOENT; + return; + } + + auto iter = user_sal->get_attrs().find(RGW_ATTR_RATELIMIT); + if(iter != user_sal->get_attrs().end()) { + try { + bufferlist& bl = iter->second; + auto biter = bl.cbegin(); + decode(ratelimit_info, biter); + } catch (buffer::error& err) { + ldpp_dout(this, 0) << "Error on decoding ratelimit info from user" << dendl; + op_ret = -EIO; + return; + } + } + flusher.start(0); + s->formatter->open_object_section("user_ratelimit"); + encode_json("user_ratelimit", ratelimit_info, s->formatter); + s->formatter->close_section(); + flusher.flush(); + } + if (global) { + std::string realm_id = store->get_zone()->get_realm().get_id(); + RGWPeriodConfig period_config; + op_ret = period_config.read(this, static_cast(store)->svc()->sysobj, realm_id, y); + if (op_ret && op_ret != -ENOENT) { + ldpp_dout(this, 0) << "Error on period config read" << dendl; + return; + } + flusher.start(0); + s->formatter->open_object_section("period_config"); + encode_json("bucket_ratelimit", period_config.bucket_ratelimit, s->formatter); + encode_json("user_ratelimit", period_config.user_ratelimit, s->formatter); + encode_json("anonymous_ratelimit", period_config.anon_ratelimit, s->formatter); + s->formatter->close_section(); + flusher.flush(); + return; + } + op_ret = -EINVAL; + return; +} + +class RGWOp_Ratelimit_Set : public RGWRESTOp { + int check_caps(const RGWUserCaps& caps) override { + return caps.check_cap("ratelimit", RGW_CAP_WRITE); + } + + void execute(optional_yield y) override; + + const char* name() const override { return "put_ratelimit_info"; } + + void set_ratelimit_info(bool have_max_read_ops, int64_t max_read_ops, bool have_max_write_ops, int64_t max_write_ops, + bool have_max_read_bytes, int64_t max_read_bytes, bool have_max_write_bytes, int64_t max_write_bytes, + bool have_enabled, bool enabled, bool& ratelimit_configured, RGWRateLimitInfo& ratelimit_info); +}; + + + void RGWOp_Ratelimit_Set::set_ratelimit_info(bool have_max_read_ops, int64_t max_read_ops, bool have_max_write_ops, int64_t max_write_ops, + bool have_max_read_bytes, int64_t max_read_bytes, bool have_max_write_bytes, int64_t max_write_bytes, + bool have_enabled, bool enabled, bool& ratelimit_configured, RGWRateLimitInfo& ratelimit_info) + { + if (have_max_read_ops) { + if (max_read_ops >= 0) { + ratelimit_info.max_read_ops = max_read_ops; + ratelimit_configured = true; + } + } + if (have_max_write_ops) { + if (max_write_ops >= 0) { + ratelimit_info.max_write_ops = max_write_ops; + ratelimit_configured = true; + } + } + if (have_max_read_bytes) { + if (max_read_bytes >= 0) { + ratelimit_info.max_read_bytes = max_read_bytes; + ratelimit_configured = true; + } + } + if (have_max_write_bytes) { + if (max_write_bytes >= 0) { + ratelimit_info.max_write_bytes = max_write_bytes; + ratelimit_configured = true; + } + } + if (have_enabled) { + ratelimit_info.enabled = enabled; + ratelimit_configured = true; + } + if (!ratelimit_configured) { + ldpp_dout(this, 0) << "No rate limit configuration arguments have been sent" << dendl; + op_ret = -EINVAL; + return; + } + + } + + +void RGWOp_Ratelimit_Set::execute(optional_yield y) +{ + std::string uid_str; + std::string ratelimit_scope; + std::string bucket_name; + std::string tenant_name; + RGWRateLimitInfo ratelimit_info; + bool ratelimit_configured = false; + bool enabled = false; + bool have_enabled = false; + bool global = false; + int64_t max_read_ops = 0; + bool have_max_read_ops = false; + int64_t max_write_ops = 0; + bool have_max_write_ops = false; + int64_t max_read_bytes = 0; + bool have_max_read_bytes = false; + int64_t max_write_bytes = 0; + bool have_max_write_bytes = false; + RESTArgs::get_string(s, "uid", uid_str, &uid_str); + RESTArgs::get_string(s, "ratelimit-scope", ratelimit_scope, &ratelimit_scope); + RESTArgs::get_string(s, "bucket", bucket_name, &bucket_name); + RESTArgs::get_string(s, "tenant", tenant_name, &tenant_name); + // check there was no -EINVAL coming from get_int64 + op_ret = RESTArgs::get_int64(s, "max-read-ops", 0, &max_read_ops, &have_max_read_ops); + op_ret |= RESTArgs::get_int64(s, "max-write-ops", 0, &max_write_ops, &have_max_write_ops); + op_ret |= RESTArgs::get_int64(s, "max-read-bytes", 0, &max_read_bytes, &have_max_read_bytes); + op_ret |= RESTArgs::get_int64(s, "max-write-bytes", 0, &max_write_bytes, &have_max_write_bytes); + if (op_ret) { + ldpp_dout(this, 0) << "one of the maximum arguments could not be parsed" << dendl; + return; + } + // RESTArgs::get_bool default value to true even if enabled or global are empty + std::string sval = s->info.args.get("enabled", &have_enabled); + if (have_enabled) { + if (!boost::iequals(sval,"true") && !boost::iequals(sval,"false")) { + ldpp_dout(this, 20) << "enabled is not equal to true or false" << dendl; + op_ret = -EINVAL; + return; + } + } + RESTArgs::get_bool(s, "enabled", false, &enabled, &have_enabled); + bool exists; + sval = s->info.args.get("global", &exists); + if (exists) { + if (!boost::iequals(sval,"true") && !boost::iequals(sval,"false")) { + ldpp_dout(this, 20) << "global is not equal to true or faslse" << dendl; + op_ret = -EINVAL; + return; + } + } + RESTArgs::get_bool(s, "global", false, &global, nullptr); + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + if (op_ret) { + return; + } + if (ratelimit_scope == "user" && !uid_str.empty() && !global) { + rgw_user user(uid_str); + std::unique_ptr user_sal; + user_sal = store->get_user(user); + if (!rgw::sal::User::empty(user_sal)) { + op_ret = user_sal->load_user(this, y); + if (op_ret) { + ldpp_dout(this, 0) << "Cannot load user info" << dendl; + return; + } + } else { + ldpp_dout(this, 0) << "User does not exist" << dendl; + op_ret = -ENOENT; + return; + } + auto iter = user_sal->get_attrs().find(RGW_ATTR_RATELIMIT); + if (iter != user_sal->get_attrs().end()) { + try { + bufferlist& bl = iter->second; + auto biter = bl.cbegin(); + decode(ratelimit_info, biter); + } catch (buffer::error& err) { + ldpp_dout(this, 0) << "Error on decoding ratelimit info from user" << dendl; + op_ret = -EIO; + return; + } + } + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + bufferlist bl; + ratelimit_info.encode(bl); + rgw::sal::Attrs attr; + attr[RGW_ATTR_RATELIMIT] = bl; + op_ret = user_sal->merge_and_store_attrs(this, attr, y); + return; + } + + if (ratelimit_scope == "bucket" && !bucket_name.empty() && !global) { + ldpp_dout(this, 0) << "getting bucket info" << dendl; + std::unique_ptr bucket; + op_ret = store->get_bucket(this, nullptr, tenant_name, bucket_name, &bucket, y); + if (op_ret) { + ldpp_dout(this, 0) << "Error on getting bucket info" << dendl; + return; + } + auto iter = bucket->get_attrs().find(RGW_ATTR_RATELIMIT); + if (iter != bucket->get_attrs().end()) { + try { + bufferlist& bl = iter->second; + auto biter = bl.cbegin(); + decode(ratelimit_info, biter); + } catch (buffer::error& err) { + ldpp_dout(this, 0) << "Error on decoding ratelimit info from bucket" << dendl; + op_ret = -EIO; + return; + } + } + bufferlist bl; + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + ratelimit_info.encode(bl); + rgw::sal::Attrs attr; + attr[RGW_ATTR_RATELIMIT] = bl; + op_ret = bucket->merge_and_store_attrs(this, attr, y); + return; + } + if (global) { + std::string realm_id = store->get_zone()->get_realm().get_id(); + RGWPeriodConfig period_config; + op_ret = period_config.read(s, static_cast(store)->svc()->sysobj, realm_id, y); + if (op_ret && op_ret != -ENOENT) { + ldpp_dout(this, 0) << "Error on period config read" << dendl; + return; + } + if (ratelimit_scope == "bucket") { + ratelimit_info = period_config.bucket_ratelimit; + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + period_config.bucket_ratelimit = ratelimit_info; + op_ret = period_config.write(s, static_cast(store)->svc()->sysobj, realm_id, y); + return; + } + if (ratelimit_scope == "anon") { + ratelimit_info = period_config.anon_ratelimit; + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + period_config.anon_ratelimit = ratelimit_info; + op_ret = period_config.write(s, static_cast(store)->svc()->sysobj, realm_id, y); + return; + } + if (ratelimit_scope == "user") { + ratelimit_info = period_config.user_ratelimit; + set_ratelimit_info(have_max_read_ops, max_read_ops, have_max_write_ops, max_write_ops, + have_max_read_bytes, max_read_bytes, have_max_write_bytes, max_write_bytes, + have_enabled, enabled, ratelimit_configured, ratelimit_info); + period_config.user_ratelimit = ratelimit_info; + op_ret = period_config.write(s, static_cast(store)->svc()->sysobj, realm_id, y); + return; + } + } + op_ret = -EINVAL; + return; +} +RGWOp* RGWHandler_Ratelimit::op_get() +{ + return new RGWOp_Ratelimit_Info; +} +RGWOp* RGWHandler_Ratelimit::op_post() +{ + return new RGWOp_Ratelimit_Set; +} \ No newline at end of file diff --git a/src/rgw/rgw_rest_ratelimit.h b/src/rgw/rgw_rest_ratelimit.h new file mode 100644 index 000000000000..e1f14a594f17 --- /dev/null +++ b/src/rgw/rgw_rest_ratelimit.h @@ -0,0 +1,34 @@ +// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*- +// vim: ts=8 sw=2 smarttab ft=cpp + +#pragma once + +#include "rgw_rest.h" +#include "rgw_rest_s3.h" +#include "rgw_sal_rados.h" + +class RGWHandler_Ratelimit : public RGWHandler_Auth_S3 { +protected: + RGWOp *op_get() override; + RGWOp *op_post() override; +public: + using RGWHandler_Auth_S3::RGWHandler_Auth_S3; + ~RGWHandler_Ratelimit() override = default; + + int read_permissions(RGWOp*, optional_yield) override { + return 0; + } +}; + +class RGWRESTMgr_Ratelimit : public RGWRESTMgr { +public: + RGWRESTMgr_Ratelimit() = default; + ~RGWRESTMgr_Ratelimit() override = default; + + RGWHandler_REST *get_handler(rgw::sal::Store* store, + struct req_state*, + const rgw::auth::StrategyRegistry& auth_registry, + const std::string&) override { + return new RGWHandler_Ratelimit(auth_registry); + } +}; \ No newline at end of file diff --git a/src/rgw/rgw_rest_user.cc b/src/rgw/rgw_rest_user.cc index d6297f547231..e715752142d6 100644 --- a/src/rgw/rgw_rest_user.cc +++ b/src/rgw/rgw_rest_user.cc @@ -455,7 +455,6 @@ void RGWOp_Subuser_Create::execute(optional_yield y) RESTArgs::get_string(s, "secret-key", secret_key, &secret_key); RESTArgs::get_string(s, "access", perm_str, &perm_str); RESTArgs::get_string(s, "key-type", key_type_str, &key_type_str); - //RESTArgs::get_bool(s, "generate-subuser", false, &gen_subuser); RESTArgs::get_bool(s, "generate-secret", false, &gen_secret); RESTArgs::get_bool(s, "gen-access-key", false, &gen_access);