[Openid-specs-fastfed] Revised SCIM interop profile
McAdams, Darin
darinm at amazon.com
Thu Feb 20 06:02:41 UTC 2020
Thanks for all the feedback on the FastFed SCIM profile. (Was able to catchup with Karl and Zhen on Tuesday.)
I think we’re at consensus on the big questions. I’ve uploaded a new version and pasted a copy in this email.
https://drive.google.com/file/d/18XTUVXIG-AC3l_xoISkr9R-zV_69C6il/view?usp=sharing
Based on this sketch, I’ll begin writing the normative version.
Key changes were to:
* Update User
* Deactivate/Delete User
* Deactivate/Delete Group
* Add/Remove Group Members
Please also check if the profile is missing operations you need for provisioning.
Examples of operations NOT currently in the profile:
* Get all Users in Directory
* Get all Groups in Directory
* Get all Users in Group
* Get all Groups for User
-Darin
---------------------------------------------------
SCIM Interop Requirements V2
Updated Feb 19
This is a non-normative sketch of the interop profile, incorporating WG feedback.
User Operations
________________________________
Create User
SCIM Operation: POST /Users
Example:
POST /Users HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas":[
"urn:ietf:params:scim:schemas:core:2.0:User",
"urn:ietf:params:scim:schemas:extension:enterprise:2.0:User"
],
"externalId": "98d78581-dd0d-4361-ab61-9511c6e5f035",
"userName":"bjensen",
"name":{
"formatted":"Ms. Barbara J Jensen III",
"familyName":"Jensen",
"givenName":"Barbara"
}
"urn:ietf:params:scim:schemas:extension:enterprise:2.0:User": {
"costCenter":"12345",
"manager":{"value": "0cca76a8-090a-4944-8e61-e7791e619d48"}
}
}
Notes:
* The “groups” attribute on the User object should NOT be included. Application Providers should ignore it, if it exists.
* If multiple values are provided for “emails”,
“phoneNumbers”, or “addresses”, then one must have “primary=true”. If only a single value is provided, the Application may treat the single value as the primary if not explicitly labeled as such.
* References to other resources, such as the “manager” attribute in the EnterpriseUser object, must contain the “id” of the referenced object in the SCIM server. The referenced object must be created in advance. (Technical note: this precludes circular references.)
* If a value for “password” is sent, and the Application doesn’t need a password (because it relies on authentication via the IdP), it should ignore the password. (This is essentially Postel’s law. In the normative spec, can reformulate in a general sense. E.g. Ignore anything you didn’t ask for & don’t need.)
Update User
SCIM Operation: PATCH /Users/{id}
Example:
PATCH /Users/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "replace"
"path": "name.formatted”
"value": "Babs Jensen"
},
{
"op": "replace"
"path": "addresses[type eq \"work\"].streetAddress”
"value": "1010 Broadway Ave"
},
]
}
Notes:
· The original version of this document proposed doing all Updates via the PUT operation.
o Feedback from WG members, including those who currently use PUT, was that it was preferable to move away from that model, instead PATCH’ing only the modified attributes.
o The reasoning is that there have been challenges with the PUT model accidentally overwriting information that the IdP shouldn’t touch.
o Implementing PUTs safely requires reading the existing entry, overlaying the changes, and then doing a conditional write back to the service with ETag semantics. This drives more traffic to the App, and requires them to implement ETags.
o As an aside, in order to simplify the burden of handling PATCH requests, it would be nice if there was a widely adopted Open Source implementation for SCIM Users & Groups that would handle the complicated logic of mutating object contents according to the SCIM rules for add/replace/remove.
· SCIM gives the option to return the full object (via an HTTP 200) or just an empty ACK (via an HTTP 204). FastFed will specify the HTTP 204 approach for update operations.
Deactivate/Reactivate User
This is a special case of updating a user.
SCIM Operation: PATCH /Users/{id}
Example:
PATCH /Users/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [{
"op": "replace"
"path": "active”
"value": false
}]
}
Notes:
· Deactivation will prevent the user from continuing to use the Application. From the existing spec: “Application Providers SHOULD respond to account deactivation by revoking the ability for the end-user to use the Application, even if that end-user has an active session with an expiration date in the future. The revocation mechanism is an implementation detail and outside the scope of this specification. Revocation should occur within 5 minutes of receiving the deactivation”
· Must be able to reactivate the User.
· It’s OK to leave a User in the deactivated status in perpetuity.
Delete User
SCIM Operation: DELETE /Users/{id}
Example:
DELETE /Users/{id}
Host: example.com
Authorization: Bearer h480djs93hd8
Notes:
· After a user is deleted, it must be possible to create a fresh user with the same username.
· It’s the choice of the IdP to determine when to invoke delete. As a reference to implementors, a best practice is to first Deactivate the user, wait for an undo window, and then permanently Delete if needed, such as when recycling a username. It’s OK to never delete, too, if that’s the IdP preference.
Get User By Id
SCIM Operation: GET /Users/{id}
Example:
GET /Users/{id}
Host: example.com
Accept: application/scim+json
Authorization: Bearer h480djs93hd8
HTTP/1.1 200 OK
Content-Type: application/scim+json
{
"schemas":["urn:ietf:params:scim:schemas:core:2.0:User"],
"meta":{
"resourceType":"User",
"created":"2011-08-01T18:29:49.793Z",
"lastModified":"2011-08-01T18:29:49.793Z",
"version":"W\/\"f250dd84f0671c3\""
},
"id":"2819c223-7f76-453a-919d-413861904646",
"externalId": "98d78581-dd0d-4361-ab61-9511c6e5f035",
"userName":"bjensen",
"name":{
"formatted":"Ms. Barbara J Jensen III",
"familyName":"Jensen",
"givenName":"Barbara"
},
"emails":[
{
"value":"bjensen at example.com",
"type":"work"
}
]
}
Notes:
* The “groups” attribute on the User object should NOT be included. Identity Providers should ignore it, if it exists.
Get User By Alternate Key
The SCIM Service must support looking up Users based on the following filters:
* “username eq”
* “emails[primary eq true].value eq” (if email is defined for Users)
* “externalId eq” (If the IdentityProvider has provided a value for externalId)
SCIM Operation: GET /Users?filter=...
Example:
GET /Users?filter=userName+eq+{username}
Host: example.com
Accept: application/scim+json
Authorization: Bearer h480djs93hd8
HTTP/1.1 200 OK
Content-Type: application/scim+json
{
"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"totalResults":1,
"Resources":[
{
"schemas":["urn:ietf:params:scim:schemas:core:2.0:User"],
"meta":{
"resourceType":"User",
"created":"2011-08-01T18:29:49.793Z",
"lastModified":"2011-08-01T18:29:49.793Z",
"version":"W\/\"f250dd84f0671c3\""
},
"id":"2819c223-7f76-453a-919d-413861904646",
"externalId": "98d78581-dd0d-4361-ab61-9511c6e5f035",
"userName":"bjensen",
"name":{
"formatted":"Ms. Barbara J Jensen III",
"familyName":"Jensen",
"givenName":"Barbara"
},
"emails":[
{
"value":"bjensen at example.com",
"type":"work"
}
]
}
]
}
All Other User Operations
Other operations are out of scope for SCIM provisioning between an Identity Provider and Application Provider.
Applications may support additional functionality, but it is not required for FastFed Compatibility.
Group Operations
________________________________
Create Group
SCIM Operation: POST /Groups
Example:
POST /Groups HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:scim:schemas:core:1.0"],
"externalId": "e5a41517-bcd6-4b8b-8590-487ae996de44",
"displayName": "Group Name"
}
Notes:
* DisplayName is required
* ExternalId is optional
* Members must be empty/undefined. (Technical note: The reasoning here is two-fold. First, some implementations can store group metadata separately from group memberships, making it difficult to adhere to the atomic update requirements for POST and PATCH operations on a two-phase commit. Second, even if that’s not the case, SCIM doesn’t provide any guidance (or way to communicate) the maximum number of membership changes in a single POST/PATCH, such that even those with transactional capabilities may only be capable of performing a limited quantity of work within a transactional unit. See later sections for recommended handling of group membership.)
Update Group Metadata
SCIM Operation: PATCH /Groups/{id}
Example:
PATCH /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "replace",
"path": "externalId",
"value": "{newExternalId}"
},
{
"op": "replace",
"path": "displayName",
"value": "{newDisplayName}"
}
]
}
Notes:
* DisplayName cannot be removed or replaced with empty/null values.
* Membership changes cannot be included when updating group metadata. (For same reasoning as in Create Group.)
Deactivate Group
SCIM has no mechanism to softly deactivate prior to permanent deletion of a group. However, experience has taught that leaping straight to permanent deletion can have undesired repercussions for any user entitlements in the Application based on group membership. If the deletion was in error, it cannot be easily undone. The entitlements inside the Application must be re-established from scratch under a new group Id.
To provide more safety, and simulate an undo window, Identity Providers are encouraged to first remove all group memberships, wait a period of time, and then permanently delete the group in the Application when comfortable with the action. It is also acceptable to leave the empty group in perpetuity, or only delete it on-demand when a new group is being created with a recycled name.
To allow this behavior, Application Providers must support removing all members via a PATCH:
SCIM Operation: PATCH /Groups/{id}
Example:
PATCH /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [{
"op": "remove"
"path": "members”
}]
}
Delete Group
SCIM Operation: DELETE /Groups/{id}
Example:
DELETE /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Notes:
* Application Providers should verify the group membership is empty prior to deletion, and may reject the deletion (via an error code TBD) if members still exist.
Add User to Group / Remove User from Group
Summary of Working Group Discussions
This was a rich topic. Three options were considered.
Option 1) PATCH 1x1
In this approach, each PATCH contains a single user to add/remove. Multiple requests can be parallelized to accelerate provisioning.
Example:
PATCH /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "add",
"path": "members",
"value": [{"value": "{id}"}]
}
]
}
The drawback here is that Identity Providers have encountered frequent rate limiting when parallelizing requests. At times, the call patterns trip DDOS detectors of an Application.
This has been painful enough that multiple Providers expressed a strong preference for avoiding this approach. Some who do it today prefer to change.
Option 2) PATCH a List
In this approach, each PATCH request contains a list of users to add/remove, up to some maximum limit of 100-1000 users.
Example:
PATCH /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "add",
"path": "members",
"value": [
{"value": "{id_1}"},
{"value": "{id_2}"},
{"value": "{id_3}"},
... truncated ...
]
}
]
}
The drawback here is that SCIM asserts PATCH operations must be atomic. However, due to the various data models and storage technologies used by Applications, it can be difficult (or impossible) to commit 100-1000 updates in a single atomic transaction.
If a PATCH partially succeeds, there is no way to communicate back to the Identity Provider which items succeeded/failed. As a result, two Providers may drift out of sync on group memberships.
Option 3) Bulk APIs
The SCIM protocol offers bulk APIs.
Be aware, this is not a mechanism for processing a list of changes via an asynchronous job. SCIM Bulk operations are regular HTTP POSTs with a list of changes. The difference between this and a PATCH operation is that a bulk response is granular enough to indicate which actions succeeded or failed, rather than assuming all is atomic.
The request still needs to be executed within the lifetime of a single HTTP POST. This lifetime is often in the range of 60-100 seconds due to inherent timeouts in the call path. (Proxies, Load Balancers, WAF, etc.)
Net result: bulk requests have the same limits upon the amount of work that can be done, often in the range of 100-1000 requests per request. The key advantage is data consistency, because it is possible to accurately communicate back to the IdP which items succeeded or failed.
The drawback is that almost nobody has implemented bulk APIs in a real production system. Therefore, we are asking Providers to do more work, and risk finding gotchas in a portion of the SCIM specification that has seen minimal use.
RECOMMENDATION
The consensus was to encourage the use of bulk APIs, but allow the use of PATCH as a practical matter, because it represents how most systems behave today.
This approach is illustrated below.
Example of Adding/Removing Group Membership via Bulk APIs
If an Application supports bulk operations, it must declare so in the /ServiceProviderConfig.
If an Identity Provider also supports bulk operations, it must query the /ServiceProviderConfig to detect Application preferences.
Example:
GET /ServiceProviderConfig HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas":
["urn:ietf:params:scim:schemas:core:2.0:ServiceProviderConfig"],
"bulk": {
"supported":true,
"maxOperations":100
},
... truncated ...
}
Applications should specify the “maxOperations”. Minimum value is 100, but Providers may support more. If not defined, the default is “100”.
Next, the Identity Provider can invoke the bulk endpoint:
Example Bulk Request:
POST /Bulk HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas":
["urn:ietf:params:scim:api:messages:2.0:BulkRequest"],
"failOnErrors": false,
"Operations": [
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "b668efbe-fe1c-4e87-b45c-6c2aacb48601",
"data":
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "remove",
"path": "members[value eq \"{user_id_1}\"",
}
]
}
},
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "933d69cc-599a-43bc-88d6-7223b27af374",
"data":
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "add",
"path": "members",
"value": [{"value": "{user_id_2}"}]
}
]
}
},
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "f8fbb531-240e-47cd-b738-ba75bb9dfd52",
"data":
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "add",
"path": "members",
"value": [{"value": "{user_id_3}"}]
}
]
}
}
]
}
* All operations must be on the same GroupId
* The “op” must be either “add” or “remove”.
* Each Operation should add/remove only a single resource.
* The resource must be a User. Nested groups are not supported. Identity Providers should flatten any nested group memberships before provisioning to an Application. (The reasoning here is that most Applications don’t support nested groups, and asking them to do so is a heavy lift in order to achieve FastFed Interoperability.)
* “failOnErrors” must be false or unspecified. (The reasoning here is that Applications may parallelize the work across multiple processes, making it challenging to track the number of accumulated errors in a distributed system. Since nobody appears to need this functionality, labelling it out of scope for FastFed Interoperability.)
The Application provider responds with the results. In the following example, two operations succeeded and one failed.
Example Bulk Response:
HTTP/1.1 200 OK
Content-Type: application/scim+json
Content-Length: ...
{
"schemas":
["urn:ietf:params:scim:api:messages:2.0:BulkResponse"],
"Operations": [
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "b668efbe-fe1c-4e87-b45c-6c2aacb48601",
"status": "204"
},
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "933d69cc-599a-43bc-88d6-7223b27af374",
"status": "204"
},
{
"method": "PATCH",
"path": "/Groups/{groupId}",
"bulkId": "f8fbb531-240e-47cd-b738-ba75bb9dfd52",
"status": "404"
"response": {
"schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
"Resource does not exist with id={user_id_3}”
"status": "404"
}
}
]
}
* As per the SCIM specifications, successful operations return a “204” status code to indicate that no data is being returned.
* If an error occurs, the response is described by Section 3.7.3 of the SCIM Protocol.
Example of Adding/Removing Group Membership via PATCH
If an Identity Provider does not support Bulk, or discovers that the Application doesn’t support Bulk, then PATCH is used.
Example:
PATCH /Groups/{id} HTTP/1.1
Host: example.com
Accept: application/scim+json
Content-Type: application/scim+json
Authorization: Bearer h480djs93hd8
Content-Length: ...
{
"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
"Operations": [
{
"op": "remove",
"path": "members[value eq \"{user_id_1}\"",
},
{
"op": "add",
"path": "members",
"value": [
{"value": "{user_id_2}"},
{"value": "{user_id_3}"}
]
}
]
}
Notes:
* Application Providers must always support group membership changes via PATCH, to be compatible with Identity Providers who lack Bulk capabilities.
* Up to 100 membership changes are allowed per PATCH request. This is measured by counting the individual users being added/removed, summed across all operations in the PATCH.
* Removal of all members, as defined in “Deactivate Group”, is counted as a single change and, if used, must be the first operation in a request.
* Most of the same rules from Bulk updates apply to PATCH. This includes:
* No nested groups
* “op” can be either “add” or “remove”
* If any membership change results in an error, the Application should stop processing and return the error response.
* (Technical Note: Even though the SCIM specification requires that PATCH operations be atomic, we have observed that implementations often are not. As a result, it’s possible that an Application may commit a number of changes before halting on an error. Hence, the bullets below are aimed at allowing a graceful recovery if this occurs.)
* If the Identity Provider receives an error, it should assume that none of the changes were committed and retry as appropriate.
* If the Application Provider receives a duplicate operations, such as adding a user to a group to which the user already belongs (or removing them from a group where they don’t belong), the Application Provider should behave idempotently and return a success.
Other Considerations
________________________________
ETags/If-Match
* Not required for FastFed Interoperability
Configuration Endpoint: /Resourcetypes
* Applications must host a /ResourceTypes endpoint. It must include Users. It may include Groups. If the /ResourceTypes doesn’t include Groups, then the Identity Provider will not provision Groups.
Configuration Endpoint: /ServiceProviderConfig
* Applications must implement a /ServiceProviderConfig endpoint to describe the operations they support.
* The operations must include, at minimum, the set of SCIM capabilities required for FastFed compatibility.
* Additional capabilities may be supported, but are not required for FastFed compatibility.
Configuration Endpoint: /Schemas
* If an Application asks for custom attributes (as described in Section 3.3.5 of FastFed Core), then it must host a /Schemas endpoint describing the attributes.
* Otherwise, if an Application only uses the predefined attributes in the SCIM Core specification, it does not need to host a /Schemas endpoint.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openid.net/pipermail/openid-specs-fastfed/attachments/20200220/1e34a57f/attachment-0001.html>
More information about the Openid-specs-fastfed
mailing list