[Openid-specs-ab] Issue #1464: Conformance testing for SIOP & VP (openid/connect)

[josephheenan]

New issue 1464: Conformance testing for SIOP & VP
https://bitbucket.org/openid/connect/issues/1464/conformance-testing-for-siop-vp

Joseph Heenan:

Gail has asked if I could estimate the work involved in creating conformance/certification tests for SIOP & verifiable presentations, in similar ways to the tests created for OpenID Connect / FAPI / FAPI-CIBA / etc.

Before I do that, it would be great to get some input from the working group.

1. Are there test systems we can test certification tests against? Ideally these would be sandbox-type systems that contain no real user data, and where any required user interactions can be automated \(this is so we can run automated testing of the tests. We have an existing system for automating simple web interactions along the ‘enter text into this field’ and ‘press this button’.\) An example client that shows exactly what is any requests/responses/redirects would also be helpful.
2. Are there any particular happy-flow or negative scenarios the WG feel are particularly important to test? \(For example, for OpenID Connect certification there is a happy flow that requires response\_type=code&scope=openid&… to work and return a fully valid id\_token, and a negative flow that requires unregistered redirect uris are rejected.\)
3. Any guidance on expected certification profiles and optional/mandatory features would be helpful. \(For example, OpenID Connect has a ‘Dynamic’ certification profile that requires the OP to publish authorization server metadata and to support dynamic client registration, and a ‘Basic’ certification profile that requires neither.\)

I’ve assumed for now that tests for identity providers are more important than tests for relying parties, as that’s generally how other working groups have viewed it, but please say if this assumption is wrong.

  
For example, this is some rough guidance that the ekyc-ida working group provided \(with some input from the certification team\): [https://docs.google.com/document/d/1SX2\_SjcMUQJ6SQEuNrhNjqAqpQjTUqkHl\_qCReSv9-Y/edit#heading=h.cuz9mnx958lj](https://docs.google.com/document/d/1SX2_SjcMUQJ6SQEuNrhNjqAqpQjTUqkHl_qCReSv9-Y/edit#heading=h.cuz9mnx958lj)