[OpenID] Backwards Compatibility

[John Bradley]

I was thinking of the API  making Unit tests and API tests easier to  
construct.

A large number of the tests on http://test-id.net could be used as a  
part of unit tests however there is no easy way to convey the results  
to the testing party for any of the tests that are more sophisticated  
than knowing you should get a positive or negative assertion.

There is a lot of debugging information generated, but it is in human  
readable form at the moment.

Also for testing things like AX there are a lot of options that you  
can set at the RP end to test the OP.

You need to both set the test options and interrogate the results.    
That requires some sort of API outside of openID to achieve in an  
automated way.

The latest Liberty Interoperability Testing report and Procedures,   
are interesting as a reference.
Testing
http://www.projectliberty.org/liberty/content/download/4435/30275/file/SAML_3Q08_%20Interoperability_PUBLIC_Final_Report_102308-FINAL.pdf
Test Procedures
http://www.projectliberty.org/liberty/content/download/2273/15001/file/ID-WSF-2-0-TestProcedures-v1-01.pdf
http://www.projectliberty.org/liberty/content/download/4160/27946/file/Liberty_Interoperability_SAML_Test_Plan_v3.1.pdf

However even there most of the testing appears to be manual.

Perhaps Brett knows if there are IPR free/Open Source tools that could  
be used to construct these sorts of Conformance and Full Matrix  
Tests.   I am certain that there are proprietary tools.  I am looking  
for open-source friendly tools.

The user oriented tests you seem to be thinking of are possible but a  
bit orthogonal to the testing needs of OPs and RPs.

I am glad the dialog on this has picked up.

Regards
John Bradley


On 18-Mar-09, at 7:13 PM, SitG Admin wrote:

>> I would like to see us come together to create a unit test API so  
>> that tests can be more automated.
>
> You take that end, I'll take this end? API's are nice as a layer of  
> abstraction covering up the messy internals (of an object), how to  
> do them as a layer of transparency to make what's going on inside  
> *more* apparent to learning developers?
>
>> However tests like checking that the OP is prompting you with a  
>> reasonable dialog in response to a AX attribute request can never  
>> be entirely automated.
>
> I was thinking of presenting dialog boxes from the OP for 'auditing'  
> mode, where, instead of asking the user to authenticate, the user  
> would be shown a page describing, step by step, how the URL was  
> separated out into component values, and what would then be *done*  
> with those values - but the values to be encoded for return would be  
> placed in *input* (text) fields before the user, fully editable  
> (also allowing developers to test for suspected vulnerabilities).  
> Scripting on the page could enable the user to perform functions  
> right there in their browser, such as calculating the hash for a  
> string.
>
> All those values would then be sent to the OP when the user was done  
> looking through them, and the OP would print out a string (and link)  
> of all those values for the user to inspect and copy into their  
> address bar. (Involving a second page is important not just because  
> the user might have scripting (which could probably make it happen)  
> enabled, but because the user might not *trust* those scripts - so,  
> of course, the hash calculations should *not* be automatic, there  
> would be a button to do so and the relevant formulae would be  
> printed out right beside it, so users could employ their own  
> calculating methods if desired.) The signature would be done OP- 
> side, appropriately keeping that private key out of the hands of the  
> user, who could verify that it decoded to the expected value with  
> the public key using another button on the second page. Of course,  
> if the user changed any of the values from their default (expected)  
> state, the OP might refuse to sign, and the *user* would have to  
> supply a private key on their end to determine what value they would  
> send in the "signature" field (or just send the hash itself, or an  
> empty string) before seeing how the RP responded to this.
>
> -Shade

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2486 bytes
Desc: not available
URL: <http://lists.openid.net/pipermail/openid-general/attachments/20090318/4690651e/attachment-0002.bin>