[OpenID board] Building an OpenID technical interop framework

John Bradley jbradley at mac.com
Sat Jan 16 16:36:48 UTC 2010

I certainly support putting some money into this effort.

The tests that Andrew Arnott and I developed on test-id.net over the last year and a half were only a start.
Without funding we were only able to concentrate on some of the new tests for ICAM over the last several months.

All of those tests source code are available in GitHub for incorporation into something new if desired.

I have to say that any new project will require the close cooperation of a library author like Andrew.

We found that many of the tests required refactoring the library to add features and test points that are not normally required.

It did uncover hidden issues and was a positive addition to the DotNetOpenAuth library.

I am glad tose companies have decided to contribute the funds to the OIDF to develop a more complete testing platform.

To David's comment though test-id is not strictly security focused about half the tests are security related.
Security was also one of the least tested areas a year ago.   We found complete security failures in some of the major RPs because of small coding errors and lack of testing.

I hope that this can be the start of a much larger project.
We should also make certain that the tests are available in multiple languages.

I wanted to do that for test-id but could not find the support.

I look forward to finding the matching funds and putting together the project.

John B.

On 2010-01-16, at 12:48 PM, Nat wrote:

> Indeed, and we were talking of something similar in Japan as well. 
> =nat at Tokyo via iPhone
> On 2010/01/16, at 15:07, Chris Messina <chris.messina at gmail.com> wrote:
>> Just want to add my support to this effort, if I haven't voiced it already!
>> Chris
>> On Fri, Jan 15, 2010 at 5:12 AM, Marc Canter <marc at broadbandmechanics.com> wrote:
>> YES!
>> This is the test lab/suite I asked for two years ago
>> 'bout time
>> right on!
>> YES!
>> On Thu, Jan 14, 2010 at 10:53 PM, David Recordon <recordond at gmail.com> wrote:
>> (bcc'd general@, please reply on the public board list)
>> One of the consistent pieces of feedback we've received from
>> developers is that it's difficult to correctly create a new OpenID
>> Relying Party or Provider due to the lack of Foundation run
>> interoperability tests that help developers understand if their
>> implementation is correct.  While JanRain used to run a set of tests
>> like this on OpenIDEnabled.com, they were taken offline almost a year
>> ago.  The Foundation has already funded some development of
>> http://test-id.net/, but it focuses largely on security driven tests.
>> Facebook and Google are each interested in contributing $10,000 to the
>> OpenID Foundation to develop an easy to use technical interoperability
>> site for OpenID if the Foundation also contributes at least $10,000 to
>> the effort, the following product specification is followed, the
>> companies are able to collaboratively choose the contractor which will
>> perform the development work, and the resulting software is released
>> under an open source license (Apache).  We believe that the existence
>> of this framework will be one of the highest leverage projects in both
>> driving broad adoption of interoperable OpenID implementations and in
>> increasing the overall quality of the open source OpenID libraries.
>> A framework to add tests:
>> Just as traditional unit tests are written, the software should
>> support the ability to add additional tests for RPs and OPs at any
>> time.  Each test should be a part of a given OpenID specification with
>> the ability to group multiple tests together based on functionality.
>> Some tests can be fully automated (i.e. discovery) and others will
>> require human interaction (i.e. sign in).
>> Built like developers think:
>> Developers implementing OpenID think in broad strokes such as "can I
>> sign in?" which the framework should be built around.  There should be
>> two major groups of tests, one which exercises a Relying Party
>> implementation and one which exercises a Provider.  Upon starting the
>> test, the software should direct the developer through the steps which
>> are needed to test their implementation in a logical order such as
>> discovery, association, authentication, and verification.  An
>> individual developer should not need to know to choose the "RP
>> protects against association poisoning" test, but it should be done
>> automatically.
>> Supports multiple specifications:
>> The framework should be extensible such that a developer can choose to
>> test their support for individual extensions to the OpenID
>> Authentication protocol.  It should include tests for AX, PAPE, and
>> the User Experience extensions.  Ideally this framework could grow to
>> support other protocols such as OAuth as well.
>> Supports multiple environments:
>> The framework should support multiple environments, with the ability
>> to override DNS settings using the equivalent of a hosts file to
>> switch between environments. A standard test framework would be an
>> invaluable resource for RPs and OPs to test their QA environment prior
>> to a production release.
>> Results should be logged:
>> The software should support recording the test results of a given RP
>> or OP and sharing them publicly.  This could ultimately evolve into
>> automated smoke testing of many different OPs and RPs.
>> It looks nice:
>> Yes, we might be software engineers but let's create something which
>> is usable.  Matching the OpenID.net site design is a fine place to
>> start.
>> Thoughts?
>> Thanks,
>> David and Eric (Sachs)
>> _______________________________________________
>> board mailing list
>> board at lists.openid.net
>> http://lists.openid.net/mailman/listinfo/openid-board
>> -- 
>> Chris Messina
>> Open Web Advocate, Google
>> Personal: http://factoryjoe.com
>> Follow me on Twitter: http://twitter.com/chrismessina
>> This email is:   [ ] shareable    [X] ask first   [ ] private
>> _______________________________________________
>> board mailing list
>> board at lists.openid.net
>> http://lists.openid.net/mailman/listinfo/openid-board
> _______________________________________________
> board mailing list
> board at lists.openid.net
> http://lists.openid.net/mailman/listinfo/openid-board

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openid.net/pipermail/openid-board/attachments/20100116/c8066120/attachment-0001.htm>

More information about the board mailing list