<div dir="ltr">Just to make sure, I hope you're referring to running a local Docker instance?<div><br><div><span style="font-size:12.8px">I disagree with Roland on using the official certification servers for continuous integration tests of participants: it will make it harder to troubleshoot, harder to do maintenance, harder to guard system resources and it is simply not what the certification server is meant for. Since we've had trouble with stability in the past and since I've been tasked with operational maintenance of the test servers, I will vote against that.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Of course I'd very much welcome any feedback on how to improve the Docker variant(s) of the test suite since continuous integration is exactly the reason why we've added that in the first place (incl. continuous integration testing of the test suite itself). That is still a work in progress but rapidly becoming useful.</span><br></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Hans.</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 3, 2017 at 8:34 PM, Dominick Baier via Openid-specs-ab <span dir="ltr"><<a href="mailto:openid-specs-ab@lists.openid.net" target="_blank">openid-specs-ab@lists.openid.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Hi Roland, </div><div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">OK - I am happy to do give that a try. I also like the idea.</div> <br> <div id="m_5086856675268783911bloop_sign_1493836447105759232" class="m_5086856675268783911bloop_sign"><div><br></div><div>-------</div><div>Dominick Baier</div></div><div><div class="h5"> <br><p class="m_5086856675268783911airmail_on">On 3. May 2017 at 17:47:29, Roland Hedberg (<a href="mailto:roland@catalogix.se" target="_blank">roland@catalogix.se</a>) wrote:</p> <blockquote type="cite" class="m_5086856675268783911clean_bq"><span><div style="word-wrap:break-word"><div></div><div>
Dominick,
<div><br></div>
<div>as I said in a discussion with William the other day.
I’m not sure there will be a performance issue if you where to
run</div>
<div>against the OIDF test server. Most of things
happening are network based events so the load on the machine is
minor.</div>
<div>The only thing we might have a problem with is the
size of the log files vs the available disc space.</div>
<div>But rotating the logs should take care of that.</div>
<div>Now, there are reasons why you may want to this
locally and as Hans wrote in another mail he’s working on making
that easier.</div>
<div>The bottom line being that there is absolutely no
reason why you wouldn’t include the test suite into your
continues-integration pipeline :-) :-)</div>
<div><br>
<div>
<blockquote type="cite">
<div>3 maj 2017 kl. 07:46 skrev Dominick Baier via
Openid-specs-ab <<a href="mailto:openid-specs-ab@lists.openid.net" target="_blank">openid-specs-ab@lists.openid.<wbr>net</a>>:</div>
<br class="m_5086856675268783911Apple-interchange-newline">
<div>
<div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;margin:0px">> Incidentally, one of the cool things about how we
implemented these tests in AppAuth, is that we
actually <b>built them into our
continuous-integration testing pipeline</b>. </div>
<div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;margin:0px"><br></div>
<div id="m_5086856675268783911bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;margin:0px">I thought about this too - but I actually didn’t want to
produce load all the time I am doing a check-in into the repo. I
added a test runner to the source code, so anyone can manually
start the tests when needed.</div>
<br style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div id="m_5086856675268783911bloop_sign_1493822708417135872" class="m_5086856675268783911bloop_sign" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div><br></div>
<div>-------</div>
<div>Dominick Baier</div>
</div>
<br style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<p class="m_5086856675268783911airmail_on" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
On 1. May 2017 at 18:55:08, William Denniss via Openid-specs-ab
(<a href="mailto:openid-specs-ab@lists.openid.net" target="_blank">openid-specs-ab@lists.openid.<wbr>net</a>) wrote:</p>
<blockquote type="cite" class="m_5086856675268783911clean_bq" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div>
<div></div>
<div>
<div dir="ltr">
<div><span>Sorry we didn't get a chance to talk
in Chicago on this topic Mike, my trip was all too brief. I'll be
around this week though, hopefully we can discuss this with the
relevant parties.<br></span></div>
<div><span><br></span></div>
<div><span>As of yesterday, AppAuth for iOS and
macOS is now<span class="m_5086856675268783911Apple-converted-space"> </span><a href="https://github.com/openid/AppAuth-iOS/pull/101" target="_blank">passing</a><span class="m_5086856675268783911Apple-converted-space"> </span>all
but those 4 signature verification tests in the "code" profile. I'm
preparing the certification packet, once we have a final decision
on the optionality of those tests, I'm hoping to
certify.</span></div>
<div><br></div>
<div>Incidentally, one of the cool things about how we
implemented these tests in AppAuth, is that we actually<span class="m_5086856675268783911Apple-converted-space"> </span><b>built them into
our continuous-integration testing pipeline</b>. The conformance
tests run alongside our unit tests for every release, and every git
push. The certification log output is automatic too, meaning anyone
can run the certification tests and produce the same output at the
click of a button.</div>
<div><br></div>
<div>I think this is a huge value-add for the RP
certification program. Previously we only had unit tests in the
library, no end-to-end tests due to the fact we didn't have an OP
with interaction-less responses that we could use for automated
testing. The RP certification program has made this available, and
by using it, our test coverage is vastly improved.</div>
<div><br></div>
<div>Thank you Roland, Mike, the Foundation and everyone
who is working on this, it's a very valuable effort!</div>
<div><br></div>
<div>Best,</div>
<div>William</div>
<div><br></div>
<div><br></div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Mar 26, 2017 at 1:29 PM, Mike
Jones<span class="m_5086856675268783911Apple-converted-space"> </span><span dir="ltr"><<a href="mailto:Michael.Jones@microsoft.com" target="_blank">Michael.Jones@<wbr>microsoft.com</a>></span><span class="m_5086856675268783911Apple-converted-space"> </span>wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_5086856675268783911m_3647925707323825383WordSection1">
<p class="MsoNormal"><span style="color:rgb(0,32,96)">One thought is that this maybe should depend upon how the RP
registers. If it registers with support for signature
algorithms, then that support should be tested – even for
response_type=code. If it registers only with support for
“alg”: “none”, then it obviously can’t be tested then.</span></p>
<div><span style="color:rgb(0,32,96)"> </span><br class="m_5086856675268783911webkit-block-placeholder"></div>
<p class="MsoNormal"><span style="color:rgb(0,32,96)">My logic is that if the RP can check signatures, the OP provides
a bad signature, and the RP doesn’t catch it, that seems like a
scenario what shouldn’t pass certification. Let’s talk about
this in person in Chicago this week. I’d love to hear what
others think about this as well.</span></p>
<div><span style="color:rgb(0,32,96)"> </span><br class="m_5086856675268783911webkit-block-placeholder"></div>
<p class="MsoNormal"><span style="color:rgb(0,32,96)"> <wbr> <span class="m_5086856675268783911Apple-converted-space"> </span>--
Mike</span></p>
<p class="MsoNormal"><a name="m_5086856675268783911_m_3647925707323825383__MailEndCompose" id="m_5086856675268783911m_3647925707323825383__MailEndCompose"><span style="color:rgb(0,32,96)"> </span></a></p>
<p class="MsoNormal"><b>From:</b><span class="m_5086856675268783911Apple-converted-space"> </span>Openid-specs-ab
[mailto:<a href="mailto:openid-specs-ab-bounces@lists.openid.net" target="_blank">openid-specs-ab-bounce<wbr>s@lists.openid.net</a>]<span class="m_5086856675268783911Apple-converted-space"> </span><b>On Behalf
Of</b><span class="m_5086856675268783911Apple-converted-space"> </span>William
Denniss via Openid-specs-ab<br>
<b>Sent:</b><span class="m_5086856675268783911Apple-converted-space"> </span>Sunday, March 26, 2017 11:13
AM<br>
<b>To:</b><span class="m_5086856675268783911Apple-converted-space"> </span><a href="mailto:openid-specs-ab@lists.openid.net" target="_blank">openid-specs-ab@lists.<wbr>openid.net</a><br>
<b>Subject:</b><span class="m_5086856675268783911Apple-converted-space"> </span>[Openid-specs-ab] RP Tests: ID
Token signature validation for code flow</p>
<div>
<div class="m_5086856675268783911h5">
<div> <br class="m_5086856675268783911webkit-block-placeholder"></div>
<div>
<p class="MsoNormal">Regarding the<span class="m_5086856675268783911Apple-converted-space"> </span><a href="https://rp.certification.openid.net:8080/list?profile=C" target="_blank">'code' response type tests</a>, my understanding
is that it's not necessary to validate the ID Token signature as it
was obtained via a HTTPS connection to the OP.</p>
<div>
<div> <br class="m_5086856675268783911webkit-block-placeholder"></div>
</div>
<div>
<p class="MsoNormal">This test follows that logic:</p>
<div>
<p class="MsoNormal">rp-id_token-sig-none</p>
</div>
<div>
<div> <br class="m_5086856675268783911webkit-block-placeholder"></div>
</div>
<div>
<p class="MsoNormal">However, these 4 tests assume signature
validation for the code flow:</p>
</div>
<div>
<p class="MsoNormal">rp-id_token-kid-absent-single-<wbr>jwks<br>
rp-id_token-kid-absent-multipl<wbr>e-jwks<br>
rp-id_token-bad-sig-rs256<br>
rp-id_token-sig-rs256</p>
</div>
</div>
<div>
<div> <br class="m_5086856675268783911webkit-block-placeholder"></div>
</div>
<div>
<p class="MsoNormal">Can they be made optional for the 'code'
response type tests?</p>
</div>
<div>
<div> <br class="m_5086856675268783911webkit-block-placeholder"></div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br></div>
______________________________<wbr>_________________<span class="m_5086856675268783911Apple-converted-space"> </span><br>
Openid-specs-ab mailing list<span class="m_5086856675268783911Apple-converted-space"> </span><br>
<a href="mailto:Openid-specs-ab@lists.openid.net" target="_blank">Openid-specs-ab@lists.openid.<wbr>net</a><span class="m_5086856675268783911Apple-converted-space"> </span><br>
<a href="http://lists.openid.net/mailman/listinfo/openid-specs-ab" target="_blank">http://lists.openid.net/<wbr>mailman/listinfo/openid-specs-<wbr>ab</a><span class="m_5086856675268783911Apple-converted-space"> </span><br>
</div>
</div>
</blockquote>
<span style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">______________________________<wbr>_________________</span><br style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">Openid-specs-ab mailing list</span><br style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<a href="mailto:Openid-specs-ab@lists.openid.net" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">Openid-specs-ab@lists.openid.<wbr>net</a><br style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<a href="http://lists.openid.net/mailman/listinfo/openid-specs-ab" style="font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">http://lists.openid.net/<wbr>mailman/listinfo/openid-specs-<wbr>ab</a></div>
</blockquote>
</div>
<br></div>
</div></div></span></blockquote></div></div></div>
<br>______________________________<wbr>_________________<br>
Openid-specs-ab mailing list<br>
<a href="mailto:Openid-specs-ab@lists.openid.net">Openid-specs-ab@lists.openid.<wbr>net</a><br>
<a href="http://lists.openid.net/mailman/listinfo/openid-specs-ab" rel="noreferrer" target="_blank">http://lists.openid.net/<wbr>mailman/listinfo/openid-specs-<wbr>ab</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div style="font-size:small"><a href="mailto:hans.zandbelt@zmartzone.eu" target="_blank">hans.zandbelt@zmartzone.eu</a></div><div style="font-size:small">ZmartZone IAM - <a href="http://www.zmartzone.eu" target="_blank">www.zmartzone.eu</a><br></div></div></div></div></div></div>
</div></div>