Progress by July 23

After solving the signature generation/verification puzzles, the two last first term milestones where reached.

More specifically, message transformation operation sets where added earlier tonight for MSN, Yahoo! and ICQ (Jabber transformation set has been added some time ago). Now OtrTransformationLayer can hook into all of these protocols (MSN, Yahoo!, ICQ and Jabber) and provide encrypted communication!

DSA Signature sign/verify using JCE/BouncyCastle/libgcrypt

In my previous update I described various interoperability problems in the signing/verification process.

- Problem 1: signing with standard JCE and verifying with BouncyCastle (and vica versa)

The answer to this problem is that... the BouncyCastle DSA code does not perform the SHA1 hash of the message; they expect you to do that and pass the 20 byte hash in rather than the message. The JCE code automatically does the hashing for you; you just pass it the message you want to get signed, as ghstark and Werner mention here and here accordingly.

- Problem 2: signing with BouncyCastle and verifying with libgcrypt (and vica versa)

This problem is slightly more complicated and in order to understand what's going on I had to debug SIP Communicator/otr4j/BC & Pidgin/libotr/libgcrypt simultaneously. Fortunately we got to an end.

The BC DSA signature algorithm implements the latest standard according to FIPS-186-3 while libgcrypt adheres to FIPS-186-1, FIPS-186-2.

According to FIPS-186-3/p. DSA Signature Generation, (r,s) should be generated as follows:

Let N be the bit length of q. Let min(N, outlen) denote the minimum of the positive integers N and outlen, where outlen is the bit length of the hash function output block. The signature of a message M consists of the pair of numbers r and s that is computed according to the following equations:

r = (g^k mod p) mod q.
z = the leftmost min(N, outlen) bits of Hash(M).
s = (k^-1 (z + xr)) mod q.

while according to FIPS-186-1, FIPS-186-2
z = Hash(M)

Full topic discussion is available here.

Finally, Ian Goldberg in this thread provided a solution to this problem to pass to the signature generator (Hash(M) mod q) rather than truncating the Hash(M) value.

Note that this solution will not work with standard JCE because it requires a DSA signer implementation that doesn't hash it's input...

First term period results

Here I present the results from my first term period. The list is based on my first term planning.

• Develop a java library that handles encryption (see The current state of OTR libraries in Java)
  • Build methods to Assemble/Dissasemble OTR Messages and tests to verify things work properly
    • OTR Query Messages
    • Tagged plaintext messages
    • OTR Error Messages (Completed)
    • D-H Commit Message
    • D-H Key Message
    • Reveal Signature Message
    • Signature Message
    • Data Message (Completed)
  • Handle state transitions for the following actions (detailed description of state transitions here),build key management infrastructure (NOTE: key management for the library, not SC) and build tests to verify things work properly
    In this period functionality for Requesting an OTR conversation, Authenticated Key Exchange (AKE), and Data Exchange will be added.
    • Plaintext message without the whitespace tag
    • Plaintext message with the whitespace tag
    • Query Message
    • Error Message
    • D-H Commit Message
    • D-H Key Message (Completed)
    • Reveal Signature Message
    • Signature Message
    • Version 1 Key Exchange Message
    • Data Message
    • User requests to start an OTR conversation
    • User requests to end an OTR conversation
    • User types a message to be sent (Completed)
• Create a transformation operation set (Completed)
  
• Implement support for the transformation set in all protocols (Failed)
  
• Implement an OTR encryption bundle prototype/proof of concept that encrypts all conversations (Failed)

I have created a transformation set for jabber as an initial effort to incorporate otr4j in SIP Communicator. I've chosen jabber because it is an open standard with server implementation, such as OpenFire, freely available.

I did not proceed into creating transformation sets for the rest of the protocols because there is an issue that the DSA signature (generated with standard JCE) does not verify in Pidgin/libotr, this action is taken in the OTR Protocol Draft, High Level Overview, Authenticated Key Exchange, Step 9. I tried to sign using BouncyCastle Lightweight API instead of the Standard JCE, but that failed too.

To make sure the signature I use is correct, I created a set of test cases to verify signatures created with bouncycastle using the Sun JCE provider and visa versa, but that failed too.

I posted a question about this issue today in several places (CodeRanch, Sun Forums, SIP Communicator dev-list), but I have received no answer yet, so I need to further investigate this issue.

OSGi + BouncyCastle Round 3

In OSGi + BouncyCastle Round 2 I described the problems I was facing while trying to wire up BouncyCastle in SIP Communicator, so as to be used as alternate JCE provider. There where two important problems.

Problem A is this exception [java] java.net.MalformedURLException: invalid url: reference:file:sc-bundles/protocol-msn.jar!/ (java.net.MalformedURLException: Unknown protocol: reference) during AES128-CTR encryption.

The solution to this problem was found by Werner Dittmann in this email. Briefly, I needed to install the "unlimited strength crypto policies" from Sun which are available here in the Other Downloads section. This is not an acceptable solution off course, so we will have to find an alternative (which is pretty obvious, drop the alternative JCE provider and use BouncyCastle Lightweight API).

Problem B is that running SIP Communicator from within Eclipse, and only from within Eclipse, the bundle-plugin-otr fails to generate a D-H key pair using BouncyCastle as alternate provider with class "org.bouncycastle.crypto.CipherParameters"'s signer information does not match signer information of other classes in the same package.

This email in the BouncyCastle developers mailing list mentions that the cause for this exception most likely is an incorrect installation of BouncyCastle, which in my case I guess that means a class path problem.

This guess makes sense, because when Eclipse launches SIP Communicator sets the classpath to whatever the buildpath is. The buildpath in my case contained both bouncycastle and zrtp4j which both export org.bouncycastle.crypto.*. Apparently there is a conflict there.

In this case we could have a bouncycastle bundle that both otr4j and zrtp4j will use or remove zrtp4j from the classpath (we could also update the "How to configure Eclipse to compile and debug SIP Communicator" to contain only the most required libs, like felix, in the classpath).

OSGi + BouncyCastle, Round 2

This is getting crazy... What I'm trying to do is wire up BouncyCastle correctly, so as to be used as alternate JCE provider by otr4j (or at least from within a sc-bundle), so the OtrTransformLayer can work properly..

What I've tried is:

A) Import every single package of BouncyCastle (I found those packages using this snippet) in org.osgi.framework.system.packages.extra + any other packages required (discover them through ClassDefNotFound exceptions + this snippet) in felix.client.run.properties file.

B) Import all of those packages in bundle-plugin-otr in build.xml.

So with this setup...

Running SIP Communicator using "ant run" seems to be partially working. We can generate D-H key pair using BouncyCastle as alternate provider, but I get (Problem A) [java] java.net.MalformedURLException: invalid url: reference:file:sc-bundles/protocol-msn.jar!/ (java.net.MalformedURLException: Unknown protocol: reference), like in this SIP Communicator issue. So let's try to debug this problem!

Running SIP Communicator using Eclipse, fails to generate a D-H key pair using BouncyCastle as alternate provider (Problem B) with class "org.bouncycastle.crypto.CipherParameters"'s signer information does not match signer information of other classes in the same package, so I cannot debug the above problem... So let's try to debug this problem!

The BouncyCastle signed jar does not have debug information, so attaching the BouncyCastle source code and installing a breakpoint in org.bouncycastle.jce.provider.JDKKeyPairGenerator$DH constructor gives me Unable to install breakpoint due to missing line numbers (Problem C), so I cannot debug the above problem, and this problem can only be resolved if we recompile the BouncyCastle lib, and it won't be a signed jar anymore.

To the bottom line, even if I didn't care about the first problem, the most important problem is that I cannot debug properly (Problem B). To the very bottom line, this probably means refactoring otr4j so as to use the BouncyCastle lightweight API instead of the BouncyCastle JCE provider. This would solve Problem C, and hopefully Problem B as well.

The setup I've used is r5496 of my branch.. I checked it in for reference.

UPDATE: There are solutions to these problems, and I discuss them in OSGi + BouncyCastle Round 3.