What could possibly go wrong when migrating plug-ins from Stash to Bitbucket Server

While the migration from Stash to Bitbucket Server should normally go well thanks to the very thorough Bitbucket Server upgrade guide, we experienced some hard issues while migrating our 11 existing Stash plug-ins. Most of the problems we faced were not due to the API changes in Bitbucket Server, but because of library and framework upgrades of the dependencies that are shipped with the product. We thought we share two of our findings as examples of these issues here so that others will not get hit by them as well.

OSGi default package issue

After we migrated one of our plug-ins to Bitbucket Server, we got the following Maven error when we tried to run it :

[ERROR] Manifest com.mycompany.myplugin: The default package '.' is not permitted by the Import-Package syntax.
This can be caused by compile errors in Eclipse because Eclipse creates valid class files regardless of compile errors.
The following package(s) import from the default package null
[ERROR] Error(s) found in manifest configuration
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

Of course, as good Java/Scala citizens as we are :-), we don’t have any source files in the default package of our plug-in. So we suspected that it must be one of our plug-in dependencies (although that seamed also kind of strange as we have never experienced this error before in Stash).

After some research, we found that JDOM is known to cause these kind of troubles because it has classes in the default Java package. There is a nice article about this issue and how to prevent it from Atlassian called Using JDOM in OSGi.

We first thought that one of our dependencies pulled in JDOM, but this was not the case:

$ mvn dependency:tree | grep -i jdom
$

So there must be another library causing these issues. We then decompiled our plug-in jar with JAD and searched for a Java file not containing a package declaration:

find . -type f -name "*.java" | xargs grep -H -c 'package .*;' |
grep 0$ | cut -d':' -f1

Et voilá, we found the problematic library:

./javacc.java
$

The JavaCC library contained a sample main program causing these issues. Thanks to the mentioned Atlassian article, we were able to extract the contents of this library, remove the problematic Java file javacc.java and build the JAR again. This whole process is now being done during our Maven build.

After some more research and discussions on the Atlassian mailing list, we were informed that the OSGi container Felix has been upgraded with Bitbucket Server. This upgrade also includes an update of the library bndlib which is used to create the OSGi bundles. bndlib added some constraints like the one we got hit with the default package with its latest upgrade.

Run-time issues with self-provided Soy functions

In one of our plug-ins we use self-provided Soy (a.k.a Google Closure Templates) functions. After migrating that plug-in, we got the following error at run-time:

[INFO] [talledLocalContainer] Caused by: java.lang.AbstractMethodError: null
[INFO] [talledLocalContainer] at com.atlassian.soy.impl.modules.SoyJavaFunctionAdapter.computeForJava
(SoyJavaFunctionAdapter.java:39) ~[na:na]
[INFO] [talledLocalContainer] at com.atlassian.soy.impl.modules.CompositeFunctionAdaptor.computeForJava(CompositeFunctionAdaptor.java:44) ~[na:na]
[INFO] [talledLocalContainer] at com.google.template.soy.sharedpasses.render.EvalVisitor.computeFunctionHelper(EvalVisitor.java:670) ~[na:na]

As an AbstractMethodError is often a sign of API incompatibilities, we suspected the issue had to do with an incompatible library upgrade. One nice thing about Atlassian is that almost all important projects are open source. So we were easily able to lookup the code of the SoyJavaFunctionAdapter on Bitbucket. The line causing this issue was

Object returnValue = soyServerFunction.apply(pluginArgs);

As you can see, the method apply is called on our Soy function which looks like follows:

Server-side SOY function to render user information in Bitbucket Server

After some analysis, we were able to spot the problem and could reproduce it with a complete, minimal example: calling an overridden Scala method of a Java interface (SoyServerFunction) with a varargs parameter results in an AbstractMethodError in Scala 2.11. This was working before as Stash shipped Scala 2.10, while Bitbucket Server now comes with the Scala standard library 2.11 (it also includes the one for Scala 2.10). After some research, we were able to find the matching bug report in the Scala JIRA project which confirmed our assumption.

As we wanted to make use of Scala 2.11 in our plug-in, we had to provide Java wrappers for our Soy functions which delegated to the Scala implementations. Here’s an example:

Java SOY function adapter for the Scala implementation

We now use these in the plug-in descriptor instead of the Scala ones:

Usage of the Java Soy function adapter

Migrating was mostly painless — apart from a few very hard issues

While the migration to Bitbucket Server was mostly painless, we got hit by a few very hard and time-consuming issues. As it is often the case in Java land, there are zillions of dependencies for every non-trivial application and hence, these dependencies are often what makes migrations like these hard. While the API transition from Stash to Bitbucket Server went very smooth thanks to the exhaustive Atlassian documentation, you can always get hit by the ecosystem.