How to create a pull tester JAR

I just checked in a change to bitcoinj git master that makes it much easier to create a pull tester jar. Here are instructions for how to do it.

You will need:

   - A Java Development Kit (JDK), version 6 or up should work. As Java 6
   was released eight years ago, this should not be a challenging requirement.
   If you have a Mac just running "java" from the command line should give you
   a GUI prompt to install it automatically. Otherwise apt-get or fetch the
   latest from the interwebs.

   - Apache Maven. This is a rough equivalent of autotools, except it does
   dependency resolution for you. Grab it from
   http://maven.apache.org/download.cgi then unzip it and make sure the bin
   directory is in your PATH. You may need to set the JAVA_HOME environment
   variable if you installed Java to an odd place.

   - git

Make sure you can run "javac" from the command line, then make sure you can run "mvn", it should complain it can't find a POM (this is a build config file) and not, say, that it can't find Java.

Now grab bitcoinj from git master:

git clone https://github.com/bitcoinj/bitcoinj.git

... and build ....

cd bitcoinj mvn -DskipTests package

It will go off and download the libraries needed, compile, and create a bundled executable JAR called core/target/pull-tests.jar. This is sort of analogous to static linking in the Java world. It should be fast - expect a full build plus downloads to take less than a minute. You can use it either with the QA scripts in the bitcoin core qa/pull-tester directory or just run things directly:

./bitcoind -regtest -connect=0.0.0.0 -listen -whitelist=127.0.0.1 -datadir=/tmp/pulltester java -jar core/target/pull-tests.jar

It should go ahead and print lots of debug spew, then at the end say it's happy.

Let me know if you encounter any problems with this.

Java JARs (which are just zip files renamed) are easily reproduced if you use the same version of javac and the same bitcoinj version. The ZIP container has timestamps, but unzipping them and simply diffing the files between two builds should reveal no differences. I am happy to provide a pull-tests.jar from my local machine if anyone would like to do this.

Oh, I forgot to mention something important. Ridiculously, the default package repository Maven uses was not protected by SSL up until a few days ago. They made it available via SSL now, but you have to tell Maven about the new URL. I guess they'll do a new release where SSL is the default soon. But for now before you run mvn save the following magic incantation to the path ~/.m2/settings.xml:

(side note: yes maven's love of XML is widely ridiculed and more modern build tools have much better config languages, but we didn't upgrade yet)

<settings>
  <activeProfiles>
    <!--make the profile active all the time -->
    <activeProfile>securecentral</activeProfile>
  </activeProfiles>
  <profiles>
    <profile>
      <id>securecentral</id>
      <!--Override the repository (and pluginRepository) "central" from the
         Maven Super POM -->
      <repositories>
        <repository>
          <id>central</id>
          <url>https://repo1.maven.org/maven2</url>
          <releases>
            <enabled>true</enabled>
          </releases>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>central</id>
          <url>https://repo1.maven.org/maven2</url>
          <releases>
            <enabled>true</enabled>
          </releases>
        </pluginRepository>
      </pluginRepositories>
    </profile>
  </profiles>
</settings>

Thanks for posting that (and implicitly archiving the knowledge). Anything that makes test improvement easier is welcomed.

On Tue, Aug 5, 2014 at 11:00 AM, Mike Hearn <mike@plan99•net> wrote:

> I just checked in a change to bitcoinj git master that makes it much
> easier to create a pull tester jar. Here are instructions for how to do it.
>
> You will need:
>
>    - A Java Development Kit (JDK), version 6 or up should work. As Java 6
>    was released eight years ago, this should not be a challenging requirement.
>    If you have a Mac just running "java" from the command line should give you
>    a GUI prompt to install it automatically. Otherwise apt-get or fetch the
>    latest from the interwebs.
>
>    - Apache Maven. This is a rough equivalent of autotools, except it
>    does dependency resolution for you. Grab it from
>    http://maven.apache.org/download.cgi then unzip it and make sure the
>    bin directory is in your PATH. You may need to set the JAVA_HOME
>    environment variable if you installed Java to an odd place.
>
>    - git
>
> Make sure you can run "javac" from the command line, then make sure you
> can run "mvn", it should complain it can't find a POM (this is a build
> config file) and not, say, that it can't find Java.
>
> Now grab bitcoinj from git master:
>
> git clone https://github.com/bitcoinj/bitcoinj.git
>
> ... and build ....
>
> cd bitcoinj
> mvn -DskipTests package
>
> It will go off and download the libraries needed, compile, and create a
> bundled executable JAR called core/target/pull-tests.jar. This is sort of
> analogous to static linking in the Java world. It should be fast - expect a
> full build plus downloads to take less than a minute. You can use it either
> with the QA scripts in the bitcoin core qa/pull-tester directory or just
> run things directly:
>
> ./bitcoind -regtest -connect=0.0.0.0 -listen -whitelist=127.0.0.1
> -datadir=/tmp/pulltester
> java -jar core/target/pull-tests.jar
>
> It should go ahead and print lots of debug spew, then at the end say it's
> happy.
>
> Let me know if you encounter any problems with this.
>
> Java JARs (which are just zip files renamed) are easily reproduced if you
> use the same version of javac and the same bitcoinj version. The ZIP
> container has timestamps, but unzipping them and simply diffing the files
> between two builds should reveal no differences. I am happy to provide a
> pull-tests.jar from my local machine if anyone would like to do this.
>
>
> ------------------------------------------------------------------------------
> Infragistics Professional
> Build stunning WinForms apps today!
> Reboot your WinForms applications with our WinForms controls.
> Build a bridge from your legacy apps to the future.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

On 08/05/2014 05:11 PM, Mike Hearn wrote:

> Oh, I forgot to mention something important. Ridiculously, the default
> package repository Maven uses was not protected by SSL up until a few
> days ago.  They made it available via SSL now, but you have to tell
> Maven about the new URL. I guess they'll do a new release where SSL is
> the default soon.

FWIW, I filed a wishlist item here: https://jira.codehaus.org/browse/MNG-5672

and here, for the old Ubuntu versions of Maven: https://bugs.launchpad.net/ubuntu/+source/maven/+bug/1352418

No problem.

The pull tester entry point can be found here:

https://github.com/bitcoinj/bitcoinj/blob/master/core/src/test/java/com/google/bitcoin/core/BitcoindComparisonTool.java

(nb: in the near future I will be re-namespacing the library from com.google.bitcoin to org.bitcoinj to reflect that it no longer has anything to do with Google and then this link will break).

The code itself is a rather bad example of copy/paste coding and I can say that, because Matt knows it and already plans to refactor things ;) So if anyone is thinking of adding tests to the framework coordinate with him first to ensure you don't end up conflicting with a big refactor/rewrite.

Once you ave the jar, you can also build with

./configure --disable-silent-rules --disable-ccache --with-comparison-tool=/path/to/your/BitcoindComparisonTool.jar

Instead of the regular

./configure

And after that "make check" will run most of the tests the pull tester does.

On 8/5/14, Mike Hearn <mike@plan99•net> wrote:

> No problem.
>
> The pull tester entry point can be found here:
>
> https://github.com/bitcoinj/bitcoinj/blob/master/core/src/test/java/com/google/bitcoin/core/BitcoindComparisonTool.java
>
> (nb: in the near future I will be re-namespacing the library from
> com.google.bitcoin to org.bitcoinj to reflect that it no longer has
> anything to do with Google and then this link will break).
>
> The code itself is a rather bad example of copy/paste coding and I can say
> that, because Matt knows it and already plans to refactor things ;) So if
> anyone is thinking of adding tests to the framework coordinate with him
> first to ensure you don't end up conflicting with a big refactor/rewrite.
>