README

Build instructions for BookKeeper

-------------------------------------------------------------------------------
Requirements:

* Unix System
* JDK 1.6
* Maven 3.0
* Autotools (if compiling native hedwig client)
* Internet connection for first build (to fetch all dependencies)

-------------------------------------------------------------------------------
The BookKeeper project contains:

 - bookkeeper-server     (BookKeeper server and client)
 - bookkeeper-benchmark  (Benchmark suite for testing BookKeeper performance)
 - hedwig-protocol       (Hedwig network protocol)
 - hedwig-client         (Hedwig client library)
 - hedwig-server         (Hedwig server)

BookKeeper is a system to reliably log streams of records. It is designed to 
store  write ahead logs, such as those found in database or database like 
applications.

Hedwig is a publish-subscribe system designed to carry large amounts of data 
across the internet in a guaranteed-delivery fashion from those who produce 
it (publishers) to those who are interested in it (subscribers).

--------------------------------------------------------------------------------
How do I build?

 BookKeeper uses maven as its build system. To build, run "mvn package" from the 
 top-level directory, or from within any of the submodules.

 Useful maven commands are:

 * Clean                     : mvn clean
 * Compile                   : mvn compile
 * Run tests                 : mvn test 
 * Create JAR                : mvn package
 * Run findbugs              : mvn compile findbugs:findbugs
 * Install JAR in M2 cache   : mvn install
 * Deploy JAR to Maven repo  : mvn deploy
 * Run Rat                   : mvn apache-rat:check
 * Build javadocs            : mvn compile javadoc:aggregate
 * Build distribution        : mvn package assembly:single

 Tests options:

 * Use -DskipTests to skip tests when running the following Maven goals:
    'package',  'install', 'deploy' or 'verify'
 * -Dtest=<TESTCLASSNAME>,<TESTCLASSNAME#METHODNAME>,....
 * -Dtest.exclude=<TESTCLASSNAME>
 * -Dtest.exclude.pattern=**/<TESTCLASSNAME1>.java,**/<TESTCLASSNAME2>.java

NOTE:

BookKeeper uses maven-shade-plugin to build shade packages for old versions for
backward compatibility testing. This shade plugin is only able to run at 'package'
phase. So there are two ways to run bookkeeper tests:

 * Run 'mvn clean package' under bookkeeper root directory
 * Run 'mvn clean test' under bookkeeper/bookkeeper-server directory after you run
   'mvn clean install -DskipTests' under bookkeeper directory

--------------------------------------------------------------------------------
How do I run the services?

 Running a Hedwig service requires a running BookKeeper service, which in turn
 requires a running ZooKeeper service (see http://zookeeper.apache.org). To 
 start a bookkeeper service quickly for testing, run:
   
   $ bookkeeper-server/bin/bookkeeper localbookie 10

 This will start a standalone, ZooKeeper instance and 10 BookKeeper bookies.
 Note that this is only useful for testing. Data is not persisted between runs.

 To start a real BookKeeper service, you must set up a ZooKeeper instance and
 run start a bookie on several machines. Modify bookkeeper-server/conf/bk_server.conf
 to point to your ZooKeeper instance. To start a bookie run:

   $ bookkeeper-server/bin/bookkeeper bookie

 Once you have at least 3 bookies runnings, you can start some Hedwig hubs. A 
 hub is a machines which is responsible for a set of topics in the pubsub 
 system. The service automatically distributes the topics among the hubs. 
 To start a hedwig hub:

   $ hedwig-server/bin/hedwig server

 You can get more help on using these commands by running:

   $ bookkeeper-server/bin/bookkeeper help
       and
   $ hedwig-server/bin/hedwig help