sync: misc doc and support edits from Strelka

.gitignore

@@ -40,3 +40,12 @@ src/.project
 *.rar
 *.tar
 *.zip
+
+# IDE and temp files #
+######################
+*.swp
+.ycm_extra_conf.py*
+tags
+*.sublime-project
+*.sublime-workspace
+src/.idea

README.md

@@ -62,8 +62,9 @@ Manta Code Development
 ----------------------
 
 For manta code development and debugging details, see the
-[Manta developer guide] [DeveloperGuide]. This includes details on
-Manta's developement protocols, recommended workflows for investigating
-suspected FP or FN SV calls, and internal documentation details.
+[Manta developer guide] [DeveloperGuide]. This includes details
+on Manta's developement protocols, special build instructions,
+recommended workflows for investigating
+calls, and internal documentation details.
 
 [DeveloperGuide]:docs/developerGuide/README.md

configure

@@ -20,8 +20,8 @@
 
 
 #
-# Top level configuration file for *nix like OS's, note that there's
-# some legacy support for both cygwin and minGW in here but neither
+# Top level configuration file for *nix like OS's, note that there
+# is legacy support for both cygwin and minGW in here but neither
 # have been supported for serveral years.
 #
 
@@ -207,7 +207,7 @@ ERROR: This project cannot be built in the source directory. Please run
 
     """
     mkdir ../build && cd ../build
-    \${MANTA_ROOT_DIR}/configure [configure_options]
+    \${MANTA_ROOT_PATH}/configure [configure_options]
     """
 
 EOF

docs/developerGuide/README.md

@@ -1,10 +1,17 @@
 Manta Developer Guide
 =====================
 
-This guide provides information for manta development, including protocols for
-contirbuting new methods, debugging stability problems, suspected false or missing variant calls and some high-level internal methods documentation.
+This guide provides:
+* protocols for contributing new or modified methods
+* methods to debug stability or runtime issues
+* methods to debug suspected false or missing variant calls
+* high-level architectural documentation
 
-For end user documentation describing how to run Manta and interpret its output, please see the [Manta User Guide](../userGuide/README.md).
+Information is added as pertinent questions/discussions come up in the contributor community,
+so this guide is not intended to provide complete coverage of the above topics.
+
+For end user documentation describing how to run an analysis and interpret its output,
+please see the [User Guide](../userGuide/README.md).
 
 ## Table of Contents
 * [Developer Build Notes](#developer-build-notes)
@@ -19,14 +26,14 @@ developers.
 
 ### Building from source repository vs. versioned code distribution:
 
-When Manta is cloned from github, it is configured for development
+When the source repository is cloned from github, it is configured for development
 rather than user distribution. In this configuration all builds are strict
 such that:
 * all warnings are treated as errors
 * if cppcheck is found any detected cppcheck issue is converted to a build error
 
-Note that in all build configurations, all of Manta's unit tests are run and required
-to pass as part of the default build procedure.
+Note that all unit tests are always run and required to pass for the build
+procedure to complete.
 
 ### Source auto-documentation
 
@@ -88,9 +95,30 @@ solutions allow the c++ code to be browsed, analyzed and compiled to
 the library level.  Note that unit test codes are compiled to
 libraries but cannot be run.
 
-C++11 features used by manta require at least VS2013. A Windows
+C++11 features in use require at least VS2013. A Windows
 installation of cmake is also required to configure and compile.
-Note that the minimum cmake version for Windows is 3.1.0
+Note that the minimum cmake version is 3.1.0 for Windows.
+
+### Automating Portable Binary Builds 
+
+A script is provided to enable a dockerized build process which
+issues Centos5+ or Centos6+ binary tarballs. To do so, ensure you
+have permission to `docker run` on the current system and execute the
+following script:
+
+```
+${MANTA_ROOT_PATH}/scratch/docker/deployment/dockerBuildBinaryTarball.bash ${MANTA_ROOT_PATH2} ${BINARY_BUILD_PREFIX}
+```
+
+The term `${MANTA_ROOT_PATH2}` can point to the current git repo (ie. `${MANTA_ROOT_PATH}`),
+or to an extracted Manta source tarball previously created using the script:
+
+```
+${MANTA_ROOT_PATH}/scratch/make_release_tarball.bash
+```
+
+The choice of virtualized build environment is hard-coded in the deploy script for the time being,
+see the `builderImage` variable.
 
 ## Coding Guidelines
 
@@ -101,10 +129,67 @@ Note that the minimum cmake version for Windows is 3.1.0
   * 4-space indents
   * "ANSI" bracket style
 * Note the above restrictions are enforced by an astyle script which is occasionally run on the master branch (see [run_cxx_formatter.bash](../../scratch/source_check_and_format/run_cxx_formatter.bash))
-* Otherwise, just follow local code conventions
+* Otherwise, follow local code conventions
+
+### Error handling
+
+#### General Policies
+* Exceptions with informative contextual details are encouraged whenever possible.
+* To quickly express invariants it is acceptable to add `assert()`'s first, and transition to exceptions as code stabilizes.
+* Note that the build process will never define `NDEBUG` to compile out assert statements, even in release code.
+* Exceptions are never thrown with the intent to recover -- this is not a web browser. The goal is to:
+  * Fail at the first sign of trouble.
+  * Provide as much helpful contextual information as possible, including context from multiple layers of the stack.
+* Warnings are discouraged. If considering a warning you should probably just fail per the above policy.
+
+#### Exception Details
+* Preferred exception pattern is to use an internal class derived from `boost::exception`:
+
+```c++
+
+#include "common/Exceptions.hh"
+
+#include <sstream>
+
+void
+foo(const char* name)
+{
+    using namespace illumina::common;
+
+    std::ostringstream oss;
+    oss << "ERROR: unrecognized variant scoring model name: '" << name << "'\n";
+    BOOST_THROW_EXCEPTION(LogicException(oss.str()));
+}
+```
+ 
+* Context at the original throw site is often supplemented by a 'catch and release' block to add
+information at a few critical points on the stack. Typically this is information which
+is unavailable at the throw site. Example code is:
+
+```c++
+try
+{
+    realign_and_score_read(_opt,_dopt,sif.sample_opt,_ref,realign_buffer_range,rseg,sif.indel_sync());
+}
+catch (...)
+{
+    log_os << "ERROR: Exception caught in align_pos() while realigning segment: "
+	   << static_cast<int>(r.second) << " of read: " << (*r.first) << "\n";
+    throw;
+}
+```
+
+#### Logging
+
+* At the workflow (python) layer, please write all logging messages through pyflow's logging interface as follows:
+```python
+self.flowLog("Initiating Starling workflow version: %s" % (__version__)
+```
+
+* At the binary (c++) layer, there is no logger at present. Direct all error messaging to `std::cerr`.
 
 ### Unit tests
-* Unit tests are enabled for a subset of the c++ code in manta
+* Unit tests are enabled for a subset of the c++ code
 * All tests use the boost unit test framework
 * All unit tests are required to run and pass as part of every build (including end-user builds)
 * Unit tests are already enabled for every library "test" subdirectory, additional tests in these directories will be automatically detected 

docs/userGuide/README.md

@@ -347,7 +347,7 @@ The VCF ID or 'identifer' field can be used for annotation, or in the case of BN
 
 An example Manta VCF ID is "MantaINS:1577:0:0:0:3:0". The value provided in this field reflects the SV association graph edge(s) from which the SV or indel was discovered. The ID value provided by Manta is primarily intended for internal use by manta developers. The value is guaranteed to be unique within any VCF file produced by Manta, and these ID values are used to link associated breakend records using the standard VCF `MATEID` key. The structure of this ID may change in the future, it is safe to use the entire value as a unique key, but parsing this value may lead to incompatibilities with future updates.
 
-The exact meaning of the ID field for the current Manta version is described in the [following section](mantaDeveloperGuideID.md) of the [Manta developer guide](mantaDeveloperGuide.md).
+The exact meaning of the ID field for the current Manta version is described in the [following section](../developerGuide/ID.md) of the [Manta developer guide](../developerGuide/README.md).
 
 
 #### Converting Manta VCF to BEDPE format

scratch/docker/deployment/dockerBuildBinaryTarball.bash

@@ -1,22 +1,28 @@
 #!/usr/bin/env bash
 
 set -o nounset
-set -o xtrace
+#set -o xtrace
 
 buildLabel=
-if ! [ $# != 3 ]; then
+if [ $# != 2 ]; then
     cat <<END
 Create binary distro tarball using docker.
 
 usage: $0 distro_root_directory binary_distro_prefix
 
-  distro_root_directory - root directory of the package git repository
-  binary_distro_prefix - filename prefix of tarball and directory name
-                         used when tarball is unpacked
+  distro_root_directory - Root directory of the package git repository or source release
+  binary_distro_prefix - Filename prefix of tarball. This will also be the
+                         directory name used when tarball is unpacked
+
 END
     exit 2
 fi
 
+error() {
+    echo "ERROR: $@" 2>&1
+    exit 1
+}
+
 rootDir="$1"
 buildLabel="$2"
 
@@ -32,15 +38,18 @@ rootDir=$(rel2abs $rootDir)
 
 # check that rootDir conatins expected files:
 if ! [ -f $rootDir/configure ]; then
-    echo "Can't find package configure script. Expected location is '$rootDir/configure'" 2>&1
-    exit 1
+    error "Can't find package configure script. Expected location is '$rootDir/configure'"
 fi
 
 
 # in dockerfile directory:
 tag="deployment:$builderImage"
 sudo docker build -t $tag $scriptDir/$builderImage
 
+if [ $? -ne 0 ] ; then
+    error "Failed to build docker deployment image"
+fi
+
 # in scratch
 #unpack src tarball and cd into tarball root
 
@@ -70,5 +79,10 @@ chmod 777 $buildLabel.tar.bz2
 ENDE
 
 sudo docker run -v $rootDir:$dmount -t $tag bash $dmount/$installScriptFilename
+
+if [ $? -ne 0 ] ; then
+    error "Failed to build binary package release"
+fi
+
 mv $rootDir/$buildLabel.tar.bz2 .