From 1d4739cce050a534947cc5298511df65c2db543e Mon Sep 17 00:00:00 2001 From: Evgeny Metelkin Date: Fri, 27 Mar 2020 12:35:38 +0300 Subject: [PATCH] draft documentation --- README.md | 93 ++++++++++++++++++--- docs/cli-references.md | 183 +++++++++++++++++++++++++++++++++++++++++ docs/cli.md | 53 ------------ 3 files changed, 265 insertions(+), 64 deletions(-) create mode 100644 docs/cli-references.md delete mode 100644 docs/cli.md diff --git a/README.md b/README.md index c52994ae..c46ec63f 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,4 @@ [![Heta project](https://img.shields.io/badge/%CD%B1-Heta_project-blue)](https://hetalang.github.io/) -[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://insysbio.github.io/heta-compiler) [![GitHub issues](https://img.shields.io/github/issues/insysbio/heta-compiler.svg)](https://GitHub.com/insysbio/heta-compiler/issues/) [![Travis](https://travis-ci.org/insysbio/heta-compiler.svg?branch=master)](https://travis-ci.org/insysbio/heta-compiler) [![Coverage Status](https://coveralls.io/repos/github/insysbio/heta-compiler/badge.svg?branch=master)](https://coveralls.io/github/insysbio/heta-compiler?branch=master) @@ -12,11 +11,58 @@ Read more about Heta format and Heta-based platforms on Heta project homepage: . +## Table of contents + +- [Introduction](#introduction) +- [Export formats](#export-formats) +- [Installation](#installation) +- [Usage of command line interface](#usage-of-command-line-interface) +- [Usage in NodeJS packages](#usage-in-nodejs-packages) +- [Known issues and limitations](#known-issues-and-limitations) +- [Getting help](#getting-help) +- [Contribute](#contribute) +- [License](#license) +- [Authors and history](#authors-and-history) + +## Introduction + +**Heta compiler** is a tool for the development of Quantitative Systems Pharmacology and Systems Biology platforms. It allows combining modules written in different formats like: [Heta language code](https://hetalang.github.io/#/specifications/), Excel sheets, [JSON](https://en.wikipedia.org/wiki/JSON)/[YAML](https://en.wikipedia.org/wiki/YAML) formatted structures, [SBML](http://sbml.org/) and transforming into the dynamical model/models of different formats. + +Quantitative Systems Pharmacology (QSP) is a discipline that uses mathematical computer models to characterize biological systems, disease processes and drug pharmacology. QSP typically deals with mechanism-based dynamical models described by ODE systems. Sometimes the modeling systems includes hundred or thousand of components and developed by a research group involving people with different expertise. + +Heta compiler can be used as the framework for a QSP modeling project of any size and complexity. It can be easily integrated with existed infrastructure, workflows or used as a part of the CI/CD strategy. The pre-formulated requirements are: + +- storing the QSP models and data in integrated infrastructure; +- support iterative modeling platform updates (continuous development approach); +- support of models written in human-readable text and table formats; +- export models and data to different popular formats on the fly. + +## Export formats + +Heta compiler was created to support exporting to different popular modeling formats. One of the main effort of the developers is to extend a list of supporting formats and allow people to have the same results working in different tools. The current version supports the following formats: + +- DBSolveOptimum .SLV files [link](http://insysbio.com/en/software/db-solve-optimum) +- SBML L2V4 [link](http://sbml.org/) +- mrgsolve +- mrgsolve .CPP files [link](https://mrgsolve.github.io/user_guide/) +- Simbiology/Matlab .M files [link](https://www.mathworks.com/products/simbiology.html) +- Matlab describing ODEs file [link](https://www.mathworks.com/help/matlab/ordinary-differential-equations.html) +- SimSolver/Julia module + +- JSON formatted file +- YAML formatted file +- Excel sheets + ## Installation -1. Install [NodeJS](https://nodejs.org/en/). Currently NodeJS v8 and newer are supported. Check version after install +Before start [NodeJS](https://nodejs.org/en/) must be installed. Currently NodeJS v8 and newer are supported. + +The next steps should be taken using console (shell): **cmd**, **PowerShell**, **sh**, **bash** depending on your operating system. + +1. Check Node version. It should be >= 8.0.0. ```bash node -v + # v8.9.2 ``` 2. The latest stable version of Heta compiiler can be installed from npm @@ -28,11 +74,13 @@ Read more about Heta format and Heta-based platforms on Heta project homepage: < npm i -g git+https://github.com/insysbio/heta-compiler.git ``` -## Command line interface (CLI) usage +## Usage of command line interface -*More options and examples see in [CLI page](./docs/cli)* +*More options and examples see on [CLI references](./docs/cli-references)* -1. Create simple Heta file: *index.heta* +The following example creates one Heta module and compiles the model it to SBML format. For example you want to create platform in directory "platform/directory/path/" + +1. Create Heta file: *index.heta* in target directory: ```heta comp1 @Compartment; s1 @Species { compartment: comp1 }; @@ -46,19 +94,20 @@ Read more about Heta format and Heta-based platforms on Heta project homepage: < sbml1 @SBMLExport; ``` -2. Compile platform +2. Be sure you are in target directory: `cd platform/directory/path` if not. Compile platform ```bash - cd ./path/to/platform/folder heta build ``` +Heta builder takes "index.heta" file (module) as default, reads it and transforms to SBML file as declared. -3. See results in directory **dist** +3. See results in directory **dist**. +Several files can be loaded using `include` statement inside "index.heta", see [specifications](https://hetalang.github.io/#/specifications/include). ## Usage in NodeJS packages *See more information on -~~[API page](./api)~~.* +~~[API references](./api-references)~~.* ```javascript const { Container } = require('heta-compiler'); @@ -86,14 +135,36 @@ console.log(output[0].content); console.log(c.logger.hasErrors); ``` +## Known issues and limitations + +The tool is under active development so there ara a lot of features to implement. To help us prioritize them write the issue. + +## Getting help + + - Read the documentation of Heta on + - Use [Gitter Chatroom](https://gitter.im/hetalang/community?utm_source=readme) to chat. + - Use [Issue Tracker](https://github.com/insysbio/heta-compiler/issues) + ## Contribute -- [Issue Tracker](https://github.com/insysbio/heta-compiler/issues) - [Source Code](https://github.com/insysbio/heta-compiler) +- [Issue Tracker](https://github.com/insysbio/heta-compiler/issues) - See also contributing in [Heta project](https://hetalang.github.io/#/CONTRIBUTING) ## License -Licensed under the Apache License, Version 2.0. +Licensed under the Apache License, Version 2.0. See the [LICENSE](./LICENSE) text. + +## Authors and history + +The original author of the project is [Evgeny Metelkin](https://github.com/metelkin). The tool was inspired by the idea that large scale dynamical systems used in QSP and SB require the specific tool which allows writing model code in unified formats and transforming them depending on one's needs. Working with large models should be as easy as with the small ones. + +- The initial prototype 0.1.x was developed in 2017 and named as **qs3p** (quantitative systems pharmacology programming platform). It was used in several internal and service [InSysBio LLC](https://insysbio.com) projects including [IRT](https://irt.insysbio.com/) and **Alzheimer disease consortium**. + +- The next version **qs3p-js** 0.3.x used the updated format of platfrorm components and a new approach for storing them (without database). A set of new exporting formats was supported. + +- The **qs3p-js** of version 0.4.x was developed in 2019 to supports Heta code including actions, modules, namespaces. It was used as the main infrastructure for the development of the large- and middle-scale QSP platforms developed in the framework of InSysBio services. + +- Starting from 2020 the tool was renamed to **Heta compiler** and published as a Free Open Source project on [GitHub](https://GitHub.com/insysbio/heta-compiler) under Apache 2.0 license. Since then Heta compiler has been developed in the framework of [Heta project](https://hetalang.github.io/). Copyright 2019-2020 InSysBio LLC \ No newline at end of file diff --git a/docs/cli-references.md b/docs/cli-references.md new file mode 100644 index 00000000..63db0fdf --- /dev/null +++ b/docs/cli-references.md @@ -0,0 +1,183 @@ +# Command Line Interface + +*This page describe how to work with Heta compiler from console (shell).* + +If command are not available check your Heta compiler installation and content of system paths (`PATH` variable in Windows). + +## Table of contents + +- ["heta" command](#"heta"-command) +- ["heta build" command](#"heta-build"-command) +- ["heta init" command](#"heta-init"-command) +- ["heta help" command](#"heta-help"-command) +- [Declaration file format](#declaration-file-format) + +## "heta" command + +`heta` is prefix command for the other commands. Writing the command alone prints the information about the tool and available options. `heta help` does the same. + +``` +$ heta +Usage: heta [options] [command] + +Command line utilities for working with Heta compiler + version: 0.4.31 + license: Apache-2.0 + +Options: + -v, --version output the version number + -h, --help output usage information + +Commands: + build [dir] Compile Heta based platform and create set of export files. + init [dir] Create template platform files in directory + help [cmd] display help for [cmd] +``` + +## "heta build" command + +`heta build` runs the compilation of the platform. +It uses the main source (index) file as a module to compile the platform. + +The default run of `heta build` (no options set, no declaration file) does the following: + +1. Looking for **index.heta** in parent working directory of shell. +2. Run parsing of index file as module of type "heta" and all files (modules) mentioned by `include` statement inside `index.heta`. +4. Creation of export files created by `@Export` instances to **dist/** directory. +5. If there are compiling errors the file **build.log** will be created in working directory. + +### Running build without declaration file + +CLI options allow setting specific options for build command. Use `heta build -h` to see the list of options. + +**dir** here is a dirpath which will be a working directory (WD) of Heta compiler run. Absolute and relative path are possible here. If not set the shell WD will be used as WD of Heta. + +List of build options: +| option | type | default | description | +|--|--|--|--| +| --source | \| index.heta | Path to main heta module. This allows using another name and path of index Heta module. | +| --type | \ | heta | Type of source file. This option allows to select type of module which will be applied for parsing. Available values: heta/xlsx/json/yaml/sbml. | +| --debug | | | Working in debugging mode. All parsed files will be saved in JSON files in **meta** directory.| +| --skip-export | | | If set no export files will be created. | +| --log-mode | string | error | The rule in which case the log file should be created. Possible values are: never/error/always | +| -d, --declaration| string | platform | The filepath to declaration file (see below) without extension. The command will search the declaration file based on option trying a set of extensions: .json/.json5/.yml. + +#### Example 1 + +Let's our shell working directory is **/path/to/platform/**. Our Heta module is located in **src/table.xlsx** subdirectory and has a type xlsx (Excel sheet). To run compilation and save export files to **dir** one should use the command. +``` +heta build --source src/table.xlsx --type xlsx +``` + +#### Example 2 + +Run compilation without exporting files. +``` +heta build --skip-export +``` + +### Running build with declaration file + +Declaration is a file in specific format which is located in working directory of modeling platform. As default it has name **platform.json** and it is JSON formatted. It has two purposes: +- It annotates the developed modeling platform by some specific properties like id, title, authors, references, etc. +- It declares the specific options of modeling platform: files location, outputs, etc. It substitutes of using `platform build` options but has much more options. + +The declaration file can be in one of the following formats: [JSON](https://en.wikipedia.org/wiki/JSON), [JSON5](https://json5.org/), [YAML](https://en.wikipedia.org/wiki/YAML) with the same schema. + +It is a good idea to start the model development from creation of **platform.json** file. If declaration file is set you have not to use additional options in `heta build`. + +To create a draft declaration file use ["heta init" command](#"heta-init"-command). To see all properties of declaration file see [declaration file format](#declaration-file-format). + +## "heta init" command + +`heta init` creates template files for QSP platform. Running the command without options will create template in current working directory. You can set another directory for creating template using **dir** at the end of command. + +| option | type | description | +|--|--|--| +| -f, --force || This option allows rewriting the existed files and directories. | +| -s, --silent || Run initialization without any questions with default options. | + +After running the command a developer should answer to series of questions about the initialized platform in prompt mode. To create the default platform use `--silent` option. + +#### Example +``` +$ heta init -f test +Creating a template platform in directory: "Y:\qs3p-js\test"... +? Platform id test-platform +? Platform notes Temporal platform to test initialization +? Platform version v0.1.0 +? Platform license UNLICENSED +? Set options No +? Select file types xlsx +Platform template is created. +{ + ... +} +DONE. +``` + +## "heta help" command + +To obtain a list of options and command description one can use command help. +It can be done by two ways: +``` +heta help +``` +or +``` +heta -h +``` + +## Declaration file format + +There are properties in declaration file which do not change compilation process. They can be used for annotation of a developed QSP platform for summarizing annotation and auxilary information. + +| option | type | CLI option | default value | description | +|--|--|--|--|--| +| id | ID ||| Identifier of platform. It should be an unique string without spaces. *Annotation element* | +| notes | string ||| Text describing QSP platform. *Annotation element.*| +| version | string ||| Version of the developed QSP platform. It is recommended to follow [semver](https://semver.org/) rules. | +| keywords | string[] ||| Array of keywords for future indexing platform. *Annotation element.* | +| homepage | string ||| URL to the page supporting the developed platform if it exists. *Annotation element.* | +| repository | object || {} | Container for storing info where project source code is located. | +| repository.type | string ||| Type of repository, for exmple: "git". *Annotation element.* | +| repository.url | string ||| URL of source repository, for example: "https://github.com/insysbio/heta-case-mini.git". *Annotation element.* | +| license | string ||| Short udentifier under which license the platform is distributed. *Annotation element.* | +| private | boolean ||| Set true if a platform must not be shared in public repositories. *Annotation element.*| +| contributors | string[] ||| Array of authors and contributors. *Annotation element.* | +| builderVersion | string ||| The required version of Heta compiler which should analyze the code. The string must follow the rules of [semantic versioning](https://docs.npmjs.com/about-semantic-versioning). See also [semantic versioning calculator](https://semver.npmjs.com/). | +| importModule | object | | {} | Container for the description of index module. | +| importModule.source | string | --source | index.heta | Path to main heta module. This allows using any name and path of index Heta module. Absolute and relative filepaths are applicable. Example: "src/table.xlsx" | +| importModule.type | string | --type | heta | Type of source file. This option set type of module which will be applied for parsing. Available values: heta/xlsx/json/yaml/sbml. | +| options | object | | {} | Container to store compiler options. | +| options.logMode | string | --log-mode | error | The rule in which case the log file should be created. Possible values are: never/error/always. | +| options.logPath | string | | build.log | Filename or filepath where the log file should be created. | +| options.logLevel | string | | info | When parsing the compiler prints the messages to the shell. Here you can set a level of printing messages. Possible values: "info", "warning", "error". For example if you set "warning", only warnings and errors will be printed. | +| options.skipExport | boolean | --skip-export | false | If set no export files will be created. | +| options.distDir | string | | dist | At default all export files are created inside **dist** directory. The option can set the another target for storing outputs. | +| options.debug | boolean | --debug | false | Working in debugging mode. All parsed files will be saved in JSON files in meta directory. | +| options.metaDir | string | | meta | If `options.debug` is set this option changes the target directory for meta files. | +| options.exitWithoutError | boolean | | false | As default if there are some errors in compilation the `heta build` command return status 1 to console (which means error). If you set true this will return 0. This can be helpful for using autotesting and CI/CI automatization. | + +Using neither declaration file nor CLI options is equivalent to the following declaration: +```json +{ + "builderVersion": "*", + "options": { + "logMode": "error", + "logPath": "build.log", + "logLevel": "info", + "distDir": "dist", + "metaDir": "meta", + "debug": false, + "skipExport": false, + "exitWithoutError": false + }, + "importModule": { + "source": "index.heta", + "type": "heta", + "sheet": 1, + "omitRows": 0 + } +} +``` diff --git a/docs/cli.md b/docs/cli.md deleted file mode 100644 index 9da28ca7..00000000 --- a/docs/cli.md +++ /dev/null @@ -1,53 +0,0 @@ -# Command Line Interface - -## "heta" command - -## "heta init" command - -## "heta build" command - -## Running build without declaration file - -## Running build with declaration file - -## Declaration file format - -**Root properties** -| option | type | CLI option | default value | description | -|--|--|--|--|--| -| id | ID ||| Identifier of platform | -| title | string ||| Title of platform | -| notes | string ||| - -**option properties** -| option | type | CLI option | default value | description | -|--|--|--|--|--| - -**importModule properties** -| option | type | CLI option | default value | description | -|--|--|--|--|--| -| source | string | --source | index.heta |Filepath of index Heta module | -| type | "heta"/"xlsx"/"json"/"yaml"/"sbml" | --type | heta | Path to the main (index) module of platform | - -Using neither declaration file nor CLI options is equivalent to the following declaration: -```json -{ - "builderVersion": "*", - "options": { - "logMode": "error", - "logPath": "build.log", - "logLevel": "info", - "distDir": "dist", - "metaDir": "meta", - "debuggingMode": false, - "skipExport": false, - "exitWithoutError": false - }, - "importModule": { - "source": "index.heta", - "type": "heta", - "sheet": 1, - "omitRows": 0 - } -} -```