Initialization of the default branch (GeomScale#6)

* add volesti as submodule

* add external and rhino interface

* rename to misha

* rename main subfolder

* update readme file

* update submodule

* fix binding files and setup.py

* implement slow and fast fva

* implement fba

* inner ball computation

* remove subfolder in folder subroutines

* implement two functions to compute the right nullspace of a matrix

* fix imports in vlestipy.pyx

* fix imports in nullspace.py

* Remove eigen from repository and fetch it from gitlab

* Refactor the repository structure

* Switch to LGPL3

* implement gmscale to compute an efcient scaling of the polytope

* implement function to aaply the sclaing derived from gmscale

* restructure the code

* initialize tests with inner_ball tester

* implement function to remove almost redundant facets

* add a script to load models given by a json file

* add implementation to load a mat file, e.g. a bigg model

* add test for the loading functions

* add tests for fva and fba

* add simplest e_coli test model from bigg database and a scaling test

* add test for the nullspace computation

* add test for computing the full dimensional polytope from the simplest e_coli model

* remove unused outputs from functions

* add minimal information to readme file

* minor update to readme file

* improve setup.py

* improve tests

* add automatic installation for PySPQR

* add requirements.txt and update the readme file accordingly

* remove unused files

* resolve review comments

* improve test files

* fix bugs in preprocessing methods and improve the tests

* add unittests

* improve tests and add copyright holdings

* update tests

* seperate fast-gurobi implementations from the default-slow ones

* add eigen submodule to version 3.3

* add poetry.lock and .toml. Change the setup.py and readme files accordingly

* fix errors in toml and setup files

* update comments

* add build.py script to habdle cython dependency

* set PEP 8 code style and description comments to all functions according to PEP 257

* fix unit_tests

* request smaller accuracy from linprog in unittests

* minor updates to unittests

* update volesti submodule and add instructions to download boost library

* reduce the tolerances in unittests

* update poetry instructions in readme file

* update installation instructions

* generalize the implmentation od fva method, modify tests accordingly

* separate the building of dingocpp from dingo package

* add import of volestipy to init file

* remove volesti as submodule

* add volesti from TolisChal fork

* fix include dirs in setup.py

* implement bindings for the mmcs algorithm

* implment argument parser

* implement the functionality of main

* improve the main and develop the main function dingo_pipeline()

* complete main pipelines

* update python comments to document the additional functions

* update volesti submodule to current branch

* expose to python the parallel mmcs

* implement a matlab wrapper for models and change the matlab loading function accordingly

* add fva and fba to the main function

* update the documentation

* update the documentation

* update the documentation

* update the documentation

* complete the documentation of the main function of dingo

* improve second part of the documentation

* write the second part of the documentation

* split the documentation to 2 files

* link the urls to the documentation files

* add a url in readme.md for the documentation

* update volesti submodule

* improve memory allocation in mmcs

* improve main pipelines in volesti

* improve the main function to declare model's name

* add histogram plotting

* add plotting in the documentation

* fix image loading in the documentation

* improve documentation

* fix the code style with black

* add covid-19 model

* add covid-19 pipeline

* black code style in new script

* update submodule and fix circular imports

* update the documentation

Co-authored-by: Vissarion Fisikopoulos <fisikop@gmail.com>

.gitignore

@@ -0,0 +1,10 @@
+build
+dist
+boost_1_75_0
+dingo.egg-info
+*.pyc
+*.so
+volestipy.cpp
+volestipy.egg-info
+*.npy
+

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "eigen"]
+	path = eigen
+	url = https://gitlab.com/libeigen/eigen.git
+[submodule "volume_approximation"]
+	path = volume_approximation
+	url = https://github.com/TolisChal/volume_approximation.git

README.md

@@ -1,2 +1,52 @@
 # dingo
-Analyzing metabolic networks with MCMC sampling
+
+A python library for metabolic networks sampling and analysis.  
+
+## Installation
+
+To load the submodules that dingo uses run: `git submodule update --init`.  
+
+You will need to download and unzip the boost library:
+```
+wget -O boost_1_75_0.tar.bz2 https://dl.bintray.com/boostorg/release/1.75.0/source/boost_1_75_0.tar.bz2
+tar xjf boost_1_75_0.tar.bz2
+rm boost_1_75_0.tar.bz2
+```
+
+Then, you need to install the dependencies for PySPQR library; for debian/ubuntu linux run:
+
+```
+apt-get install libsuitesparse-dev
+```
+
+To install the python dependencies install [poetry](https://python-poetry.org/). Then, run:  
+```
+poetry shell
+poetry install
+```
+
+To exploit the fast implementations of dingo you have to install the [Gurobi solver](https://www.gurobi.com/). Run:  
+
+```
+pip3 install -i https://pypi.gurobi.com gurobipy
+```
+
+Then, you will need a [license](https://www.gurobi.com/downloads/end-user-license-agreement-academic/). For more information we refer to gurobi [download center](https://www.gurobi.com/downloads/).  
+
+
+## Unit tests
+
+Now, you can run the unit tests by the following commands:  
+```
+python3 tests/unit_tests.py
+```
+
+If you have installed gurobi sucesfully then run:  
+```
+python3 tests/fast_implementation_test.py
+```
+
+## Documentation
+
+Read [dingo documentation](https://github.com/GeomScale/dingo/tree/initialize_develop/doc)
+

build.py

@@ -0,0 +1,43 @@
+import os
+
+# See if Cython is installed
+try:
+    from Cython.Build import cythonize
+# Do nothing if Cython is not available
+except ImportError:
+    # Got to provide this function. Otherwise, poetry will fail
+    def build(setup_kwargs):
+        pass
+
+
+# Cython is installed. Compile
+else:
+    from setuptools import Extension
+    from setuptools.dist import Distribution
+    from distutils.command.build_ext import build_ext
+
+    # This function will be executed in setup.py:
+    def build(setup_kwargs):
+        # The file you want to compile
+        extensions = ["dingo/volestipy.pyx"]
+
+        # gcc arguments hack: enable optimizations
+        os.environ["CFLAGS"] = [
+            "-std=c++11",
+            "-O3",
+            "-DBOOST_NO_AUTO_PTR",
+            "-ldl",
+            "-lm",
+        ]
+
+        # Build
+        setup_kwargs.update(
+            {
+                "ext_modules": cythonize(
+                    extensions,
+                    language_level=3,
+                    compiler_directives={"linetrace": True},
+                ),
+                "cmdclass": {"build_ext": build_ext},
+            }
+        )

dingo/__init__.py

@@ -0,0 +1,225 @@
+# dingo : a python library for metabolic networks sampling and analysis
+# dingo is part of GeomScale project
+
+# Copyright (c) 2021 Apostolos Chalkis
+
+# Licensed under GNU LGPL.3, see LICENCE file
+
+import numpy as np
+import sys
+import os
+import pickle
+from dingo.fva import slow_fva
+from dingo.fba import slow_fba
+from dingo.loading_models import read_json_file
+from dingo.inner_ball import slow_inner_ball
+from dingo.nullspace import nullspace_dense, nullspace_sparse
+from dingo.scaling import (
+    gmscale,
+    apply_scaling,
+    remove_almost_redundant_facets,
+    map_samples_to_steady_states,
+)
+from dingo.parser import dingo_args
+from dingo.pipelines import (
+    from_model_to_steady_states_pipeline,
+    from_polytope_to_steady_states_pipeline,
+    fva_pipeline,
+    fba_pipeline,
+    plot_histogram,
+)
+
+try:
+    import gurobipy
+    from dingo.gurobi_based_implementations import fast_fba, fast_fva, fast_inner_ball
+except ImportError as e:
+    pass
+
+from volestipy import HPolytope
+
+
+def get_name(args_network):
+
+    position = [pos for pos, char in enumerate(args_network) if char == "/"]
+
+    if args_network[-4:] == "json":
+        if position == []:
+            name = args_network[0:-5]
+        else:
+            name = args_network[(position[-1] + 1) : -5]
+    elif args_network[-3:] == "mat":
+        if position == []:
+            name = args_network[0:-4]
+        else:
+            name = args_network[(position[-1] + 1) : -4]
+    print(name)
+    return name
+
+
+def dingo_main():
+    """A function that (a) reads the inputs using argparse package, (b) calls the proper dingo pipeline
+    and (c) saves the outputs using pickle package
+    """
+
+    args = dingo_args()
+
+    if args.metabolic_network is None and args.polytope is None and not args.histogram:
+        raise Exception(
+            "You have to give as input either a model or a polytope derived from a model."
+        )
+
+    if args.metabolic_network is None and ((args.fva) or (args.fba)):
+        raise Exception("You have to give as input a model to apply FVA or FBA method.")
+
+    if args.output_directory == None:
+        output_path_dir = os.getcwd()
+    else:
+        output_path_dir = args.output_directory
+
+    if os.path.isdir(output_path_dir) == False:
+        os.mkdir(output_path_dir)
+
+    # Move to the output directory
+    os.chdir(output_path_dir)
+
+    if args.model_name is None:
+        if args.metabolic_network is not None:
+            name = get_name(args.metabolic_network)
+    else:
+        name = args.model_name
+
+    if args.histogram:
+
+        if args.steady_states is None:
+            raise Exception(
+                "A path to a pickle file that contains steady states of the model has to be given."
+            )
+
+        if args.metabolites_reactions is None:
+            raise Exception(
+                "A path to a pickle file that contains the names of the metabolites and the reactions of the model has to be given."
+            )
+
+        if int(args.reaction_index) <= 0:
+            raise Exception("The index of the reaction has to be a positive integer.")
+
+        file = open(args.steady_states, "rb")
+        steady_states = pickle.load(file)
+        file.close()
+
+        steady_states = steady_states[0]
+
+        file = open(args.metabolites_reactions, "rb")
+        meta_reactions = pickle.load(file)
+        file.close()
+
+        reactions = meta_reactions[1]
+
+        plot_histogram(
+            steady_states[int(args.reaction_index) - 1],
+            reactions[int(args.reaction_index) - 1],
+            int(args.n_bins),
+        )
+
+    elif args.fva:
+
+        result_obj = fva_pipeline(args)
+        result_obj = result_obj[4:]
+
+        with open("dingo_fva_" + name, "wb") as dingo_fva_file:
+            pickle.dump(result_obj, dingo_fva_file)
+
+    elif args.fba:
+
+        result_obj = fba_pipeline(args)
+
+        with open("dingo_fba_" + name, "wb") as dingo_fba_file:
+            pickle.dump(result_obj, dingo_fba_file)
+
+    elif args.metabolic_network is not None:
+
+        result_obj = from_model_to_steady_states_pipeline(args)
+
+        if args.preprocess_only:
+
+            polytope_info = result_obj[:4]
+            network_info = result_obj[4:6]
+            metabol_reaction = result_obj[6:]
+
+            polytope_info = polytope_info + (name,)
+
+            with open("dingo_polytope_" + name, "wb") as dingo_polytope_file:
+                pickle.dump(polytope_info, dingo_polytope_file)
+
+            with open(
+                "dingo_metabolites_reactions_" + name, "wb"
+            ) as dingo_metabolreactions_file:
+                pickle.dump(metabol_reaction, dingo_metabolreactions_file)
+
+            with open("dingo_minmax_fluxes_" + name, "wb") as dingo_network_file:
+                pickle.dump(network_info, dingo_network_file)
+
+        else:
+
+            polytope_info = result_obj[:7]
+            network_info = result_obj[7:9]
+            metabol_reaction = result_obj[9:11]
+            steadystates = result_obj[11:]
+
+            polytope_info = polytope_info + (name,)
+
+            with open("dingo_rounded_polytope_" + name, "wb") as dingo_polytope_file:
+                pickle.dump(polytope_info, dingo_polytope_file)
+
+            with open("dingo_minmax_fluxes_" + name, "wb") as dingo_network_file:
+                pickle.dump(network_info, dingo_network_file)
+
+            with open(
+                "dingo_metabolites_reactions_" + name, "wb"
+            ) as dingo_metabolreactions_file:
+                pickle.dump(metabol_reaction, dingo_metabolreactions_file)
+
+            with open("dingo_steady_states_" + name, "wb") as dingo_steadystates_file:
+                pickle.dump(steadystates, dingo_steadystates_file)
+
+    else:
+
+        file = open(args.polytope, "rb")
+        object_file = pickle.load(file)
+        file.close()
+
+        if len(object_file) == 5:
+            result_obj = from_polytope_to_steady_states_pipeline(
+                args, object_file[0], object_file[1], object_file[2], object_file[3]
+            )
+        elif len(object_file) == 8:
+            result_obj = from_polytope_to_steady_states_pipeline(
+                args,
+                object_file[0],
+                object_file[1],
+                object_file[2],
+                object_file[3],
+                object_file[4],
+                object_file[5],
+            )
+        else:
+            raise Exception("The input file has to be generated by dingo package.")
+
+        polytope_info = result_obj[:7]
+        network_info = result_obj[7:]
+
+        if args.model_name is None:
+            name = object_file[-1]
+
+        polytope_info = polytope_info + (name,)
+
+        with open("dingo_double_rounded_polytope_" + name, "wb") as dingo_polytope_file:
+            pickle.dump(polytope_info, dingo_polytope_file)
+
+        with open("dingo_steady_states_" + name, "wb") as dingo_network_file:
+            pickle.dump(network_info, dingo_network_file)
+
+
+if __name__ == "__main__":
+
+    dingo_main()

dingo/__main__.py

@@ -0,0 +1,10 @@
+# dingo : a python library for metabolic networks sampling and analysis
+# dingo is part of GeomScale project
+
+# Copyright (c) 2021 Apostolos Chalkis
+
+# Licensed under GNU LGPL.3, see LICENCE file
+
+from dingo import dingo_main
+
+dingo_main()