Better Eol Handling && Install Functionality && Partial Rework of Rea…

…dme (#12)

* Reworked config section in readme. Fixed wrong project description.

* Added && as eof in examples.

* Added pretty print functions for chained commands.

* Changed section in readme from installation_instruction to installation-instruction.

* Added non functional install command.

* Added install functionality. Install command will now execute template rendered command. Fixed error echo/print in main. Added colors to echo/print of success or errors. Added functionality to get_flags_and_options regarding detecting flags and required flags. Added a test config for errors.

* Extended test of get_flags_and_options.

* Added changelog.

CHANGELOG.md

@@ -9,10 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+* Added installation functionality.
+* Added pretty print to show command.
+* Added many colors for cli output.
 * Added PyPi version badge.
 
 ### Changed
 
+* Flags are now handled properly (requiered and default).
+* Reworked config section in readme.
+* Fixed wrong description of project in readme.
+* Fixed wrong section title in readme.
+
 ### Removed
 
 

README.md

@@ -2,7 +2,7 @@
 
 # `installation-instruction`
 
-**Library for checking and parsing installation instruction schemas.**
+**Library and CLI for generating installation instructions from json schema and jinja templates.**
 
 [![GitHub License](https://img.shields.io/github/license/instructions-d-installation/installation-instruction)](./LICENSE)
 [![PyPI - Version](https://img.shields.io/pypi/v/installation-instruction)](https://pypi.org/project/installation-instruction/)
@@ -28,7 +28,7 @@ python -m pip install installation-instruction
 ```
 
 
-### installation_instruction
+### installation-instruction
 
 *(Don't try at home.)*
 ```yaml
@@ -69,18 +69,30 @@ Options are dynamically created with the schema part of the config file.
 
 ## Config
 
-The config is comprised of a single file. (Currently there is no fixed filename.) 
-For ease of use you should use the file extension `.yml.jinja` and develope said config file as two seperate files at first.
-The config file has two parts delimited by `------` (6 or more `-`).   
-The first part is the schema (*What is valid user input?*). The second part is the template (*What is the actual command for said user input?*).
-The first part must be a valid [JSON Schema](https://json-schema.org/) in [JSON](https://www.json.org/json-en.html) or to JSON capabilites restricted [YAML](https://yaml.org/) and the second part must be a valid [jinja2 template](https://jinja.palletsprojects.com/en/3.0.x/templates/).
-The exception to this is that `anyOf` and `oneOf` are only usable for enum like behaviour on the schema side.
-Instead of an `enum` you might want to use `anyOf` with `const` and `tile` properties.
-The `title` of a property is used for the pretty print name, while the `description` is used for the help message.
-There exists a jinja2 macro called `raise`, which is usefull if there is actually no installation instruction for said user input.
-All lineends in the template are removed after render, which means that commands can be splitted within the template (`conda install {{ "xyz" if myvar else "abc" }}` ).
-This also means that multiple commands need to be chained via `&&`.
-For examples please look at the [examples folder](./examples/).
+* The config is comprised of a single file `install.cfg`.
+* The config has two parts delimited by `------` (6 or more `-`).
+* Both parts should be developed in different files for language server support.
+
+
+### Schema
+
+* The first section of the config is a [json-schema](https://json-schema.org/).
+* It can be written in [JSON](https://www.json.org/json-en.html) or to JSON capabilites restricted [YAML](https://yaml.org/).
+* `title` are used for pretty print option names.
+* `description` is used for the options help message.
+* `anyOf` with nested `const` and `title` are a special case as a replacement for `enum` but with pretty print name.
+
+
+### Template
+
+* You can have as much whitespace and line breaks as you wish in and inbetween your commands.
+* Commands must be seperated by `&&`! (`pip install installation-instruction && pip uninstall installation-instruction`.)
+* If you wish to stop the render from within the template you can use the macro `raise`. (`{{ raise("no support!") }}`.) 
+
+
+### MISC
+
+Please have a look at the [examples](./examples/).
 
 
 ## Development installation

examples/scikit-learn/scikit-learn-instruction.schema.yml.jinja

@@ -36,31 +36,26 @@ additionalProperties: false
 ------
 
 {% if package == "conda" %}
-    conda create -n sklearn-env -c conda-forge scikit-learn \n
-    conda activate sklearn-env
+    conda create -n sklearn-env -c conda-forge scikit-learn &&
+    conda activate sklearn-env &&
 {% else %}
     {% if os == "Linux"%}
         {% if virtualenv%}
-            python3 -m venv sklearn-venv \n
-            source sklearn-venv/bin/activate \n
-            pip3 install -U scikit-learn
-        {% else %}
-            pip3 install -U scikit-learn
+            python3 -m venv sklearn-venv &&
+            source sklearn-venv/bin/activate &&
         {% endif %}
+        pip3 install -U scikit-learn
     {% else %}
         {% if virtualenv%}
             {% if os == "macOS"%}
-                python -m venv sklearn-venv \n
-                source sklearn-venv/bin/activate \n
-                pip install -U scikit-learn 
+                python -m venv sklearn-venv &&
+                source sklearn-venv/bin/activate &&
             {% else %}
-                python -m venv sklearn-venv \n
-                sklearn-venv\Scripts\activate \n
-                pip install -U scikit-learn 
+                python -m venv sklearn-venv &&
+                sklearn-venv\Scripts\activate &&
             {% endif %}
-        {% else %}
-            pip install -U scikit-learn
         {% endif %}
+        pip install -U scikit-learn
     {% endif %}
 {% endif %}
 

examples/spacy/spacy-instruction.schema.yml.jinja

@@ -92,7 +92,7 @@ additionalProperties: false
 ------
 
 {% if package == "pip" %}
-  pip install -U pip setuptools wheel
+  pip install -U pip setuptools wheel &&
   pip install -U spacy
 
     {% if hardware == "cpu"  %}
@@ -121,10 +121,10 @@ additionalProperties: false
 {%endif%}
 
 {%if package == "source"%}
-  pip install -U pip setuptools wheel
-  git clone https://github.com/explosion/spaCy
-  cd spaCy
-  pip install -r requirements.txt
+  pip install -U pip setuptools wheel &&
+  git clone https://github.com/explosion/spaCy &&
+  cd spaCy &&
+  pip install -r requirements.txt &&
   pip install --no-build-isolation --editable .
 
   {% if hardware == "gpu"%}

installation_instruction/__main__.py

@@ -14,11 +14,13 @@
 
 from sys import exit
 from os.path import isfile
+from subprocess import run
 
 import click
 
 from .get_flags_and_options_from_schema import get_flags_and_options
 from .installation_instruction import InstallationInstruction
+from .helpers import _make_pretty_print_line_breaks
 
 
 class ConfigReadCommand(click.MultiCommand):
@@ -43,14 +45,32 @@ def get_command(self, ctx, config_file: str) -> click.Command|None:
             instruction = InstallationInstruction.from_file(config_file)
             options = get_flags_and_options(instruction.schema)
         except Exception as e:
-            click.echo(str(e))
+            click.echo(click.style("Error (parsing options from schema): " + str(e), fg="red"))
             exit(1)
 
 
         def callback(**kwargs):
             inst = instruction.validate_and_render(kwargs)
-            click.echo(inst[0])
-            exit(0 if not inst[1] else 1)
+            if inst[1]:
+                click.echo(click.style("Error: " + inst[0], fg="red"))
+                exit(1)
+            if ctx.obj["MODE"] == "show":
+                if ctx.obj["RAW"]:
+                    click.echo(inst[0])
+                else:
+                    click.echo(_make_pretty_print_line_breaks(inst[0]))
+            elif ctx.obj["MODE"] == "install":
+                result = run(inst[0], shell=True, text=True, capture_output=True)
+                if result.returncode != 0:
+                    click.echo(click.style("Installation failed with:\n" + str(result.stdout) + "\n" + str(result.stderr), fg="red"))
+                    exit(1)
+                else:
+                    if ctx.obj["INSTALL_VERBOSE"]:
+                        click.echo(str(result.stdout))
+                    click.echo(click.style("Installation successful.", fg="green"))
+
+            exit(0)
+
 
         return click.Command(
             name=config_file,
@@ -60,14 +80,26 @@ def callback(**kwargs):
 
 
 @click.command(cls=ConfigReadCommand, help="Shows installation instructions for your specified config file and parameters.")
-def show():
-    pass
+@click.option("--raw", is_flag=True, help="Show installation instructions without pretty print.", default=False)
+@click.pass_context
+def show(ctx, raw):
+    ctx.obj['MODE'] = "show"
+    ctx.obj['RAW'] = raw
+
+@click.command(cls=ConfigReadCommand, help="Installs with config and parameters given.")
+@click.option("-v", "--verbose", is_flag=True, help="Show verbose output.", default=False)
+@click.pass_context
+def install(ctx, verbose):
+    ctx.obj['MODE'] = "install"
+    ctx.obj['INSTALL_VERBOSE'] = verbose
 
 @click.group()
-def main():
-    pass
+@click.pass_context
+def main(ctx):
+    ctx.ensure_object(dict)
 
 main.add_command(show)
+main.add_command(install)
 
 if __name__ == "__main__":
     main()

installation_instruction/get_flags_and_options_from_schema.py

@@ -35,6 +35,7 @@ def get_flags_and_options(schema: dict) -> list[Option]:
     required_args = set(schema.get('required', []))
 
     for key, value in schema.get('properties', {}).items():
+        orig_key = key
         key = key.replace('_', '-').replace(' ', '-')
         option_name = '--{}'.format(key)
         option_type = value.get('type', 'string')
@@ -48,7 +49,10 @@ def get_flags_and_options(schema: dict) -> list[Option]:
         else:
             option_type = SCHEMA_TO_CLICK_TYPE_MAPPING.get(option_type, click.STRING)
 
-        required = (key in required_args) and not option_default
+        required = (orig_key in required_args) and option_default is None
+        is_flag=(option_type == click.BOOL)
+        if is_flag and required:
+            option_name = option_name + "/--no-{}".format(key)
 
         options.append(Option(
             param_decls=[option_name],
@@ -57,7 +61,8 @@ def get_flags_and_options(schema: dict) -> list[Option]:
             required=required,
             default=option_default,
             show_default=True,
-            show_choices=True
+            show_choices=True,
+            is_flag=is_flag,
         ))
 
     return options

installation_instruction/helpers.py

@@ -16,6 +16,17 @@
 from jinja2 import Environment, Template
 
 
+def _make_pretty_print_line_breaks(string: str) -> str:
+    """
+    Replaces `&& ` with a newline character.
+
+    :param string: String to be processed.
+    :type string: str
+    :return: String with `&& ` replaced with newline character.
+    :rtype: str
+    """
+    return re.sub(r"\s?&&\s?", "\n", string, 0, re.S)
+
 def _get_error_message_from_string(string: str) -> str | None:
     """
     Parses error message of error given by using jinja macro `RAISE_JINJA_MACRO_STRING`. If no error message is found returns `None`.

tests/data/flags_options_example.schema.yml

@@ -36,9 +36,17 @@ properties:
       - title: CUDA 12.1
         const: cu121
     default: cu118
+
+  verbose:
+    type: boolean
+    default: false
+
+  requiered_flag:
+    type: boolean
 
 required:
   - os
   - packager
+  - requiered_flag
 
 additionalProperties: false

tests/data/test_install/install.cfg

@@ -0,0 +1,22 @@
+$schema: https://json-schema.org/draft/2020-12/schema
+$id: https://github.com/instructions-d-installation/installation-instruction/tests/data/test_install/install.cfg
+name: test-install
+type: object
+properties:
+  error_install:
+    type: boolean
+  error_template:
+    type: boolean
+    default: false
+required:
+  - error_install
+  - error_template
+------
+echo "start" &&
+{%- if error_install %}
+    abcdefghijklmnop
+{%- endif %}
+{%- if error_template %}
+    {{ raise("error error error") }}
+{%- endif %}
+echo "end"

tests/test_get_flags_and_options_from_schema.py

@@ -19,7 +19,7 @@ def test_get_flags_and_options(test_data_flags_options):
     example_schema = test_data_flags_options
     options = get_flags_and_options(example_schema)
 
-    assert len(options) == 4
+    assert len(options) == 6
 
     assert options[0].opts == ["--os"]
     assert options[0].help == "The operating system in which the package is installed."
@@ -40,3 +40,11 @@ def test_get_flags_and_options(test_data_flags_options):
     assert options[3].help == "Should your gpu or your cpu handle the task?"
     assert options[3].required == False
     assert options[3].default == "cu118"
+
+    assert options[4].opts == ["--verbose"]
+    assert options[4].required == False
+    assert options[4].default == False
+
+    assert options[5].opts == ["--requiered-flag"]
+    assert options[5].required == True
+    assert options[5].default == None

tests/test_installation_instruction.py

@@ -49,7 +49,7 @@ def test_validate_and_render_spacy():
     install = InstallationInstruction.from_file("examples/spacy/spacy-instruction.schema.yml.jinja")
 
     good_installation_instruction = install.validate_and_render(valid_user_input)
-    assert ('pip install -U pip setuptools wheel pip install -U spacy', False) == good_installation_instruction
+    assert ('pip install -U pip setuptools wheel && pip install -U spacy', False) == good_installation_instruction
 
     with pytest.raises(Exception):
         install.validate_and_render(invalid_user_input)