Ported customize/filters/tests functionality from j2cli

README.md

@@ -133,7 +133,12 @@ Password: {{ "APP_PASSWORD" | env }}
 * `data`: (optional) path to the data used for rendering.
     The default is `-`: use stdin.
 
-Options:
+There is some special behavior with environment variables:
+
+* When `data` is not provided (data is `-`), `--format` defaults to
+  `env` and thus reads environment variables.
+
+### Options:
 
 * `--format FMT, -f FMT`: format for the data file. The default is
   `?`: guess from file extension. Supported formats are YAML (.yaml or
@@ -155,10 +160,33 @@ Options:
   error will be raised).
 * `--version`: prints the version of the tool and the Jinja2 package installed.
 
-There is some special behavior with environment variables:
+### Customization Options:
 
-* When `data` is not provided (data is `-`), `--format` defaults to
-  `env` and thus reads environment variables.
+For details on the behavior of these options, see the
+[Customization](#customization) section.
+
+* `--filters PYTHON_FILE` - specify a file of Python source code,
+  containing additional Jinja2 filters as simple functions. You can
+  use this option more than once to include multiple files.
+
+  * NOTE: While this option's behavior matches the `j2cli`
+    documentation, it does not match the `j2cli` implementation. If
+    you are migrating from `j2cli` and use more than one filters file,
+    you will need to specify this option once for each file.
+
+* `--tests PYTHON_FILE` - specify a file of Python source code,
+  containing additional Jinja2 tests as simple functions. You can use
+  this option more than once to include multiple files.
+
+  * NOTE: While this option's behavior matches the `j2cli`
+    documentation, it does not match the `j2cli` implementation. If
+    you are migrating from `j2cli` and use more than one tests file,
+    you will need to specify this option once for each file.
+
+* `--customize PYTHON_FILE` - specify a file of Python source code
+  containing customization functions. This file can modify the Jinja2
+  context, add filters/tests, or change Jinja2's configuration. Unlike
+  `--filters` and `--tests`, this option can only be specified once.
 
 ## Usage Examples
 
@@ -350,8 +378,175 @@ Pass: {{ env("USER_PASSWORD") }}
 
 Notice that there must be quotes around the environment variable name
 when it is a literal string.
-<!-- fancy-readme end -->
 
+## Customization
+
+Jinjanator supports customizing Jinja2 template processing using two
+methods - via simple files containing custom filters or tests, or via
+a more advanced "customizations" file that allows you to do all of the
+above as well as modify core configuration of the Jinja2 engine.
+
+### Using filters and tests files
+
+The simplest way to add additional filters or tests is via "filters"
+and "tests" files. These files contain Python source code consisting
+of simple functions. Each function becomes a filter or test.
+
+Examples:
+
+`filters.py`
+
+```python
+# Simple filters file
+
+def parentheses(message):
+    """ Put message in parenthesis """
+    return f"({message})"
+```
+
+`tests.py`
+
+```python
+#  Example of simple tests file
+
+def an_odd_number(number):
+    """ test if number is odd """
+    return True if (number % 2) else False
+```
+
+And a template that uses them:
+
+```
+{% for x in range(4) %}
+{{x}} is: {% if x is an_odd_number %}
+    {{- "odd" | parentheses }}
+  {%- else %}
+    {{- "even" | parentheses }}
+  {%- endif %}
+{%- endfor %}
+```
+
+The output is:
+
+```
+$ jinjanate --filter ./filters.py --test ./tests.py simple.j2
+
+0 is: (even)
+1 is: (odd)
+2 is: (even)
+3 is: (odd)
+```
+
+You can include multiple functions in each file and/or use multiple
+files as needed.
+
+### Using a customizations file
+
+A more advanced way to customize your template processing is by using
+a "customizations" file.
+
+Customizations files allow you to:
+
+* Pass additional keywords to the Jinja2 environment
+* Modify the context before it is used for rendering
+* Register custom filters and tests
+
+This is done through *hooks* that you implement in a customization
+file in Python code. Each hook is a plain function at the module
+level with the exact name as shown below.
+
+The following hooks are available:
+
+* `j2_environment_params() -> dict`: returns a `dict` of additional parameters for
+  [Jinja2 Environment](https://jinja.pocoo.org/docs/2.10/api/#jinja2.Environment).
+
+* `j2_environment(env: Environment) -> Environment`: lets you
+  customize the `Environment` object.
+
+* `alter_context(context: dict) -> dict`: lets you modify the context
+  variables that are going to be used for template rendering. You can
+  do all sorts of pre-processing here.
+
+* `extra_filters() -> dict`: returns a `dict` with extra filters for
+  Jinja2
+
+* `extra_tests() -> dict`: returns a `dict` with extra tests for
+  Jinja2
+
+All of them are optional.
+
+The example `customization.py file` for your reference:
+
+```python
+#
+# Example customization.py file for jinjanator
+# Contains hooks that modify the way Jinja2 is initialized and used
+
+def j2_environment_params():
+    """ Extra parameters for the Jinja2 Environment """
+    # Jinja2 Environment configuration
+    # https://jinja.pocoo.org/docs/2.10/api/#jinja2.Environment
+    return dict(
+        # Just some examples
+
+        # Change block start/end strings
+        block_start_string='<%',
+        block_end_string='%>',
+        # Change variable strings
+        variable_start_string='<<',
+        variable_end_string='>>',
+        # Remove whitespace around blocks
+        trim_blocks=True,
+        lstrip_blocks=True,
+        # Enable line statements:
+        # http://jinja.pocoo.org/docs/2.10/templates/#line-statements
+        line_statement_prefix='#',
+        # Keep \n at the end of a file
+        keep_trailing_newline=True,
+        # Enable custom extensions
+        # http://jinja.pocoo.org/docs/2.10/extensions/#jinja-extensions
+        extensions=('jinja2.ext.i18n',),
+    )
+
+def j2_environment(env):
+    """ Modify Jinja2 environment
+
+    :param env: jinja2.environment.Environment
+    :rtype: jinja2.environment.Environment
+    """
+    env.globals.update(
+        my_function=lambda v: 'my function says "{}"'.format(v)
+    )
+    return env
+
+def alter_context(context):
+    """ Modify the context and return it """
+    # An extra variable
+    context['ADD'] = '127'
+    return context
+
+def extra_filters():
+    """ Declare some custom filters.
+
+        Returns: dict(name = function)
+    """
+    return dict(
+        # Example: {{ var | parentheses }}
+        parentheses=lambda t: '(' + t + ')',
+    )
+
+def extra_tests():
+    """ Declare some custom tests
+
+        Returns: dict(name = function)
+    """
+    return dict(
+        # Example: {% if a|int is custom_odd %}odd{% endif %}
+        custom_odd=lambda n: True if (n % 2) else False
+    )
+```
+
+<!-- fancy-readme end -->
 ## Chat
 
 If you'd like to chat with the jinjanator community, join us on

changelog.d/46.adding.md

@@ -0,0 +1,2 @@
+Added support for '--filters', '--tests' and '--customize' from j2cli
+(contributed by @mlasevich).

src/jinjanator/cli.py

@@ -18,8 +18,9 @@
 import jinjanator_plugins
 import pluggy
 
-from . import filters, formats, version
+from . import customize, filters, formats, version
 from .context import read_context_data
+from .customize import CustomizationModule
 
 
 class FilePathLoader(jinja2.BaseLoader):
@@ -71,23 +72,23 @@ def __init__(
         j2_env_params.setdefault("extensions", self.ENABLED_EXTENSIONS)
         j2_env_params.setdefault("loader", FilePathLoader(cwd))
 
-        self._env = jinja2.Environment(**j2_env_params, autoescape=False)  # noqa: S701
+        self.env = jinja2.Environment(**j2_env_params, autoescape=False)  # noqa: S701
 
         for plugin_globals in plugin_hook_callers.plugin_globals():
-            self._env.globals |= plugin_globals
+            self.env.globals |= plugin_globals
 
         for plugin_filters in plugin_hook_callers.plugin_filters():
-            self._env.filters |= plugin_filters
+            self.env.filters |= plugin_filters
 
         for plugin_tests in plugin_hook_callers.plugin_tests():
-            self._env.tests |= plugin_tests
+            self.env.tests |= plugin_tests
 
         for plugin_extensions in plugin_hook_callers.plugin_extensions():
             for extension in plugin_extensions:
-                self._env.add_extension(extension)
+                self.env.add_extension(extension)
 
     def render(self, template_name: str, context: Mapping[str, str]) -> str:
-        return self._env.get_template(template_name).render(context)
+        return self.env.get_template(template_name).render(context)
 
 
 class UniqueStore(argparse.Action):
@@ -220,6 +221,9 @@ def parse_args(
         help="Suppress informational messages",
     )
 
+    # add args for customize support
+    customize.add_args(parser)
+
     parser.add_argument(
         "-o",
         "--output-file",
@@ -304,7 +308,7 @@ def render_command(
 
     # We always expect a file;
     # unless the user wants 'env', and there's no input file provided.
-    if args.format == "env":
+    if args.format == "env" and args.data is None:
         """
         With the "env" format, if no dotenv filename is provided,
         we have two options: 1. The user wants to use the current
@@ -321,12 +325,7 @@ def render_command(
         And this is what we're going to do here as well. The script,
         however, would give the user a hint that they should use '-'.
         """
-        if str(args.data) == "-":
-            input_data_f = stdin
-        elif args.data is None:
-            input_data_f = None
-        else:
-            input_data_f = args.data.open()
+        input_data_f = None
     else:
         input_data_f = stdin if args.data is None or str(args.data) == "-" else args.data.open()
 
@@ -342,13 +341,19 @@ def render_command(
             args.import_env,
         )
 
+    customizations = CustomizationModule.from_file(args.customize)
+
+    context = customizations.alter_context(context)
+
     renderer = Jinja2TemplateRenderer(
         cwd,
         args.undefined,
-        j2_env_params={},
+        j2_env_params=customizations.j2_environment_params(),
         plugin_hook_callers=plugin_hook_callers,
     )
 
+    customize.apply(customizations, renderer.env, filters=args.filters, tests=args.tests)
+
     try:
         result = renderer.render(args.template, context)
     except jinja2.exceptions.UndefinedError as e: