Skip to content

Commit

Permalink
From refs/heads/dev be1de5b
Browse files Browse the repository at this point in the history
  • Loading branch information
pedohorse committed Nov 16, 2023
1 parent 3b5c9b0 commit 2abb85b
Show file tree
Hide file tree
Showing 43 changed files with 10,050 additions and 24 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
GitHub Pages of [pedohorse/lifeblood](https://github.com/pedohorse/lifeblood.git)
===
Sphinx html documentation of [3501aea5](https://github.com/pedohorse/lifeblood/tree/3501aea55a5e18e74b4afac2ceffa9b1f0e370dd)
Sphinx html documentation of [be1de5ba](https://github.com/pedohorse/lifeblood/tree/be1de5baff832a0522e236f3b9509544034dccab)
12 changes: 12 additions & 0 deletions _sources/concepts.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ Addressing
Address Chain
-------------

.. _environment_resolver:

Environment Resolver
--------------------

Task Processing
===============

Expand All @@ -35,9 +40,16 @@ Task Processing
Processing stage
----------------

.. _dead_tasks:

Dead Tasks
----------

Task Relations
==============

.. _task_parent_hierarchy:

Task Parent Hierarchy
---------------------

Expand Down
16 changes: 16 additions & 0 deletions _sources/nodes.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,26 @@ Nodes
:maxdepth: 1
:caption: Core Nodes:

nodes/core/attribute_splitter
nodes/core/delete_attrib
nodes/core/environment_resolver_arguments_setter
nodes/core/killer
nodes/core/mod_attrib
nodes/core/null
nodes/core/parent_children_waiter
nodes/core/python
nodes/core/rename_attrib
nodes/core/set_attrib
nodes/core/spawn_children
nodes/core/split_waiter
nodes/core/switch
nodes/core/wait_for_task_value
nodes/core/wedge

.. toctree::
:maxdepth: 1
:caption: Stock Nodes:

nodes/stock/houdini/karma
nodes/stock/houdini/mantra
nodes/stock/matrixnotifier
91 changes: 91 additions & 0 deletions _sources/nodes/core/attribute_splitter.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
.. _nodes/core/attribute_splitter:

==================
Attribute Splitter
==================

Splits task's attribute value into chunks of "chunk size" elements.

All created tasks are **splits** of the original task, so they need to be split-awaited later

:ref:`Split concept<concept_split>` is one of the core Lifeblood's task concepts.

It **List** more works ONLY on array/list attributes.

.. note::
that task wil ALWAYS be splitted, even if just in 1 piece

For ex. attribute "something" of value ``[3, 4, 5, 6, 7, 8, 9, 10 , 11, 12, 13]`` with chunk size of 4
will split task into 3 tasks with attributes "something" equals to:

* ``[3, 4, 5, 6]``
* ``[7, 8, 9, 10]``
* ``[11, 12, 13]``

respectively

Parameters
==========

Mode
Split mode:

* list
Will take the value of given attribute and if it is a list - split it into chunks of given size
and create a task for each chunk. Each new task will have the value of the given attribute to be set
to one of the chunks.

So splitting a task with ``attr`` that has value ``[1, 2, 3, 4, "five", 6.6]`` with chunk size 3 will result
in a split into 3 tasks with the value of ``attr`` being ``[1, 2]``, ``[3, 4]`` and ``["five", 6.6]`` respectively.
* range
Given float or int range start, end and increment will split this range into either given number of chunks, or some number of chunks
with given number of elements

For example, splitting range with start=15, end=29, increment=3 and number of chunks 4
will result in incoming task being split into 4 tasks with values of given attributes
``out start name``, ``out end name`` or ``out size name`` being set to
``(15, 21, 2)``, ``(21, 24, 1)``, ``(24, 27, 1)`` and ``(27, 29, 1)`` respectively
Attribute To Split
In List mode - the name of the attribute to split and set on split tasks
Chunk Size
Size of the chunks to split list/range into
Range Mode
For range mode - type of range to output to splitted tasks

* **start-end** - set starting and ending elements
* **start-size** - set starting element and the number of elements
* **start-end-size** - set starting element and both end element and the number of elements
Start
For range mode - start of the range to be split
End
For range mode - end of the range to be split
Inc
For range mode - increment of the range to be split
Split By
How to split given list/range

* chunk size
Split by the desired number of elements in a single chunk

* number of chunks
Split by desired number of chunks
Chunk Size
Desired number of elements in each chunk
Chunks Count
Desired number of chunks
Type
For range mode - type of output range
Output Range Start Attribute Name
For range mode - name of the attribute to set on split tasks that holds the starting number of the range
Output Range End Attribute Name
For range mode - name of the attribute to set on split tasks that holds the ending number of the range
Output Range Size Attribute Name
For range mode - name of the attribute to set on split tasks that holds the number of elements in the range


Attributes Set
==============

In List mode attribute named as the value of ``Attribute To Split`` parameter is set

In Range mode - attributes named as the values of ``Output Range Start Attribute Name``, ``Output Range End Attribute Name`` and ``Output Range Size Attribute Name`` are set
22 changes: 22 additions & 0 deletions _sources/nodes/core/delete_attrib.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.. _nodes/core/delete_attrib:

=================
Delete Attributes
=================

Deletes attributes from the Task

Parameters
==========

delete
Name of the attribute to delete

This does not currently support patterns, only exact names

If attribute does not exist - nothing is be done

Attributes Set
==============

Given attributes are removed
31 changes: 31 additions & 0 deletions _sources/nodes/core/environment_resolver_arguments_setter.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
.. _nodes/core/environment_resolver_arguments_setter:

=====================================
Modify Environment Resolver Arguments
=====================================

Modifies Environment Resolver information of the task.

:ref:`Environment Resolver<environment_resolver>` is one of core concepts related to invocation running.

Parameters
==========

Mode
* Set - sets/resets environment resolver information
* Modify - modifies existing environment resolver information
Resolver
Environment Resolver type name to set
Arguments To Set
Environment Resolver arguments to set.

For default Lifeblood's environment resolvers, here you most likely want to set such arguments as
package names in form of ``package.<package_name>`` to value that can be iterpreted as version specification by Semantic Versioning standard,
for example ``~=1.2.3``
Arguments To Delete
In Modify mode - list of environment resolver arguments to delete

Attributes Set
==============

None
23 changes: 23 additions & 0 deletions _sources/nodes/core/killer.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
.. _nodes/core/killer:

======
Killer
======

Kills incoming tasks

:ref:`"Dead" tasks<dead_tasks>` - are a special state of tasks that are excluded from any processing by Lifeblood, yet kept for keeping all invocaions and other references valid.

Dead tasks can also be easily filtered in the viewer

This should be the logical end point of any task processing

Parameters
==========

None

Attributes Set
==============

None
18 changes: 18 additions & 0 deletions _sources/nodes/core/mod_attrib.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.. _nodes/core/mod_attrib:

=================
Modify Attributes
=================

Modify incoming task's attributes

Parameters
==========

Name
Name of the attribute to modify

Attributes Set
==============

Attributes of given names will be modified
14 changes: 13 additions & 1 deletion _sources/nodes/core/null.rst.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
.. _nodes/core/null:

====
null
Null
====

Does nothing.

Parameters
==========

None

Attributes Set
==============

None
43 changes: 43 additions & 0 deletions _sources/nodes/core/parent_children_waiter.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.. _nodes/core/parent_children_waiter:

======================
Parent-Children Waiter
======================

Core task synchronization node.

Tasks' :ref:`parent-children hierarchy<task_parent_hierarchy>` is a core concept of Lifeblood.

Tasks coming from input 1 are treated as parents, tasks coming from input 2 are treated as children.

In standard mode, a parent task is not let through until all of it's **active** children have arrived in the same node.
In the same way children tasks are not let through until their parent and **all siblings** have arrived.

Again: a parent and all it's active children will be waiting in this node until all of them arrive.

After every required task has arrived - they may exchange attributes.

When tasks are let through - they leave through the output that corresponds to the input they came through

Parameters
==========

Recursive
if this is set - tasks coming through second input are treated as both parents and children, therefore tasks coming from input 1 will be waiting
for all their children, grandchildren and so on.

Transfer Attributes

attribute
name of an attribute to promote from children to parent
as
set promoted value to attribute of this name on parent
sort by
how to sort children task before gathering values for promotion
reversed
reverse the sort

Attributes Set
==============

Promoted attributes are set to the parent task
26 changes: 26 additions & 0 deletions _sources/nodes/core/python.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
.. _nodes/core/python:

=================
Modify Attributes
=================

Execute some python code

Parameters
==========

Process
Python code to be executed by scheduler as part of processing stage. Do not perform any heavy operations here,
instead perform them in the **invoke** code.

You can call special function ``schedule()`` here, in that case code from **Invoke** parameter will be scheduled to be executed on a worker.
Invoke
Python code to be scheduled and executed on a worker.

Use ``task`` variable to get task attributes here, e.g. ``task['attr'] = 123`` will set attribute ``attr`` on the task to value 123.
While this code will be executed on remote worker, all calls to task attribute getting methods, like ``task['smth']`` will be resolved beforehand.

Attributes Set
==============

Attributes set by code with something like ``task['attr_name'] = "something"`` will be set to the task
22 changes: 22 additions & 0 deletions _sources/nodes/core/rename_attrib.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.. _nodes/core/rename_attrib:

=================
Rename Attributes
=================

Rename task attributes

Parameters
==========

from
rename attribute from this name
to
to this name

If attribute named **from** is not found - nothing will be done

Attributes Set
==============

Attributes renamed, nothing is set
20 changes: 20 additions & 0 deletions _sources/nodes/core/set_attrib.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
.. _nodes/core/set_attrib:

==============
Set Attributes
==============

Set attributes on task

Parameters
==========

Name
Name of the attribute to set
Val
Value to set for the attribute

Attributes Set
==============

Specified attributes will be created
Loading

0 comments on commit 2abb85b

Please sign in to comment.