-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
43 changed files
with
10,050 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
GitHub Pages of [pedohorse/lifeblood](https://github.com/pedohorse/lifeblood.git) | ||
=== | ||
Sphinx html documentation of [3501aea5](https://github.com/pedohorse/lifeblood/tree/3501aea55a5e18e74b4afac2ceffa9b1f0e370dd) | ||
Sphinx html documentation of [be1de5ba](https://github.com/pedohorse/lifeblood/tree/be1de5baff832a0522e236f3b9509544034dccab) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
.. _nodes/core/attribute_splitter: | ||
|
||
================== | ||
Attribute Splitter | ||
================== | ||
|
||
Splits task's attribute value into chunks of "chunk size" elements. | ||
|
||
All created tasks are **splits** of the original task, so they need to be split-awaited later | ||
|
||
:ref:`Split concept<concept_split>` is one of the core Lifeblood's task concepts. | ||
|
||
It **List** more works ONLY on array/list attributes. | ||
|
||
.. note:: | ||
that task wil ALWAYS be splitted, even if just in 1 piece | ||
|
||
For ex. attribute "something" of value ``[3, 4, 5, 6, 7, 8, 9, 10 , 11, 12, 13]`` with chunk size of 4 | ||
will split task into 3 tasks with attributes "something" equals to: | ||
|
||
* ``[3, 4, 5, 6]`` | ||
* ``[7, 8, 9, 10]`` | ||
* ``[11, 12, 13]`` | ||
|
||
respectively | ||
|
||
Parameters | ||
========== | ||
|
||
Mode | ||
Split mode: | ||
|
||
* list | ||
Will take the value of given attribute and if it is a list - split it into chunks of given size | ||
and create a task for each chunk. Each new task will have the value of the given attribute to be set | ||
to one of the chunks. | ||
|
||
So splitting a task with ``attr`` that has value ``[1, 2, 3, 4, "five", 6.6]`` with chunk size 3 will result | ||
in a split into 3 tasks with the value of ``attr`` being ``[1, 2]``, ``[3, 4]`` and ``["five", 6.6]`` respectively. | ||
* range | ||
Given float or int range start, end and increment will split this range into either given number of chunks, or some number of chunks | ||
with given number of elements | ||
|
||
For example, splitting range with start=15, end=29, increment=3 and number of chunks 4 | ||
will result in incoming task being split into 4 tasks with values of given attributes | ||
``out start name``, ``out end name`` or ``out size name`` being set to | ||
``(15, 21, 2)``, ``(21, 24, 1)``, ``(24, 27, 1)`` and ``(27, 29, 1)`` respectively | ||
Attribute To Split | ||
In List mode - the name of the attribute to split and set on split tasks | ||
Chunk Size | ||
Size of the chunks to split list/range into | ||
Range Mode | ||
For range mode - type of range to output to splitted tasks | ||
|
||
* **start-end** - set starting and ending elements | ||
* **start-size** - set starting element and the number of elements | ||
* **start-end-size** - set starting element and both end element and the number of elements | ||
Start | ||
For range mode - start of the range to be split | ||
End | ||
For range mode - end of the range to be split | ||
Inc | ||
For range mode - increment of the range to be split | ||
Split By | ||
How to split given list/range | ||
|
||
* chunk size | ||
Split by the desired number of elements in a single chunk | ||
|
||
* number of chunks | ||
Split by desired number of chunks | ||
Chunk Size | ||
Desired number of elements in each chunk | ||
Chunks Count | ||
Desired number of chunks | ||
Type | ||
For range mode - type of output range | ||
Output Range Start Attribute Name | ||
For range mode - name of the attribute to set on split tasks that holds the starting number of the range | ||
Output Range End Attribute Name | ||
For range mode - name of the attribute to set on split tasks that holds the ending number of the range | ||
Output Range Size Attribute Name | ||
For range mode - name of the attribute to set on split tasks that holds the number of elements in the range | ||
|
||
|
||
Attributes Set | ||
============== | ||
|
||
In List mode attribute named as the value of ``Attribute To Split`` parameter is set | ||
|
||
In Range mode - attributes named as the values of ``Output Range Start Attribute Name``, ``Output Range End Attribute Name`` and ``Output Range Size Attribute Name`` are set |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
.. _nodes/core/delete_attrib: | ||
|
||
================= | ||
Delete Attributes | ||
================= | ||
|
||
Deletes attributes from the Task | ||
|
||
Parameters | ||
========== | ||
|
||
delete | ||
Name of the attribute to delete | ||
|
||
This does not currently support patterns, only exact names | ||
|
||
If attribute does not exist - nothing is be done | ||
|
||
Attributes Set | ||
============== | ||
|
||
Given attributes are removed |
31 changes: 31 additions & 0 deletions
31
_sources/nodes/core/environment_resolver_arguments_setter.rst.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
.. _nodes/core/environment_resolver_arguments_setter: | ||
|
||
===================================== | ||
Modify Environment Resolver Arguments | ||
===================================== | ||
|
||
Modifies Environment Resolver information of the task. | ||
|
||
:ref:`Environment Resolver<environment_resolver>` is one of core concepts related to invocation running. | ||
|
||
Parameters | ||
========== | ||
|
||
Mode | ||
* Set - sets/resets environment resolver information | ||
* Modify - modifies existing environment resolver information | ||
Resolver | ||
Environment Resolver type name to set | ||
Arguments To Set | ||
Environment Resolver arguments to set. | ||
|
||
For default Lifeblood's environment resolvers, here you most likely want to set such arguments as | ||
package names in form of ``package.<package_name>`` to value that can be iterpreted as version specification by Semantic Versioning standard, | ||
for example ``~=1.2.3`` | ||
Arguments To Delete | ||
In Modify mode - list of environment resolver arguments to delete | ||
|
||
Attributes Set | ||
============== | ||
|
||
None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
.. _nodes/core/killer: | ||
|
||
====== | ||
Killer | ||
====== | ||
|
||
Kills incoming tasks | ||
|
||
:ref:`"Dead" tasks<dead_tasks>` - are a special state of tasks that are excluded from any processing by Lifeblood, yet kept for keeping all invocaions and other references valid. | ||
|
||
Dead tasks can also be easily filtered in the viewer | ||
|
||
This should be the logical end point of any task processing | ||
|
||
Parameters | ||
========== | ||
|
||
None | ||
|
||
Attributes Set | ||
============== | ||
|
||
None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
.. _nodes/core/mod_attrib: | ||
|
||
================= | ||
Modify Attributes | ||
================= | ||
|
||
Modify incoming task's attributes | ||
|
||
Parameters | ||
========== | ||
|
||
Name | ||
Name of the attribute to modify | ||
|
||
Attributes Set | ||
============== | ||
|
||
Attributes of given names will be modified |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,17 @@ | ||
.. _nodes/core/null: | ||
|
||
==== | ||
null | ||
Null | ||
==== | ||
|
||
Does nothing. | ||
|
||
Parameters | ||
========== | ||
|
||
None | ||
|
||
Attributes Set | ||
============== | ||
|
||
None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
.. _nodes/core/parent_children_waiter: | ||
|
||
====================== | ||
Parent-Children Waiter | ||
====================== | ||
|
||
Core task synchronization node. | ||
|
||
Tasks' :ref:`parent-children hierarchy<task_parent_hierarchy>` is a core concept of Lifeblood. | ||
|
||
Tasks coming from input 1 are treated as parents, tasks coming from input 2 are treated as children. | ||
|
||
In standard mode, a parent task is not let through until all of it's **active** children have arrived in the same node. | ||
In the same way children tasks are not let through until their parent and **all siblings** have arrived. | ||
|
||
Again: a parent and all it's active children will be waiting in this node until all of them arrive. | ||
|
||
After every required task has arrived - they may exchange attributes. | ||
|
||
When tasks are let through - they leave through the output that corresponds to the input they came through | ||
|
||
Parameters | ||
========== | ||
|
||
Recursive | ||
if this is set - tasks coming through second input are treated as both parents and children, therefore tasks coming from input 1 will be waiting | ||
for all their children, grandchildren and so on. | ||
|
||
Transfer Attributes | ||
|
||
attribute | ||
name of an attribute to promote from children to parent | ||
as | ||
set promoted value to attribute of this name on parent | ||
sort by | ||
how to sort children task before gathering values for promotion | ||
reversed | ||
reverse the sort | ||
|
||
Attributes Set | ||
============== | ||
|
||
Promoted attributes are set to the parent task |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
.. _nodes/core/python: | ||
|
||
================= | ||
Modify Attributes | ||
================= | ||
|
||
Execute some python code | ||
|
||
Parameters | ||
========== | ||
|
||
Process | ||
Python code to be executed by scheduler as part of processing stage. Do not perform any heavy operations here, | ||
instead perform them in the **invoke** code. | ||
|
||
You can call special function ``schedule()`` here, in that case code from **Invoke** parameter will be scheduled to be executed on a worker. | ||
Invoke | ||
Python code to be scheduled and executed on a worker. | ||
|
||
Use ``task`` variable to get task attributes here, e.g. ``task['attr'] = 123`` will set attribute ``attr`` on the task to value 123. | ||
While this code will be executed on remote worker, all calls to task attribute getting methods, like ``task['smth']`` will be resolved beforehand. | ||
|
||
Attributes Set | ||
============== | ||
|
||
Attributes set by code with something like ``task['attr_name'] = "something"`` will be set to the task |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
.. _nodes/core/rename_attrib: | ||
|
||
================= | ||
Rename Attributes | ||
================= | ||
|
||
Rename task attributes | ||
|
||
Parameters | ||
========== | ||
|
||
from | ||
rename attribute from this name | ||
to | ||
to this name | ||
|
||
If attribute named **from** is not found - nothing will be done | ||
|
||
Attributes Set | ||
============== | ||
|
||
Attributes renamed, nothing is set |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
.. _nodes/core/set_attrib: | ||
|
||
============== | ||
Set Attributes | ||
============== | ||
|
||
Set attributes on task | ||
|
||
Parameters | ||
========== | ||
|
||
Name | ||
Name of the attribute to set | ||
Val | ||
Value to set for the attribute | ||
|
||
Attributes Set | ||
============== | ||
|
||
Specified attributes will be created |
Oops, something went wrong.