Skip to content

Commit

Permalink
Added documentation.
Browse files Browse the repository at this point in the history
Partially complete documentation added.
  • Loading branch information
jkrukoff committed Jan 28, 2019
1 parent 1a2053e commit 9194289
Show file tree
Hide file tree
Showing 4 changed files with 336 additions and 19 deletions.
177 changes: 177 additions & 0 deletions doc/overview.edoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,181 @@
@version 1.0.0
@title pipe_line
@doc <h3>Overview</h3>

<img src="pipeline.jpg" alt="Pipeline Construction"/>

This is an Erlang/OTP library for piping a value through a list of functions,
in the spirit of <a
href="https://en.wikibooks.org/wiki/Haskell/do_notation">Haskell's do
notation</a>. It is distinguished from Elixir's pipe operator "|>" by allowing
for programmatic control over whether or not each stage of the pipeline is
executed.

For example, we can execute the following series of computations via a call to
one of the pipe functions:

```
> pipe:line(lists:seq(1, 100000),
[fun (Xs) -> [X * 3 || X <- Xs] end,
fun (Xs) -> [X || X <- Xs, X rem 2 == 1] end,
fun lists:sum/1]).
7500000000
'''

<h3>Getting Started</h3>

This library is published to <a href="https://hex.pm">hex.pm</a> as <a
href="https://hex.pm/packages/pipe_line">pipe_line</a>. If you're using <a
href="https://www.rebar3.org/">rebar3</a> as your build tool, it can be added
as a dependency to your rebar.config as follows:

```
{deps, [{pipe_line}]}.
'''

Additionally, you may wish to also include the <a
href="https://github.com/jkrukoff/partial">partial</a> parse transform in your
application as it cooperates well with this library.

<h3>Usage</h3>

This library introduces two concepts: pipelines which execute a series of
single argument functions, and application functions which can control how
those functions are executed. These are combined to create ready made
pipelines which propagate errors.

<h4>Application Functions</h4>

Application functions are functions which take a function and a value and
decide if and how to execute the function based on the value. A set of related
error handling application functions are provided as pipe:if_ok/2,
pipe:if_not_error/2, pipe:if_not_throw/2 and pipe:if_not_exception/2. These
check the value for various error forms and if found, stop executing and
instead pass the value along unchanged.

An additional class of application functions is included for adapting
functions that would not normally fit into this model. These include:
pipe:ignore/2 for calling functions that work by side effect and pipe:apply/2
for calling functions that take more than a single argument.

<h4>Pipelines</h4>

Pipeline functions use the application functions to decide how to execute a
list of functions. Pre-composed versions are available for the error handling
variants.

The base pipeline function is pipe:pipe/3, on which all others are based.

<h4>Examples</h4>

We'll demonstrate how these functions are used by example. First, let's look
at how a sequence of transformations is represented. In order to split a set
of comma separated words and sort the result the sequence could look like
this:

```
> pipe:line("Sphinx, Of, Black, Quartz, Judge, My, Vow",
[fun(Str) -> string:lowercase(Str) end,
fun(Str) -> string:split(Str, ",", all) end,
fun(Words) -> [string:strip(W, both) || W <- Words] end,
fun(Words) -> lists:sort(Words) end]).
["black","judge","my","of","quartz","sphinx","vow"]
'''

Importantly, we can see here how each stage of the pipeline is represented by
a single argument function. The pipe:line/2 function simply passes the result
of each function along to the next stage.

Now, what about if our functions can return errors? Let's convert and display
the bytes of a UTF-8 encoded string:

```
> Message = [90,831,842,796,807,97,775,855,858,857,108,836,844,846,814,
103,844,842,815,793,111,777,835,841,796],
> pipe:not_error(Message,
[fun (Str) -> unicode:characters_to_binary(Str) end,
fun (Bin) -> io_lib:format("~tw~n", [Bin]) end,
fun (Formatted) -> lists:flatten(Formatted) end]).
"<<90,204,191,205,138,204,156,204,167,97,204,135,205,151,205,154,205,153,108,
205,132,205,140,205,142,204,174,103,205,140,205,138,204,175,204,153,111,204,
137,205,131,205,137,204,156>>\n"
'''

As no errors were encountered the string was converted as expected. What about
if we pass an invalid unicode string to the same pipe?

```
> Message = [90, 55359],
> pipe:not_error(Message,
[fun (Str) -> unicode:characters_to_binary(Str) end,
fun (Bin) -> io_lib:format("~tw~n", [Bin]) end,
fun (Formatted) -> lists:flatten(Formatted) end]).
{error,<<"Z">>,[55359]}
'''

Only the first function runs. The error from that function is then propagated
out as the result of the entire pipeline.

<h3>Partial Function Application</h3>

The <a href="https://github.com/jkrukoff/partial">partial</a> library provides
a parse transform that can make use of this library more convenient. For
example with the partial library the first pipe:line/2 example could instead
be written as:

```
> pipe:line("Sphinx, Of, Black, Quartz, Judge, My, Vow",
[partial:cut(string:lowercase(_)),
partial:cut(string:split(_, ",", all)),
partial:cut(lists:map(partial:cut(string:strip(_, both)), _)),
partial:cut(lists:sort(_))]).
["black","judge","my","of","quartz","sphinx","vow"]
'''

Allowing to more directly see how arguments flow between pipeline stages.

<h3>Contributing</h3>

Please fork the repo and submit a PR. Tests are run via:

```
rebar3 eunit
'''

Documentation is autogenerated using edown and edoc via:

```
rebar3 as markdown edoc
'''

The application has only been tested with Erlang/OTP 21 on Windows 10. Reports
of success (or failure!) on other versions and operating systems are
appreciated.

<h3>Lineage</h3>

With code that has to handle a series of errors, the obvious answer of nested
case statements can quickly become quite unwieldy. Many people have already
adapted the ideas of <a
href="https://en.wikibooks.org/wiki/Haskell/do_notation">Haskell s do
notation</a> and <a href="https://fsharpforfunandprofit.com/rop/">railway
oriented programming</a> to Erlang/OTP as a way to handle errors across a
sequence of operations. I decided to attempt the same as a standalone library
with no syntactic changes and instead implement partial function application
via parse transform as a separate optional library.

<a href="https://github.com/rabbitmq/erlando">Erlando</a> and <a
href="https://github.com/fogfish/datum">Datum</a> were both useful resources
in understanding how to adapt haskell's monads to a dynamically typed
language. However, this <a
href="http://erlang.org/pipermail/erlang-questions/2015-July/085109.html">mailing
list discussion</a> ended up being closest to what I ultimately decided to
build and hinted at how the functionality here could be layered.

<h3>Attribution</h3>

Image by Jukka Isokoski

CC BY-SA 3.0 [https://creativecommons.org/licenses/by-sa/3.0]

@end
Binary file added doc/pipeline.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 9194289

Please sign in to comment.