Skip to content

Commit

Permalink
feature: conditional attributes, filtering, and text-alignment (#12)
Browse files Browse the repository at this point in the history 
* align and optional

* README

* documentation update
  • Loading branch information
@solidiquis
solidiquis authored Jan 21, 2025
1 parent cf42232 commit a0b8b44
Show file tree
Hide file tree
Showing 15 changed files with 818 additions and 478 deletions.
4 changes: 2 additions & 2 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
name = "grits"
authors = ["benjamin.van.nguyen@gmail.com"]
description = """
A text line processor that applies regular expressions with named captures
to input lines and transforms them using a user-generated template.
A line-text processor that applies regular expressions with named captures to input lines
and transforms them using a template string. See the long help '--help' for further details and examples.
"""
version = "0.2.0"
edition = "2021"
Expand Down
213 changes: 21 additions & 192 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,210 +4,39 @@
[![Crates.io](https://img.shields.io/crates/v/grits.svg)](https://crates.io/crates/grits)
[![Crates.io](https://img.shields.io/crates/d/grits)](https://crates.io/crates/grits)

`grits` is a minimal, simple, and easy-to-use line text-processor that applies regular expressions with named captures to input lines
and transforms them using a custom template string. It allows for efficient parsing, extracting, and formatting of text,
including support for colorization and other attributes you'd get using ANSI escape sequences.
A simple line-text formatter that makes it simple to parse, filter, and format live logs, turning noise into meaningful insights.

The following example demonstrates how to apply `grits` to `tcpdump` to extract a packet's source (`src`) and destination (`dst`) IP address:
![demo gif](images/log.gif)

```bash
sudo tcpdump -nn | grits -p '^(?<ts>[^ ]+)' \
-p 'IP\w? (?<src>[^ ]+)' \
-p '> (?<dst>[^ ]+):' \
-t '[{(cyan|bold):ts}] {(green|underlined):"src"}={src} {(yellow|underlined):"dst"}={dst}'
```

![demo image](images/demo.png)
The top pane in the above screenshot is the raw output of `tcpdump` while the bottom pane shows the output being piped into `grits`.

## Table of Contents

* [Usage](#usage)
* [Installation](#installation)
* [Templating language](#templating-language)
- [Indexing](#indexing)
- [Default values](#default-values)
- [Attributes](#attributes)
- [Other examples](#other-examples)
* [Completions](#completions)
* [Contributing](#contributing)
* [Donating](#donating)
* [FAQ](#faq)

## Usage

```
A text line processor that applies regular expressions with named captures to input lines
and transforms them using a user-generated template. See the long help '--help' for further
details and examples.
Usage: grits [OPTIONS] --template <TEMPLATE> [FILES]...
Arguments:
[FILES]... Input files
Options:
-p, --pattern <PATTERN> A regular expression with named captures. Can be specified multiple times
-t, --template <TEMPLATE> A template string that defines how to transform a line input. See long '--help'
-r, --require <REQUIRE> Name of capture that must have at least one match for the output to show. Can be specified multiple times
--line-buffered Force output to be line-buffered. By default, output is line buffered when stdout is a terminal and block-buffered otherwise
-c, --completions <COMPLETIONS> Produce completions for shell and exit [possible values: bash, elvish, fish, powershell, zsh]
-h, --help Print help (see more with '--help')
-V, --version Print version
```

See the long `--help` description for further details and example from the command-line.

## Installation

### crates.io
<p align="center">
<em>An example of Grits in action: the left pane shows Grits being applied, and the right pane displays the raw logs.</em>
</p>

```bash
cargo install grits
```

### cURL

```bash
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/solidiquis/grits/releases/download/v0.2.0/grits-installer.sh | sh
```
At its core, `grits` applies regular expressions with named captures to input lines. These captures are then available as variables
(a.k.a. anchors) which can then be used in a `grits` template string. The template string supports text-alignment, colorization,
as well as other attributes you'd expect using ANSI escape sequences.

### Powershell
The following example demonstrates how to apply `grits` to `tcpdump` to extract an output line's timestamp (`ts`) and
a packet's source (`src`) and destination (`dst`) IP address:

```bash
powershell -ExecutionPolicy Bypass -c "irm https://github.com/solidiquis/grits/releases/download/v0.2.0/grits-installer.ps1 | iex"
```

### Manual installation

Check the [releases page](https://github.com/solidiquis/grits/releases) for prebuilt binaries.

## Templating language

`grits` uses a simple templating language to transform text, where templates consist of anchors.
Anchors are placeholders enclosed within `{...}` that correspond to named capture groups from
the regular expression applied to the input. Once a match is found, the value from the
capture group is inserted into the anchor’s position in the template string.

Here's an example:
```bash
echo 'level=info msg=foobar path=/baz' | grit -p 'msg=(?<log>[^ ]+)' -t 'transformed={log}'
```

In this command, we use a regular expression to capture the value associated with the msg field.
The capture group is named `log`. The template string `transformed={log}` will replace `{log}` with
the value captured from the input. The output will then be:

```
transformed=foobar
```

To summarize:
- The regular expression `msg=(?<log>[^ ]+)` captures the value `foobar` into the `log` capture group.
- The template `transformed={log}` uses the value of `log` to generate the output.

The following are additional features of `grits` templating system:

### Indexing

When there are multiple matches for a given named capture group, you can use **indexing**
to specify which occurrence of the capture to use. The index is placed in square brackets
immediately after the anchor name. For example, to access the second match of the `log`
capture group, you would use:

```
{log[1]}
```

### Default values

If a particular anchor doesn't have an associated match, default values can be chained using the `||`
operator like so:

```
{log || foo || bar[1] || "NO MATCH"}
```

The first default value that doesn't produce a blank string will be used. Default values can be
other anchors or a string literal.

### Attributes

Attributes offer additional means to transform text. Attributes are applied to anchors like so:

```
{(red|bold):ipaddr_v4}
```

Here is an example using attributes with default values:


```
{(red|bold):ipaddr_v4 || ipaddr_v6 || "NOMATCH"}
```

In the above example, `red` and `bold` will be applied the entire anchor.


The following attributes are currently available:

- `black` (apply a black foreground)
- `red` (apply a red foreground)
- `green` (apply a green foreground)
- `yellow` (apply a yellow foreground)
- `blue` (apply a blue foreground)
- `magenta` (apply a magenta foreground)
- `cyan` (apply a cyan foreground)
- `white` (apply a white foreground)
- `bg_black` (apply a black background)
- `bg_red` (apply a red background)
- `bg_green` (apply a green background)
- `bg_yellow` (apply a yellow background)
- `bg_blue` (apply a blue background)
- `bg_magenta` (apply a magenta background)
- `bg_cyan` (apply a cyan background)
- `bg_white` (apply a white background)
- `bold` (make text bold)
- `underlined` (underline text)
- `reverse` (reverse text)
- `crossed_out` (crossout text)

### Other examples

1. Multi-file processing:

```bash
grits -p 'sysctl=(?<sysctl>.*)'` -t 'sysctl output: {sysctl}' file1 file2
```

2. Piping:

```bash
docker logs -f 93670ea0964c | grits -p 'log_level=info(?<log>.*)' -t 'INFO LOG: {log}'
```

3. Attributes, default values, and multiple regular expressions:

```bash
kubectl logs -f -n foo -l app=bar | grits \
-p '^kernel:(?<kern>.*)' \
-p '^sysctl:(?<sys>.*)' \
-t 'kernel={(cyan):kern || \"NONE\"} sysctl={(magenta):sys || \"NONE\"}'
tcpdump -nn | grits -- \
-p '^(?<ts>[^ ]+)' \
-p 'IP\w? (?<src>[^ ]+)' \
-p '> (?<dst>[^ ]+):' \
-t '[{(cyan|bold):ts}] {(green|underlined):"src"}={(lalign(45)):src} {(yellow|underlined):"dst"}={dst}'
```

## Completions

Completions for supported shells can be generated using `grits --completions <SHELl>`. Consult your shell's documentation
for how to setup completions. For `zsh`, completions are bootstrapped like so:
![demo image](images/demo.png)
<p align="center">
<em>The top pane in the above screenshot is the raw output of tcpdump while the bottom pane shows the output being piped into grits.</em>
</p>

```bash
grits --completions zsh > ~/.oh-my-zsh/completions/_grits
```

## Colorization
## Documentation

`grits` follows the informal [NO_COLOR](https://no-color.org/) standard. Setting `NO_COLOR` to a non-blank value will disable output colorization.
If stdout is not a terminal, colorization is automatically disabled.
The documentation for `grits` can be found [here](./docs/help.md). It is also available in the CLI via `--help`.

## Contributing

Expand Down
Loading

0 comments on commit a0b8b44

Please sign in to comment.