Skip to content

Commit

Permalink
docs: add workflow descriptions for onboarding data and labels (#139)
Browse files Browse the repository at this point in the history
  • Loading branch information
frederik-encord authored Feb 2, 2023
1 parent 1e36fb1 commit e4d6478
Show file tree
Hide file tree
Showing 8 changed files with 232 additions and 28 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ You have some additional options to tailor the initialisation of the project for

#### `--glob` (or `-g`)

> **Default:** `"**/*.jpg"`, `"**/*.png"`, `"**/*.jpeg"`, `"**/*.tiff"`.
> **Default:** `"**/*.jpg"`, `"**/*.jpeg"`, `"**/*.png"`, `"**/*.tiff"`.
Glob patterns are used to choose files.
You can specify multiple options if you wish to include files from specific subdirectories.
Expand Down
27 changes: 0 additions & 27 deletions docs/docs/workflows/import-data.mdx

This file was deleted.

5 changes: 5 additions & 0 deletions docs/docs/workflows/importing-data-and-labels/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"label": "Importing Data and Labels",
"position": 1,
"collapsible": true
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
sidebar_position: 4
---

# Custom Data Import

> 🔥 HARD 🔥 If you have a custom data and label structure.
In the situation where you have a custom data and label structure, you have two options:

1. [Migrate your data to Encord](/sdk/migrating-data) before [importing](./import-encord-project) it with
`encord-active import project`
2. Convert your data and labels to the [COCO data format][coco-format] before [importing](./import-coco-project) it with
`encord-active import project --coco -i ./images -a ./annotations.json`.

:::info

We are working hard on a smoother data onboarding process so you won't need an Encord account and won't have to ship your data anywhere.
If you want to know more, please contact us via the [slack community][slack-invite] or by sending us an email on [active@encord.com](mailto:active@encord.com).

:::

[slack-invite]: https://join.slack.com/t/encordactive/shared_invite/zt-1hc2vqur9-Fzj1EEAHoqu91sZ0CX0A7Q
[coco-format]: https://cocodataset.org/#format-data
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
sidebar_position: 2
---

# Importing COCO Project

> Import COCO Images and anotations stored on your local file system.
This requires you to have a coco project on your local machine with images and an annotations JSON file.

To import the project, run

```shell
encord-active import project --coco -i ./images -a ./annotations.json
```

This will create a new Encord Active project in a new directory in you current working directory.
Afterwards, you can run

```shell
encord-active visualize
```

This will let you choose your newly imported project and open the app.

:::info

For the full documentation of importing COCO projects, please see [here](/cli/import-coco-project).

:::
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
sidebar_position: 1
---

# Importing Encord Project

> For projects already in the Encord platform.
This workflow will get you through importing your data from Encord into a local Encord Active project.
You will get a directory containing all the data, labels, and [metrics](/category/metrics) of the project.

To import your project, run this command:

```shell
encord-active import project
```

If this is the first time you import a project, the command line interface (CLI) will ask you for your ssh key.
To associate an ssh key with Encord, you can follow [the documentation here][encord-docs-ssh].
You ssh key path will be stored for later reuse.

Next, the CLI will ask you what project to import based on all the projects you have access to at Encord.
You can type in a search word to find the project you want to import.
Use the keyboard arrows to select your project and hit <kbd>enter</kbd>.

This will create a new Encord Active project in a new directory in you current working directory.
Afterwards, you can run

```shell
encord-active visualize
```

This will let you choose your newly imported project and open the app.

:::info

For the full documentation of importing Encord projects, please see [here](/cli/import-encord-project).

:::

[encord-docs-ssh]: https://docs.encord.com/admins/settings/public-keys/#set-up-public-key-authentication
88 changes: 88 additions & 0 deletions docs/docs/workflows/importing-data-and-labels/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
import DocCardList from "@theme/DocCardList";

# Importing Data and Labels

To use Encord Active, you need data.
This page shows you ways in which you can import your data into Encord Active.

Depending on where your data if currently stored, you can choose the option that best suits you.

<DocCardList />

Current data and label support of the four workflows are:

<table>
<thead>
<tr>
<td align="center">
<b>Import Type</b>
</td>
<td colspan="2" align="center">
<b>Data Type</b>
</td>
<td colspan="3" align="center">
<b>Label Type</b>
</td>
</tr>
</thead>
<tbody>
<tr>
<td align="center"></td>
<td align="center">
<b>Images</b>
</td>
<td align="center">
<b>Videos</b>
</td>
<td align="center">
<b>Classification</b>
</td>
<td align="center">
<b>Bounding Boxes</b>
</td>
<td align="center">
<b>Polygons</b>
</td>
</tr>
<tr>
<td>
<a href="./import-encord-project">Encord</a>
</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
</tr>
<tr>
<td>
<a href="./import-coco-project">COCO</a>
</td>
<td align="center">✅</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">✅</td>
<td align="center">✅</td>
</tr>
<tr>
<td>
<a href="./initialize-from-data-directory">Data directory</a>
</td>
<td align="center">✅</td>
<td align="center">-</td>
<td align="center">✅</td>
<td align="center">-</td>
<td align="center">-</td>
</tr>
<tr>
<td>
<a href="./custom-data-import">Custom Data - 🔥 hard 🔥</a>
</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
<td align="center">✅</td>
</tr>
</tbody>
</table>
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
sidebar_position: 3
---

# Initializing from Image Directory

> Grep arbitrary images from within a dataset directory.
If you have images stored in a (potentially nested) directory, you can grep all the images and initialize an Encord Active project from these images.
To do this run

```shell
encord-active init /path/to/image/dir
```

By default, the command will find all `.jpg`, `.jpeg`, `.png`, and `.tiff` files.
If you want to change this, you can use the `--glob` option to target your file selection.

:::tip

The `--dryrun` flag will print the matched files without actually running the import.

:::
:::tip

The `--symlinks` flag will initialize the project symlinks to your files without copying them to save you disk space.

:::

The `init` command will import all the found images into an Encord Active project stored in a directory within your current working directory.
Afterwards, you can run

```shell
encord-active visualize
```

This will let you choose your newly imported project and open the app.

:::info

For the full documentation of initializing a project from an image directory, please see [here](/cli/initialising-project-from-image-directories).

:::

0 comments on commit e4d6478

Please sign in to comment.