-
Notifications
You must be signed in to change notification settings - Fork 26
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add workflow descriptions for onboarding data and labels (#139)
- Loading branch information
1 parent
1e36fb1
commit e4d6478
Showing
8 changed files
with
232 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
5 changes: 5 additions & 0 deletions
5
docs/docs/workflows/importing-data-and-labels/_category_.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"label": "Importing Data and Labels", | ||
"position": 1, | ||
"collapsible": true | ||
} |
24 changes: 24 additions & 0 deletions
24
docs/docs/workflows/importing-data-and-labels/custom-data-import.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
sidebar_position: 4 | ||
--- | ||
|
||
# Custom Data Import | ||
|
||
> 🔥 HARD 🔥 If you have a custom data and label structure. | ||
In the situation where you have a custom data and label structure, you have two options: | ||
|
||
1. [Migrate your data to Encord](/sdk/migrating-data) before [importing](./import-encord-project) it with | ||
`encord-active import project` | ||
2. Convert your data and labels to the [COCO data format][coco-format] before [importing](./import-coco-project) it with | ||
`encord-active import project --coco -i ./images -a ./annotations.json`. | ||
|
||
:::info | ||
|
||
We are working hard on a smoother data onboarding process so you won't need an Encord account and won't have to ship your data anywhere. | ||
If you want to know more, please contact us via the [slack community][slack-invite] or by sending us an email on [active@encord.com](mailto:active@encord.com). | ||
|
||
::: | ||
|
||
[slack-invite]: https://join.slack.com/t/encordactive/shared_invite/zt-1hc2vqur9-Fzj1EEAHoqu91sZ0CX0A7Q | ||
[coco-format]: https://cocodataset.org/#format-data |
30 changes: 30 additions & 0 deletions
30
docs/docs/workflows/importing-data-and-labels/import-coco-project.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
--- | ||
sidebar_position: 2 | ||
--- | ||
|
||
# Importing COCO Project | ||
|
||
> Import COCO Images and anotations stored on your local file system. | ||
This requires you to have a coco project on your local machine with images and an annotations JSON file. | ||
|
||
To import the project, run | ||
|
||
```shell | ||
encord-active import project --coco -i ./images -a ./annotations.json | ||
``` | ||
|
||
This will create a new Encord Active project in a new directory in you current working directory. | ||
Afterwards, you can run | ||
|
||
```shell | ||
encord-active visualize | ||
``` | ||
|
||
This will let you choose your newly imported project and open the app. | ||
|
||
:::info | ||
|
||
For the full documentation of importing COCO projects, please see [here](/cli/import-coco-project). | ||
|
||
::: |
41 changes: 41 additions & 0 deletions
41
docs/docs/workflows/importing-data-and-labels/import-encord-project.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
--- | ||
sidebar_position: 1 | ||
--- | ||
|
||
# Importing Encord Project | ||
|
||
> For projects already in the Encord platform. | ||
This workflow will get you through importing your data from Encord into a local Encord Active project. | ||
You will get a directory containing all the data, labels, and [metrics](/category/metrics) of the project. | ||
|
||
To import your project, run this command: | ||
|
||
```shell | ||
encord-active import project | ||
``` | ||
|
||
If this is the first time you import a project, the command line interface (CLI) will ask you for your ssh key. | ||
To associate an ssh key with Encord, you can follow [the documentation here][encord-docs-ssh]. | ||
You ssh key path will be stored for later reuse. | ||
|
||
Next, the CLI will ask you what project to import based on all the projects you have access to at Encord. | ||
You can type in a search word to find the project you want to import. | ||
Use the keyboard arrows to select your project and hit <kbd>enter</kbd>. | ||
|
||
This will create a new Encord Active project in a new directory in you current working directory. | ||
Afterwards, you can run | ||
|
||
```shell | ||
encord-active visualize | ||
``` | ||
|
||
This will let you choose your newly imported project and open the app. | ||
|
||
:::info | ||
|
||
For the full documentation of importing Encord projects, please see [here](/cli/import-encord-project). | ||
|
||
::: | ||
|
||
[encord-docs-ssh]: https://docs.encord.com/admins/settings/public-keys/#set-up-public-key-authentication |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
import DocCardList from "@theme/DocCardList"; | ||
|
||
# Importing Data and Labels | ||
|
||
To use Encord Active, you need data. | ||
This page shows you ways in which you can import your data into Encord Active. | ||
|
||
Depending on where your data if currently stored, you can choose the option that best suits you. | ||
|
||
<DocCardList /> | ||
|
||
Current data and label support of the four workflows are: | ||
|
||
<table> | ||
<thead> | ||
<tr> | ||
<td align="center"> | ||
<b>Import Type</b> | ||
</td> | ||
<td colspan="2" align="center"> | ||
<b>Data Type</b> | ||
</td> | ||
<td colspan="3" align="center"> | ||
<b>Label Type</b> | ||
</td> | ||
</tr> | ||
</thead> | ||
<tbody> | ||
<tr> | ||
<td align="center"></td> | ||
<td align="center"> | ||
<b>Images</b> | ||
</td> | ||
<td align="center"> | ||
<b>Videos</b> | ||
</td> | ||
<td align="center"> | ||
<b>Classification</b> | ||
</td> | ||
<td align="center"> | ||
<b>Bounding Boxes</b> | ||
</td> | ||
<td align="center"> | ||
<b>Polygons</b> | ||
</td> | ||
</tr> | ||
<tr> | ||
<td> | ||
<a href="./import-encord-project">Encord</a> | ||
</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
</tr> | ||
<tr> | ||
<td> | ||
<a href="./import-coco-project">COCO</a> | ||
</td> | ||
<td align="center">✅</td> | ||
<td align="center">-</td> | ||
<td align="center">-</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
</tr> | ||
<tr> | ||
<td> | ||
<a href="./initialize-from-data-directory">Data directory</a> | ||
</td> | ||
<td align="center">✅</td> | ||
<td align="center">-</td> | ||
<td align="center">✅</td> | ||
<td align="center">-</td> | ||
<td align="center">-</td> | ||
</tr> | ||
<tr> | ||
<td> | ||
<a href="./custom-data-import">Custom Data - 🔥 hard 🔥</a> | ||
</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
<td align="center">✅</td> | ||
</tr> | ||
</tbody> | ||
</table> |
43 changes: 43 additions & 0 deletions
43
docs/docs/workflows/importing-data-and-labels/initialize-from-data-directory.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
--- | ||
sidebar_position: 3 | ||
--- | ||
|
||
# Initializing from Image Directory | ||
|
||
> Grep arbitrary images from within a dataset directory. | ||
If you have images stored in a (potentially nested) directory, you can grep all the images and initialize an Encord Active project from these images. | ||
To do this run | ||
|
||
```shell | ||
encord-active init /path/to/image/dir | ||
``` | ||
|
||
By default, the command will find all `.jpg`, `.jpeg`, `.png`, and `.tiff` files. | ||
If you want to change this, you can use the `--glob` option to target your file selection. | ||
|
||
:::tip | ||
|
||
The `--dryrun` flag will print the matched files without actually running the import. | ||
|
||
::: | ||
:::tip | ||
|
||
The `--symlinks` flag will initialize the project symlinks to your files without copying them to save you disk space. | ||
|
||
::: | ||
|
||
The `init` command will import all the found images into an Encord Active project stored in a directory within your current working directory. | ||
Afterwards, you can run | ||
|
||
```shell | ||
encord-active visualize | ||
``` | ||
|
||
This will let you choose your newly imported project and open the app. | ||
|
||
:::info | ||
|
||
For the full documentation of initializing a project from an image directory, please see [here](/cli/initialising-project-from-image-directories). | ||
|
||
::: |