Skip to content
This repository has been archived by the owner on Feb 29, 2024. It is now read-only.

Commit

Permalink
Merge pull request #26 from ingonyama-zk/add-colab-instructions
Browse files Browse the repository at this point in the history
add colab instructions
  • Loading branch information
ImmanuelSegol authored Jan 22, 2024
2 parents 7c3a16d + 6812cce commit adfbe4a
Show file tree
Hide file tree
Showing 8 changed files with 201 additions and 40 deletions.
1 change: 1 addition & 0 deletions .codespellignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ICICLE
18 changes: 18 additions & 0 deletions .github/workflows/codespell.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Check Spelling

on:
pull_request:
branches:
- main

jobs:
spelling-checker:
name: Check Spelling
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: codespell-project/actions-codespell@v2
with:
path: docs
# https://github.com/codespell-project/actions-codespell?tab=readme-ov-file#parameter-ignore_words_file
ignore_words_file: .codespellignore
138 changes: 138 additions & 0 deletions docs/icicle/colab-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Run ICICLE on Google Colab

Google Colab let's you use a GPU free of charge, it's an Nvidia T4 GPU with 16 GB of memory, capable of running latest CUDA (tested on Cuda 12.2)
As Colab is able to interact with shell commands, a user can also install a framework and load git repositories into Colab space.

## Prepare Colab environment

First thing to do in a notebook is to set the runtime type to a T4 GPU.

* in the upper corner click on the dropdown menu and select "change runtime type"

![Change runtime](../../static/img/colab_change_runtime.png)

* In the window select "T4 GPU" and press Save

![T4 GPU](../../static/img/t4_gpu.png)

Installing Rust is rather simple, just execute the following command:

```sh
!apt install rustc cargo
```

To test the installation of Rust:

```sh
!rustc --version
!cargo --version
```

A successful installation will result in a rustc and cargo version print, a faulty installation will look like this:

```sh
/bin/bash: line 1: rustc: command not found
/bin/bash: line 1: cargo: command not found
```

Now we will check the environment:

```sh
!nvcc --version
!gcc --version
!cmake --version
!nvidia-smi
```

A correct environment should print the result with no bash errors for `nvidia-smi` command and result in a **Teslt T4 GPU** type:

```sh
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

cmake version 3.27.9

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Wed Jan 17 13:10:18 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 39C P8 9W / 70W | 0MiB / 15360MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
```

## Cloning ICICLE and running test

Now we are ready to clone ICICE repository,

```sh
!git clone https://github.com/ingonyama-zk/icicle.git
```

We now can browse the repository and run tests to check the runtime environment:

```sh
!ls -la
%cd icicle
```

Let's run a test!
Navigate to icicle/wrappers/rust/icicle-curves/icicle-bn254 and run cargo test:

```sh
%cd icicle/wrappers/rust/icicle-curves/icicle-bn254/
!cargo test --release
```

:::note

Compiling the first time may take a while

:::

Test run should end like this:

```sh
running 15 tests
test curve::tests::test_ark_point_convert ... ok
test curve::tests::test_ark_scalar_convert ... ok
test curve::tests::test_affine_projective_convert ... ok
test curve::tests::test_point_equality ... ok
test curve::tests::test_field_convert_montgomery ... ok
test curve::tests::test_scalar_equality ... ok
test curve::tests::test_points_convert_montgomery ... ok
test msm::tests::test_msm ... ok
test msm::tests::test_msm_skewed_distributions ... ok
test ntt::tests::test_ntt ... ok
test ntt::tests::test_ntt_arbitrary_coset ... ok
test msm::tests::test_msm_batch has been running for over 60 seconds
test msm::tests::test_msm_batch ... ok
test ntt::tests::test_ntt_coset_from_subgroup ... ok
test ntt::tests::test_ntt_device_async ... ok
test ntt::tests::test_ntt_batch ... ok

test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 99.39s
```

Viola, ICICLE in Colab!
77 changes: 38 additions & 39 deletions docs/icicle/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,10 @@ ICICLE Core would typically be compiled into a static library and used in a thir
### ICICLE Rust and Golang bindings

- [ICICLE Rust bindings](https://github.com/ingonyama-zk/icicle/tree/main/wrappers/rust)
- [ICICLE Golang bindings](https://github.com/ingonyama-zk/icicle/tree/main/goicicle)
- [ICICLE Golang bindings](https://github.com/ingonyama-zk/icicle/tree/main/goicicle)

These bindings allow you to easily use ICICLE in a Rust or Golang project. Setting up Golang bindings requires a bit of extra steps compared to the Rust bindings which utilize the `cargo build` tool.


## Running ICICLE

This guide assumes that you have a Linux or Windows machine with a Nvidia GPU installed. If you don't have access to a Nvidia GPU you can access one for free on [Google Colab](https://colab.google/).
Expand All @@ -44,53 +43,51 @@ This guide assumes that you have a Linux or Windows machine with a Nvidia GPU in

#### Optional Prerequisites

- Docker, latest version.
- Docker, latest version.
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html)


If you don't wish to install these prerequisites you can follow this tutorial using a [ZK-Container](https://github.com/ingonyama-zk/icicle/blob/main/Dockerfile) (docker container). To learn more about using ZK-Containers [read this](../ZKContainers.md).

### Setting up ICICLE and running tests

The objective of this guide is to make sure you can run the ICICLE Core, Rust and Golang tests. Achieving this will ensure you know how to setup ICICLE and run a ICICLE program. For simplicity, we will be using the ICICLE docker container as our environment, however, you may install the prerequisites on your machine and follow the same commands in your terminal.


#### Setting up our environment

Lets begin by cloning the ICICLE repository:

```
```sh
git clone https://github.com/ingonyama-zk/icicle
```

We wil proceed to build the docker image [found here](https://github.com/ingonyama-zk/icicle/blob/main/Dockerfile):
We will proceed to build the docker image [found here](https://github.com/ingonyama-zk/icicle/blob/main/Dockerfile):

```
```sh
docker build -t icicle-demo .
docker run -it --runtime=nvidia --gpus all --name icicle_container icicle-demo
```

- `-it` runs the container in interactive mode with a terminal.
- `-it` runs the container in interactive mode with a terminal.
- `--gpus all` Allocate all available GPUs to the container. You can also specify which GPUs to use if you don't want to allocate all.
- `--runtime=nvidia` Use the NVIDIA runtime, necessary for GPU support.

To read more about these settings reference this [article](https://developer.nvidia.com/nvidia-container-runtime).

If you accidentally close your terminal and want to reconnect just call:

```
```sh
docker exec -it icicle_container bash
```

Lets make sure that we have the correct CUDA version before proceeding

```
```sh
nvcc --version
```

You should see something like this

```
```sh
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Expand All @@ -104,31 +101,32 @@ Make sure the release version is at least 12.0.

ICICLE Core is found under [`<project_root>/icicle`](https://github.com/ingonyama-zk/icicle/tree/main/icicle). To build and run the tests first:

```
```sh
cd icicle
```

We are going to compile ICICLE for a specific curve

```
```sh
mkdir -p build
cmake -S . -B build -DCURVE=bn254 -DBUILD_TESTS=ON
cmake --build build
```

`-DBUILD_TESTS=ON` compiles the tests, without this flag `ctest` wont work.
`-DBUILD_TESTS=ON` compiles the tests, without this flag `ctest` won't work.
`-DCURVE=bn254` tells the compiler which curve to build. You can find a list of supported curves [here](https://github.com/ingonyama-zk/icicle/tree/main/icicle/curves).

The output in `build` folder should include the static libraries for the compiled curve.


:::info
Make sure to only use `-DBUILD_TESTS=ON` for running test as the archive output will only be available when `-DBUILD_TESTS=ON` is not supplied.

Make sure to only use `-DBUILD_TESTS=ON` for running tests as the archive output will only be available when `-DBUILD_TESTS=ON` is not supplied.

:::

To run the test
To run the test

```
```sh
cd build
ctest
```
Expand All @@ -141,29 +139,29 @@ Similar to ICICLE Core here we also have to compile per curve.

Lets compile curve `bn254`

```
```sh
cd wrappers/rust/icicle-curves/icicle-bn254
```

Now lets build our library

```
```sh
cargo build --release
```

This may take a couple of minutes since we are compiling both the CUDA and Rust code.

To run the tests

```
```sh
cargo test -- --test-threads=1
```

`--test-threads=1` is needed because currently some tests might interfere with one another inside the GPU.

We also include some benchmarks

```
```sh
cargo bench
```

Expand All @@ -177,42 +175,43 @@ ICICLE examples can be found [here](https://github.com/ingonyama-zk/icicle-examp

In each example directory, ZK-container files are located in a subdirectory `.devcontainer`.

* example-name/
* .devcontainer/
* Dockerfile
* devcontainer.json

```sh
msm/
├── .devcontainer
├── devcontainer.json
└── Dockerfile
```

Lets run one of our C++ examples, in this case the [MSM example](https://github.com/ingonyama-zk/icicle-examples/blob/main/c%2B%2B/msm/example.cu).

Clone the repository

```
```sh
git clone https://github.com/ingonyama-zk/icicle-examples.git
cd icicle-examples
```

Enter the test directory

```
```sh
cd c++/msm
```

Now lets build our docker file and run the test inside it. Make sure you have installed the [optional prerequisites](#optional-prerequisites).

```
```sh
docker build -t icicle-example-msm -f .devcontainer/Dockerfile .
```

Lets start and enter the container

```
```sh
docker run -it --rm --gpus all -v .:/icicle-example icicle-example-msm
```

to run the example

```
```sh
rm -rf build
mkdir -p build
cmake -S . -B build
Expand All @@ -230,7 +229,6 @@ static libraries can be loaded into memory once and used by multiple programs, r

Lets review the Golang bindings since its a pretty verbose example (compared to rust which hides it pretty well) of using static libraries. Golang has a library named `CGO` which can be used to link static libraries. Here's a basic example on how you can use cgo to link these libraries:


```go
/*
#cgo LDFLAGS: -L/path/to/shared/libs -lbn254 -lbls12_381 -lbls12_377 -lbw6_671
Expand All @@ -239,13 +237,13 @@ Lets review the Golang bindings since its a pretty verbose example (compared to
import "C"

func main() {
// Now you can call the C functions from the ICICLE libraries.
// Note that C function calls are prefixed with 'C.' in Go code.
// Now you can call the C functions from the ICICLE libraries.
// Note that C function calls are prefixed with 'C.' in Go code.

out := (*C.BN254_projective_t)(unsafe.Pointer(p))
in := (*C.BN254_affine_t)(unsafe.Pointer(affine))
out := (*C.BN254_projective_t)(unsafe.Pointer(p))
in := (*C.BN254_affine_t)(unsafe.Pointer(affine))

C.projective_from_affine_bn254(out, in)
C.projective_from_affine_bn254(out, in)
}
```

Expand All @@ -260,4 +258,5 @@ One of the core ideas behind ICICLE is that developers can gradually accelerate
Therefore we offer adapters for various popular libraries, these adapters allow us to convert points and scalars between different formats defined by various libraries. Here is a list:

Golang adapters:

- [Gnark crypto adapter](https://github.com/ingonyama-zk/iciclegnark)
Loading

0 comments on commit adfbe4a

Please sign in to comment.