Skip to content

Commit

Permalink
ok
Browse files Browse the repository at this point in the history
  • Loading branch information
aljo242 committed Dec 3, 2024
1 parent 7a115b1 commit dfbef67
Show file tree
Hide file tree
Showing 6 changed files with 10 additions and 9 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ $ go install github.com/skip-mev/connect/v2
The connect repository is composed of the following core packages:

* **abci** - This package contains the [vote extension](./abci/ve/README.md), [proposal](./abci/proposals/README.md), and [preblock handlers](./abci/preblock/oracle/README.md) that are used to broadcast oracle data to the network and to store it in the blockchain.
* **oracle** - This [package](./oracle/) contains the main oracle that aggregates external data sources before broadcasting it to the network. You can reference the provider documentation [here](./providers/base/README.md) to get a high level overview of how the oracle works.
* **oracle** - This [package](./oracle/) contains the main oracle that aggregates external data sources before broadcasting it to the network. You can reference the provider documentation [here](providers/README.md) to get a high level overview of how the oracle works.
* **providers** - This package contains a collection of [websocket](./providers/websockets/README.md) and [API](./providers/apis/README.md) based data providers that are used by the oracle to collect external data.
* **x/oracle** - This package contains a Cosmos SDK module that allows you to store oracle data on a blockchain.
* **x/marketmap** - This [package](./x/marketmap/README.md) contains a Cosmos SDK module that allows for market configuration to be stored and updated on a blockchain.
Expand Down
1 change: 1 addition & 0 deletions providers/EXAMPLE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Cr
12 changes: 6 additions & 6 deletions providers/base/README.md → providers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ Each base provider implementation will be run in a separate goroutine by the mai

The base provider constructs a response channel that it is always listening to and making updates as needed. Every interval, the base provider will fetch the data from the underlying data source and send the response to the response channel, respecting the number of concurrent requests to the rate limit parameters of the underlying source (if it has any).

![Architecture Overview](./architecture.png)
![Architecture Overview](architecture.png)


## API (HTTP) Based Providers

In order to implement API based providers, you must implement the [`APIDataHandler`](./api/handlers/api_data_handler.go) interface and the [`RequestHandler`](./api/handlers/request_handler.go) interfaces. The `APIDataHandler` is responsible for creating the URL to be sent to the HTTP client and parsing the response from the HTTP response. The `RequestHandler` is responsible for making the HTTP request and returning the response.
In order to implement API based providers, you must implement the [`APIDataHandler`](base/api/handlers/api_data_handler.go) interface and the [`RequestHandler`](base/api/handlers/request_handler.go) interfaces. The `APIDataHandler` is responsible for creating the URL to be sent to the HTTP client and parsing the response from the HTTP response. The `RequestHandler` is responsible for making the HTTP request and returning the response.

Once these two interfaces are implemented, you can then instantiate an [`APIQueryHandler`](./api/handlers/api_query_handler.go) and pass it to the base provider. The `APIQueryHandler` is abstracts away the logic for making the HTTP request and parsing the response. The base provider will then take care of the rest. The responses from the `APIQueryHandler` are sent to the base provider via a buffered channel. The base provider will then store the data in a thread safe map. To read more about the various API provider configurations available, please visit the [API provider configuration](../../oracle/config/api.go) documentation.
Once these two interfaces are implemented, you can then instantiate an [`APIQueryHandler`](base/api/handlers/api_query_handler.go) and pass it to the base provider. The `APIQueryHandler` is abstracts away the logic for making the HTTP request and parsing the response. The base provider will then take care of the rest. The responses from the `APIQueryHandler` are sent to the base provider via a buffered channel. The base provider will then store the data in a thread safe map. To read more about the various API provider configurations available, please visit the [API provider configuration](../oracle/config/api.go) documentation.

Alternatively, you can directly implement the [`APIFetcher`](./api/handlers/api_query_handler.go) interface. This is appropriate if you want to abstract over the various processes of interacting with GRPC, JSON-RPC, REST, etc. APIs.
Alternatively, you can directly implement the [`APIFetcher`](base/api/handlers/api_query_handler.go) interface. This is appropriate if you want to abstract over the various processes of interacting with GRPC, JSON-RPC, REST, etc. APIs.

### APIDataHandler

Expand Down Expand Up @@ -93,9 +93,9 @@ type APIFetcher[K providertypes.ResponseKey, V providertypes.ResponseValue] inte

## Websocket-Based Providers

In order to implement websocket-based providers, you must implement the [`WebSocketDataHandler`](./websocket/handlers/ws_data_handler.go) interface and the [`WebSocketConnHandler`](./websocket/handlers/ws_conn_handler.go) interfaces. The `WebSocketDataHandler` is responsible for parsing messages from the websocket connection, constructing heartbeats, and constructing the initial subscription message(s). This handler must manage all state associated with the websocket connection i.e. connection identifiers. The `WebSocketConnHandler` is responsible for making the websocket connection and maintaining it - including reads, writes, dialing, and closing.
In order to implement websocket-based providers, you must implement the [`WebSocketDataHandler`](base/websocket/handlers/ws_data_handler.go) interface and the [`WebSocketConnHandler`](base/websocket/handlers/ws_conn_handler.go) interfaces. The `WebSocketDataHandler` is responsible for parsing messages from the websocket connection, constructing heartbeats, and constructing the initial subscription message(s). This handler must manage all state associated with the websocket connection i.e. connection identifiers. The `WebSocketConnHandler` is responsible for making the websocket connection and maintaining it - including reads, writes, dialing, and closing.

Once these two interfaces are implemented, you can then instantiate an [`WebSocketQueryHandler`](./websocket/handlers/ws_query_handler.go) and pass it to the base provider. The `WebSocketQueryHandler` abstracts away the logic for connecting, reading, sending updates, and parsing responses all using the two interfaces above. The base provider will then take care of the rest - including storing the data in a thread safe manner. To read more about the various configurations available for websocket providers, please visit the [websocket provider configuration](../../oracle/config/websocket.go) documentation.
Once these two interfaces are implemented, you can then instantiate an [`WebSocketQueryHandler`](base/websocket/handlers/ws_query_handler.go) and pass it to the base provider. The `WebSocketQueryHandler` abstracts away the logic for connecting, reading, sending updates, and parsing responses all using the two interfaces above. The base provider will then take care of the rest - including storing the data in a thread safe manner. To read more about the various configurations available for websocket providers, please visit the [websocket provider configuration](../oracle/config/websocket.go) documentation.

### WebSocketDataHandler

Expand Down
2 changes: 1 addition & 1 deletion providers/apis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

API providers utilize rest APIs to retrieve data from external sources. The data is then transformed into a common format and aggregated across multiple providers. To implement a new provider, please read over the base provider documentation in [`providers/base/README.md`](../base/README.md).
API providers utilize rest APIs to retrieve data from external sources. The data is then transformed into a common format and aggregated across multiple providers. To implement a new provider, please read over the base provider documentation in [`providers/base/README.md`](../README.md).

## Supported Providers

Expand Down
File renamed without changes
2 changes: 1 addition & 1 deletion providers/websockets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

Websocket providers utilize websocket APIs / clients to retrieve data from external sources. The data is then transformed into a common format and aggregated across multiple providers. To implement a new provider, please read over the base provider documentation in [`providers/base/README.md`](../base/README.md).
Websocket providers utilize websocket APIs / clients to retrieve data from external sources. The data is then transformed into a common format and aggregated across multiple providers. To implement a new provider, please read over the base provider documentation in [`providers/base/README.md`](../README.md).

Websockets are preferred over REST APIs for real-time data as they only require a single connection to the server, whereas HTTP APIs require a new connection for each request. This makes websockets more efficient for real-time data. Additionally, web sockets typically have lower latency than HTTP APIs, which is important for real-time data.

Expand Down

0 comments on commit dfbef67

Please sign in to comment.