Create a hydra-explorer executable that can track all heads on-chain #1235

ffakenz · 2024-01-05T12:23:37Z

What

Create a hydra-explorer that can track all Babbage heads on-chain in a devnet network.

no multi-version support needed (for now).

Then expose a REST-API to query the "List of all heads and their current state (Initializing, Open, Closed, Final) on network"

How

Write an e2e test for hydra-explorer to drive the development; opening a single head and using the process std out to assert.
Create hydra-explorer submodule and base process.
Reuse the hydra-chain-observer library code to collect the observations
Write an e2e test for hydra-explorer driving the development; opening multiple heads and using a REST API to assert.
Store observations in-memory and maintain a dictionary of heads and states.
Create a backend http-server for hydra-explorer that binds port 9090.
Expose an api to query the "List of all heads and their current state (Initializing, Open, Closed, Final) on network" observed

GET /heads

CHANGELOG updated or not needed
Documentation updated or not needed
Haddocks updated or not needed
No new TODOs introduced or explained herafter

github-actions · 2024-01-05T12:38:48Z

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-01-29 16:32:02.175490277 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb	4072
νCommit	7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd	2043
νHead	7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772	8816
μHead	8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463*	3851

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4378	10.50	4.06	0.46
2	4576	12.57	4.83	0.49
3	4777	14.79	5.67	0.52
5	5184	19.16	7.33	0.59
10	6182	30.20	11.51	0.75
41	12415	99.34	37.73	1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	529	11.37	4.44	0.30
2	724	15.04	6.07	0.35
3	911	18.85	7.75	0.40
5	1282	26.90	11.27	0.51
10	2226	49.55	20.97	0.80
19	3904	99.43	41.75	1.43

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	480	21.78	8.51	0.41
2	114	590	33.30	13.14	0.54
3	169	700	45.91	18.28	0.68
4	227	810	58.87	23.66	0.83
5	283	920	81.34	32.63	1.08

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	558	16.72	7.70	0.37
2	613	16.72	7.30	0.37
3	912	20.13	10.89	0.43
5	1219	23.44	13.88	0.49
10	2152	32.20	22.02	0.66
50	9077	99.94	84.59	1.97

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	592	20.18	8.96	0.40
2	739	22.21	10.65	0.44
3	952	24.19	12.38	0.48
5	1271	27.66	15.42	0.54
10	2159	37.53	23.91	0.72
43	7695	99.41	77.33	1.85

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4307	18.85	8.11	0.55
2	4468	31.96	13.92	0.71
3	4644	47.75	20.94	0.89
4	4813	66.26	29.15	1.11
5	4983	86.68	38.28	1.35

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4205	8.26	3.46	0.43
5	1	57	4239	9.49	4.22	0.44
5	5	285	4375	14.40	7.27	0.51
5	10	569	4544	20.55	11.08	0.60
5	20	1139	4884	32.85	18.71	0.77
5	30	1706	5223	45.15	26.34	0.93
5	40	2278	5566	57.25	33.88	1.10
5	50	2848	5905	69.56	41.52	1.27
5	74	4210	6715	98.92	59.78	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-01-29 16:34:31.041690661 UTC

Baseline Scenario

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	21.668470432
P99	58.62517313000006ms
P95	30.014975149999987ms
P50	19.614939999999997ms
Number of Invalid txs	0

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.322876609
P99	7.28139856999999ms
P95	5.430088599999999ms
P50	4.1407445ms
Number of Invalid txs	0

github-actions · 2024-01-05T12:46:19Z

Test Results

407 tests +2 400 ✅ +2 16m 58s ⏱️ +51s
137 suites +1 7 💤 ±0
5 files ±0 0 ❌ ±0

Results for commit 594558b. ± Comparison against base commit 3659f50.

♻️ This comment has been updated with latest results.

Co-authored-by: Sasha Bogicevic <Sasha.Bogicevic@iohk.io>

As we are not serving any files yet. Co-authored-by: Sebastian Nagel <ch1bo@users.noreply.github.com>

…r helpers Co-authored-by: Sebastian Nagel <ch1bo@users.noreply.github.com>

As we do not need use http and to support TLS. Also: - moved the seedFromFaucet_ calls before starting a hydra-node. - fixed the /heads endpoint header to use Accept. - removed some unused packages after this refactor. - remove e2e spec relying on stdout as we are only interested in the http api.

…JSON value

And assert REST API behavior rather than std out logs.

…stening on port Because it was flaky as it was defined.

Added arbitrary instances for head observations.

Also: - refactor HeadState.seedTxIn to be Observed. - refactor HeadState.contestationPeriod to be Observed. - refactor HeadMember.onChainId to be Observed. Finally updated the explorer openapi to be compliant with above changes.

…vation

…ach party member

ffakenz assigned locallycompact and ffakenz Jan 5, 2024

ffakenz linked an issue Jan 5, 2024 that may be closed by this pull request

Build and deploy a Hydra heads explorer #696

Closed

6 tasks

ffakenz changed the title ~~Head explorer backend~~ Create a hydra-explorer executable that can track all heads on-chain Jan 5, 2024

ffakenz removed a link to an issue Jan 5, 2024

Build and deploy a Hydra heads explorer #696

Closed

6 tasks

ffakenz mentioned this pull request Jan 5, 2024

Build and deploy a Hydra heads explorer #696

Closed

6 tasks

ffakenz force-pushed the head-explorer-backend branch from 349f802 to e15a8fa Compare January 5, 2024 12:29

ffakenz force-pushed the head-explorer-backend branch 5 times, most recently from 83a688f to 3815053 Compare January 11, 2024 11:03

ch1bo force-pushed the head-explorer-backend branch from e2918e7 to 928e706 Compare January 12, 2024 09:13

ffakenz force-pushed the head-explorer-backend branch 2 times, most recently from 477be8d to e2ec89a Compare January 12, 2024 10:20

ch1bo force-pushed the head-explorer-backend branch from 0c1ec19 to 3f47c00 Compare January 12, 2024 10:31

ch1bo assigned ch1bo and unassigned locallycompact Jan 12, 2024

ffakenz force-pushed the head-explorer-backend branch 6 times, most recently from cb496a3 to 1f21d95 Compare January 16, 2024 13:14

ch1bo removed their assignment Jan 17, 2024

ffakenz force-pushed the head-explorer-backend branch from 70f4afc to 6ba9da4 Compare January 18, 2024 07:23

ch1bo force-pushed the master branch 2 times, most recently from 46a4f1a to aa19397 Compare January 18, 2024 15:56

ffakenz and others added 26 commits January 29, 2024 17:28

Improve README.md spelling

b221cb8

Co-authored-by: Sasha Bogicevic <Sasha.Bogicevic@iohk.io>

Remove not needed endpoints

f3e3a0a

As we are not serving any files yet. Co-authored-by: Sebastian Nagel <ch1bo@users.noreply.github.com>

Remove unnecessary handle from withChainObserver and withHydraExplore…

92bfead

…r helpers Co-authored-by: Sebastian Nagel <ch1bo@users.noreply.github.com>

Update README

155fad5

Remove unimplemented endpoint from openapi spec

9de27ed

Rename test module to match where the actual code is implemented

8590af7

Improve explorer spec to only assert on headers that we care about

41b6cf7

Update explorer api spec to specify required fields

de9e943

Use eitherDecode to print why the responded bytes where not a proper …

d9d9853

…JSON value

Refactor HydraExplorerHandle as an easy access to the API (= a client)

bdb28b8

And assert REST API behavior rather than std out logs.

Remove unused import from chain observer e2e spec

4f0b0eb

Replace PartyCommit by UTxO type

291c792

Increase the time dealy to wait for the explorer http server to be li…

c49c38a

…stening on port Because it was flaky as it was defined.

Draft property spec for aggregate head observations

24fde0f

Added arbitrary instances for head observations.

Refactor aggregation to work over OnChainTx type instead of HeadObser…

6740648

…vation

Add tracking of contestation deadline as part of explorer's head states

a9e4809

Add head state haddock

dca46e5

Remove unsupported attribute propertyNames by swagger

44e3d76

Minor fix after rebasing

9ce5965

Use 5 ADA for internal wallets

ea0aa8e

Remove misplaced comment

35a9479

Refactor http server to use servant

0f3ba3b

Fix explorer e2e specs by increasing the amount of fuel provided to e…

3e7b2d2

…ach party member

Minor formatting changes

594558b

ch1bo force-pushed the head-explorer-backend branch from 11d904f to 594558b Compare January 29, 2024 16:28

ffakenz merged commit b5ef1fa into master Jan 29, 2024
21 checks passed

ffakenz deleted the head-explorer-backend branch January 29, 2024 18:19

ch1bo linked an issue Mar 8, 2024 that may be closed by this pull request

Build and deploy a Hydra heads explorer #696

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a hydra-explorer executable that can track all heads on-chain #1235

Create a hydra-explorer executable that can track all heads on-chain #1235

ffakenz commented Jan 5, 2024 •

edited

Loading

github-actions bot commented Jan 5, 2024 •

edited

Loading

github-actions bot commented Jan 5, 2024 •

edited

Loading

Create a hydra-explorer executable that can track all heads on-chain #1235

Create a hydra-explorer executable that can track all heads on-chain #1235

Conversation

ffakenz commented Jan 5, 2024 • edited Loading

What

How

github-actions bot commented Jan 5, 2024 • edited Loading

Transactions Costs

Script summary

Cost of Init Transaction

Cost of Commit Transaction

Cost of CollectCom Transaction

Cost of Close Transaction

Cost of Contest Transaction

Cost of Abort Transaction

Cost of FanOut Transaction

End-To-End Benchmark Results

Baseline Scenario

Baseline Scenario

github-actions bot commented Jan 5, 2024 • edited Loading

Test Results

ffakenz commented Jan 5, 2024 •

edited

Loading

github-actions bot commented Jan 5, 2024 •

edited

Loading

github-actions bot commented Jan 5, 2024 •

edited

Loading