Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor apply_delta for Storage Maps #758

Merged
merged 5 commits into from
Jun 24, 2024
Merged

Conversation

Dominik1999
Copy link
Collaborator

@Dominik1999 Dominik1999 commented Jun 18, 2024

Closes #751

Refactoring apply_delta for StorageSlots and StorageMaps

  • AccountStorageDelta.cleared_items and AccountStorageDelta.updated_items only contain updates to value slots.
  • AccountStorageDelta.updated_maps only contains updates to storage maps.
  1. This PR introduces a new condition for on_account_storage_after_set_item. If the new value is to be applied to a StorageSlot whose index already has a maps_update, the value is NOT added as slot_update.

    That means the roots of StorageMaps do not end up in cleared_items or updated_items, because

    • in the kernel, the procedure fails if the index doesn't point to a slot of type map
    • in the kernel, maps update events happen in MASM before slot updates
    • there cannot be a new root without a change in the tree (~ entry in maps_update)

    Note: We need to ignore the StorageSlot update when it is triggered by a map update in the Host, because we cannot validate it. At this stage (when the event is being processed in the Host) we don't have access to the whole storage map.

  2. There is also a set_map_item and get_map_item in AccountStorage now replicating the MASM procedures.

@Dominik1999 Dominik1999 requested a review from bobbinth June 18, 2024 08:52
@Dominik1999 Dominik1999 force-pushed the dominik_refactor_maps_delta branch 5 times, most recently from edcda8d to 04a5a7b Compare June 18, 2024 09:56
Copy link
Contributor

@bobbinth bobbinth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you! I left some comments inline. The main one about potentially enforcing storage layout restrictions in MASM.

objects/src/accounts/storage/mod.rs Outdated Show resolved Hide resolved
objects/src/accounts/storage/mod.rs Outdated Show resolved Hide resolved
objects/src/accounts/storage/mod.rs Outdated Show resolved Hide resolved
miden-tx/src/host/mod.rs Outdated Show resolved Hide resolved
miden-tx/src/host/mod.rs Outdated Show resolved Hide resolved
miden-tx/src/host/mod.rs Outdated Show resolved Hide resolved
Comment on lines 371 to 387
// get old values to return
let old_map_root = storage_map.root();
let old_value = storage_map.get_value(&Digest::from(key));

// apply the delta
storage_map.insert(key.into(), value);

// update the root of the storage map in the corresponding storage slot
let new_root = storage_map.root().into();
let index = LeafIndex::new(index.into()).expect("index is u8 - index within range");
let slot_value = self.slots.insert(index, value);
Ok(slot_value)
self.slots.insert(index, new_root);

Ok((old_map_root.into(), old_value))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for returning both the old root and the old value? Could we get away with returning just the old value? If so, this could look something like this:

let old_value = storage_map.insert(key.into(), value);

// update the root of the storage map in the corresponding storage slot
let index = LeafIndex::new(index.into()).expect("index is u8 - index within range");
self.slots.insert(index, storage_map.root().into());

Ok(old_value)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We return both in the MASM procedure and we wanted to have set_map_item here as close to the MASM implementation as possible.

#! Sets a map item in the account storage. Panics if 
#! - the index for the map is out of bounds, means >255
#! - the slot item at index is not a map
#!
#! Stack: [index, KEY, VALUE]
#! Output: [OLD_MAP_ROOT, OLD_MAP_VALUE, 0]

but we can also change this in MASM?

objects/src/accounts/storage/map.rs Outdated Show resolved Hide resolved
Comment on lines 279 to 285
for &key in map_delta.cleared_leaves.iter() {
self.set_map_item(slot_idx, key, Word::default())?;
}

// pick the right storage map and apply delta
let storage_map =
self.maps.get_mut(&slot_idx).ok_or(AccountError::StorageMapNotFound(slot_idx))?;
storage_map.apply_delta(map_delta)?;
for &(key, value) in map_delta.updated_leaves.iter() {
self.set_map_item(slot_idx, key, value)?;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we bring back StorageMap::apply_delta() as mentioned, this could look something like:

let new_root = self.maps.get_mut(&index)
    .ok_or(AccountError::StorageMapNotFound(index))?
    .apply_delta(map_delta);
let index = LeafIndex::new(slot_idx.into()).expect("index is u8 - index within range");
self.slots.insert(index, new_root);

Copy link
Collaborator Author

@Dominik1999 Dominik1999 Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which means we don't need AccountStorage::set_map_item anymore, right?

Separately, we should also implement AccountStorage::set_map_item() and AccountStorage::get_map_item() methods to mimic the ones in the transaction kernel.

Originally, we wanted to have those methods. Or we just expose those methods to the user and client and don't need them to use ourselves.

@Dominik1999 Dominik1999 force-pushed the dominik_refactor_maps_delta branch 2 times, most recently from 01cbfbe to 641e89c Compare June 19, 2024 16:28
@Dominik1999
Copy link
Collaborator Author

@bobbinth I refactored the MASM code quite a bit. Now we have a new procedure, set_item_raw, that doesn't trigger any events.

Copy link
Contributor

@bobbinth bobbinth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you! I left some comments inline. Most of them are small and we can merge after they are addressed. There is one bigger comment, but it could be done in a future PR.

Also, let's update the changelog.

miden-lib/asm/miden/kernels/tx/account.masm Outdated Show resolved Hide resolved
miden-lib/asm/miden/kernels/tx/constants.masm Outdated Show resolved Hide resolved
miden-lib/asm/miden/kernels/tx/constants.masm Outdated Show resolved Hide resolved
miden-lib/asm/miden/kernels/tx/account.masm Outdated Show resolved Hide resolved
miden-lib/asm/kernels/transaction/api.masm Outdated Show resolved Hide resolved
miden-tx/src/error.rs Outdated Show resolved Hide resolved
objects/src/errors.rs Outdated Show resolved Hide resolved
objects/src/accounts/storage/mod.rs Outdated Show resolved Hide resolved
@Dominik1999 Dominik1999 force-pushed the dominik_refactor_maps_delta branch from 88512a9 to 2dd5ef8 Compare June 24, 2024 07:39
@Dominik1999 Dominik1999 merged commit 0bef1ca into next Jun 24, 2024
12 checks passed
@Dominik1999 Dominik1999 deleted the dominik_refactor_maps_delta branch June 24, 2024 07:49
bobbinth pushed a commit that referenced this pull request Jul 4, 2024
* refactor: decouple AccountStorageDelta.updated_items and AccountStorageDelta.updated_maps

* refactor set_map_item

* after review

* refactoring masm code

* after last feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants