Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AccountStorage refactor from Smt to sequential hash #811

Closed
wants to merge 46 commits into from

Conversation

phklive
Copy link
Contributor

@phklive phklive commented Jul 31, 2024

In this PR I propose to refactor the AccountStorage from Smt to a sequential hash.

Closes: #802

phklive and others added 30 commits July 18, 2024 11:10
@phklive phklive changed the title Phklive account storage refactor AccountStorage refactor from Smt to sequential hash Jul 31, 2024
@phklive
Copy link
Contributor Author

phklive commented Jul 31, 2024

This is a first pass at tackling this issue:

  • The refactor from Smt to sequential hash is mostly if not fully done on the AccountStorage
  • The refactor of the delta's is slightly more involved, I took a pass at it, the upper logic seems correct but I am not updating the inner values correctly (i.e storage slots are correctly updated, but maps are only replaced by new deltas and not updated using StorageMapDelta )
  • The tests will need a substantial refactor
  • The MASM part has not yet been implemented

@bobbinth, I would like to know if this seems like a general good route of if you would advise for another way of tackling this problem.

@phklive
Copy link
Contributor Author

phklive commented Jul 31, 2024

Also, these commits coming from the previous PR I built on which was merged in next: #763

SCR-20240731-taya

Why are they appearing here? Is there something I did wrong or a merge conflict?

Copy link
Contributor

@bobbinth bobbinth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Not a full review yet - but I left some comments inline.

Comment on lines +44 to +45
/// Returns a new instance of account storage initialized with the provided slots.
pub fn new(slots: &[StorageSlot]) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to pass in a slice instead of a Vec as a parameter here?

Comment on lines +65 to +75
/// Returns an item from the storage at the specified index.
///
/// If the item is not present in the storage, [crate::EMPTY_WORD] is returned.
pub fn get_map_item(&self, index: u8, key: Word) -> Result<Word, AccountError> {
let storage_map = self.maps.get(&index).ok_or(AccountError::StorageMapNotFound(index))?;

Ok(storage_map.get_value(&Digest::from(key)))
}

/// Returns a reference to the Sparse Merkle Tree that backs the storage slots.
pub fn slots(&self) -> &SimpleSmt<STORAGE_TREE_DEPTH> {
&self.slots
}

/// Returns layout info for this storage.
pub fn layout(&self) -> &[StorageSlotType] {
&self.layout
}

/// Returns a commitment to the storage layout.
pub fn layout_commitment(&self) -> Digest {
layout_commitment(&self.layout)
}

/// Returns the storage maps for this storage.
pub fn maps(&self) -> &BTreeMap<u8, StorageMap> {
&self.maps
pub fn get_item(&self, index: u8) -> Digest {
match self.slots.get(index as usize) {
Some(storage_slot) => match storage_slot {
StorageSlot::Value(word) => word.into(),
StorageSlot::Map(map) => map.root(),
},
None => EMPTY_WORD.into(),
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments here:

  • get_item() should return a Word.
  • get_item() should panic if the index is out of bounds.
  • We also need to have get_map_item() (similar to how we had it before).

We need to have some accessors to get info about the storage. Specifically, we need to be able to learn:

  • How many slots are in the storage.
  • What types these slots have.

These could be done via separate methods or via a single layout() method (similar to what we had before).

Comment on lines 91 to +92
/// Updates the value of the storage slot at the specified index.
///
/// This method should be used only to update simple value slots. For updating values
/// in storage maps, please see [AccountStorage::set_map_item()].
///
/// # Errors
/// Returns an error if:
/// - The index specifies a reserved storage slot.
/// - The update tries to set a slot of type array.
/// - The update has a value arity different from 0.
pub fn set_item(&mut self, index: u8, value: Word) -> Result<Word, AccountError> {
// layout commitment slot cannot be updated
if index == Self::SLOT_LAYOUT_COMMITMENT_INDEX {
return Err(AccountError::StorageSlotIsReserved(index));
}
pub fn set_item(&mut self, index: u8, storage_slot: StorageSlot) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep the signature of this method as it was before. Basically, we want to keep mutators parallel to their counterparts in the MASM code.

Comment on lines -348 to -353
pub fn set_map_item(
&mut self,
index: u8,
key: Word,
value: Word,
) -> Result<(Word, Word), AccountError> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the previous comment: we should keep this method.

Comment on lines +35 to 39
impl Serializable for StorageSlot {
fn write_into<W: vm_core::utils::ByteWriter>(&self, target: &mut W) {
target.write_u16(self.into());
target.write(self);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this not lead to infinite recursion?

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StorageSlot {
Value(Word),
Map(StorageMap),
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may still need to keep StorageSlotType enum. For example, it would be used in AccountStorage::get_slot_type() or AccountStorage::layout().

We could also have methods like StorageSlotType::as_element() and StorageSlotType::as_word() to simplify conversion of storage into a vector of elements.

_ => false,
}
}
impl StorageSlot {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also probably need something like

  • StorageSlot::get_type() -> StorageSlotType
  • StorageSlot::to_word() -> Word. This would return the value for simple value slots, and the root for maps.

Comment on lines +110 to +132
match storage_slot {
StorageSlot::Value(word) => {
for element in word {
elements.push(*element)
}
for _ in 0..4 {
elements.push(ZERO)
}
},
StorageSlot::Map(map) => {
let smt_root = map.root();

for element in smt_root.as_elements() {
elements.push(*element)
}

elements.push(ONE);

for _ in 0..3 {
elements.push(ZERO)
}
},
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have methods mentioned in the previous comment, this could be simplified quite a bit. For example:

elements.extend_from_slice(&slot.to_word());
elements.extend_from_slice(&slot.get_type().as_word());

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to modify the deltas as a part of this PR. The current delta structs should be sufficient for the new account storage structure and we'll do a more general refactoring as a part of #798.

@bobbinth
Copy link
Contributor

bobbinth commented Aug 1, 2024

Why are they appearing here? Is there something I did wrong or a merge conflict?

Hmm - not sure. I don't mind having them here as they'll go away once we do "squash and merge", but also if there is an easy way to git rid of them here, that would be even better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants