introduce Narfuse for fun #89
Conversation
This brings the Header structure a bit closer to the nar/ls Node structure. Sometimes it's useful to first build the file index, and access the file contents in a second step.
This is a bit of an experiment that tests the nar interfaces a bit.
There was a problem hiding this comment.
Nice work! :-D
I still think this is a bit too experimental, at least with the current set of TODOs - I marked this as draft.
Some of the problems you're currently having with the NAR format (index creation, efficient seeking, reading primitives for one file, …) would be a non-issue if this would set up on top the Store introduced in #86.
There's already code importing a NAR file (https://github.com/nix-community/go-nix/pull/86/files#diff-a7b13b4b9c9587f5f4cf67303e9221ba55678af016682fb445104ea15a5cab18R20), and https://github.com/nix-community/go-nix/pull/86/files#diff-8e2a166376955f9c576d3ad9fb4f0c7cd716dbe787ccdd6cba37d285df0f360dR21 would allow seeking individual files inside the NAR. What do you think about setting up on top of that?
| // Contents seeks rs to the header Offset and returns a LimitReader that | ||
| // gives you access to the content of the file. | ||
| func (h *Header) Contents(rs io.ReadSeeker) (io.Reader, error) { | ||
| // Start reader rs at Offset | ||
| _, err := rs.Seek(h.Offset, io.SeekStart) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| // Only allow to read up to Size bytes | ||
| return io.LimitReader(rs, h.Size), nil | ||
| } | ||
|
|
There was a problem hiding this comment.
This doesn't sound like it belongs into Header tbh…
I banged this together until it worked. And it works! The code is quite garbage right now, but it was fun, and I learned a few things.
go-fuse wants to build the filesystem structure when mounted. Then later, when individual files get accessed, there is a callback to read the content. There are multiple ways to handle this, but I wanted to avoid keeping the whole NAR in memory, so I added the Offset to the nar.Header. This allows the first pass to read the NAR file for the headers and build the filesystem layout. Then later, when individual files get read, re-open the NAR file, seek at Offset and read up to Size.
Once I did this, I found that the nar/ls.Node structure also has an offset. Maybe better to use that, but right now we're missing a method that converse the NAR to a nar/ls structure. That's something I might do in another iteration.