How a sectorID is defined, related to content?

#1

The file I am studying is /sector-base/src/api/disk_backed_storage.rs in project rust-fil-proofs. And it looks the sectorID, no matter for staging or sealed, is randomly selected by the minor. Anyone could confirm?

#2

Hi Steven,

The rust-fil-proofs SectorBuilder is initialized with a last_used_sector_id value (see here). This value ultimately makes its way to the SectorMetadataManager, which uses the value when provisioning new yet-to-be-sealed sectors (see here).

Note: This values originates from go-filecoin - specifically the last committed sector id from the miner actor associated with the SectorBuilder being initialized (see here).

When a piece is added, the SectorMetadataManager looks through the yet-to-be-sealed-sectors which it manages to see if any contains enough space for the piece (see here). If there is not sufficient space to store the piece, a new yet-to-be-sealed sector is provisioned using the value of the (now-incremented) sector_id_nonce (see here).

Let me know if this does not address your question sufficiently.

Regards,

Erin

#3

Hi Erin,

What are you talking is for staged sectors, i.e. yet-to-be-sealed-sectors.

Regarding sealed sectors, after read some codes, I think it is using random alphabetic 32-byte string, generated by function rand_alpha_string, which is invoked by function new_sector_access, then, new_sealed_sector_access, seal, in the up-layer, SealerWorker is calling seal.

Would you please confirm this, or correct me? thanks a lot.

Regards,
Steven

#4

Hi Steven,

What are you talking is for staged sectors, i.e. yet-to-be-sealed-sectors.

I understand your question now.

The “sector id” is a concept that exists in the FIL protocol. It is referenced in the commitSector message, for example. When a yet-to-be-sealed sector is provisioned, it is assigned a sector id (as I described in my previous message). When the yet-to-be-sealed sector is sealed, it retains that original sector id.

The code which you have linked to is used to create a new “sector access,” which is a not-very-well-thought out concept. Today, it’s just a randomly-generated file path (rand_alpha_string) which tells the Rust code where it can locate the various sealed and yet-to-be-sealed sector files on disk. The sector access values are an implementation detail and are not visible in the FIL protocol.

When we want to seal the yet-to-be-sealed sector, we create a new sector access (a file path). That sector access is provided to the crate::api::internal::seal function along with the sector access of the yet-to-be-sealed sector. After sealing has successfully completed, the sealed sector can be found at the SealedSectorMetadata.sector_access path.

Whew. That was a bit long-winded, but I hope it helps. If not, feel free to ask for further clarification.

Regards,

Erin

#5

Really appreciate, Erin.

It’s quite clear to me now. Thanks very much for the explanation in such a detail. I think I could draw the conclusion based on the current implementation: the sector_id, which is used in the protocol is not relevant to content (cid) or replica_id, but determined by the local miner, with a designed mechanism in code (incremental somehow) as you mentioned; in addition, the access of a sector is randomly assigned 32-byte alphabetic string.

With the currently design, I would think:

  1. The network does not care how the sector_id is defined. It could be changed later for new sectors without any compatibility issue;
  2. Since the disk access (filename) is randomly produced, there must be a mapping table (or file) locally to maintain it, and that is not on the chain.

Right? if so, regarding 2), where could I find the mapping table (or file). thanks again.