How a sectorID is defined, related to content?

The file I am studying is /sector-base/src/api/disk_backed_storage.rs in project rust-fil-proofs. And it looks the sectorID, no matter for staging or sealed, is randomly selected by the minor. Anyone could confirm?

Hi Steven,

The rust-fil-proofs SectorBuilder is initialized with a last_used_sector_id value (see here). This value ultimately makes its way to the SectorMetadataManager, which uses the value when provisioning new yet-to-be-sealed sectors (see here).

Note: This values originates from go-filecoin - specifically the last committed sector id from the miner actor associated with the SectorBuilder being initialized (see here).

When a piece is added, the SectorMetadataManager looks through the yet-to-be-sealed-sectors which it manages to see if any contains enough space for the piece (see here). If there is not sufficient space to store the piece, a new yet-to-be-sealed sector is provisioned using the value of the (now-incremented) sector_id_nonce (see here).

Let me know if this does not address your question sufficiently.

Regards,

Erin

Hi Erin,

What are you talking is for staged sectors, i.e. yet-to-be-sealed-sectors.

Regarding sealed sectors, after read some codes, I think it is using random alphabetic 32-byte string, generated by function rand_alpha_string, which is invoked by function new_sector_access, then, new_sealed_sector_access, seal, in the up-layer, SealerWorker is calling seal.

Would you please confirm this, or correct me? thanks a lot.

Regards,
Steven

Hi Steven,

What are you talking is for staged sectors, i.e. yet-to-be-sealed-sectors.

I understand your question now.

The “sector id” is a concept that exists in the FIL protocol. It is referenced in the commitSector message, for example. When a yet-to-be-sealed sector is provisioned, it is assigned a sector id (as I described in my previous message). When the yet-to-be-sealed sector is sealed, it retains that original sector id.

The code which you have linked to is used to create a new “sector access,” which is a not-very-well-thought out concept. Today, it’s just a randomly-generated file path (rand_alpha_string) which tells the Rust code where it can locate the various sealed and yet-to-be-sealed sector files on disk. The sector access values are an implementation detail and are not visible in the FIL protocol.

When we want to seal the yet-to-be-sealed sector, we create a new sector access (a file path). That sector access is provided to the crate::api::internal::seal function along with the sector access of the yet-to-be-sealed sector. After sealing has successfully completed, the sealed sector can be found at the SealedSectorMetadata.sector_access path.

Whew. That was a bit long-winded, but I hope it helps. If not, feel free to ask for further clarification.

Regards,

Erin

Really appreciate, Erin.

It’s quite clear to me now. Thanks very much for the explanation in such a detail. I think I could draw the conclusion based on the current implementation: the sector_id, which is used in the protocol is not relevant to content (cid) or replica_id, but determined by the local miner, with a designed mechanism in code (incremental somehow) as you mentioned; in addition, the access of a sector is randomly assigned 32-byte alphabetic string.

With the currently design, I would think:

  1. The network does not care how the sector_id is defined. It could be changed later for new sectors without any compatibility issue;
  2. Since the disk access (filename) is randomly produced, there must be a mapping table (or file) locally to maintain it, and that is not on the chain.

Right? if so, regarding 2), where could I find the mapping table (or file). thanks again.