This Week in Lotus - Week 22 - 2023

We have a work-in-progress PR that is looking to address the syncing issues storage providers has observed during rapid SnapDeals onboarding. We plan to finish up the PR next week and test that it fixes the issues people have observed. It will be back-ported to the current v1.23.1 release 🚆.

🆕 proof API´s The FilCrypto team has created numerous new API endpoints 💙 that the Lotus team can start to leverage to bring a lot of new desired behavior and fixes. First and foremost we are looking to improve some unsealing behavior:

• Implement new SnapDeals decodes API endpoint that will fix some issues around unsealing SnapDeals.
• Implement a “Split PC1” when unsealing. This will allow unsealing to be computed without moving the sealed sector to an unseal-worker. This will make it possible to unseal on lotus-workers that are not connected to the long-term storage where the sealed sector that is being unsealed is located.

We will report back when we have PRs available to test, but with these two improvements we hope to make unsealing both scalable and easy to perform 🚀

🍾 Unblocking retrieval bottlenecks

We have merged a PR this week that re-tunes the pieceReader code to allow for parallel access to unblock retrieval bottlenecks.

The pieceReader code was originally designed for sequential full-piece retrievals, but given that there are now a lot more random reads of data, this piece of code needed to be re-tuned to allow for random, parallel access of data.

It’s estimated that this will bring almost a 10x improvement in booster-bitswap retrievals. We will work with the Boost team to load test this PR, to see what the actual improvements are!

We will report back when we have some actual numbers before and after leveraging this code.

That’s it for the week! Have a great weekend! ☀️