Discussed:
Separation of Concerns:
- Commands like MGET/MSET - Key used across shards,
- Similar for Qwatch - when we are watching a pattern, the pattern can exist across multiple shards.
- Transactional commands like MGET/MSET - We need an abstraction layer outside shards that can manage the transaction across teh shards.
IOLayer - Should it wait for all shards to complete the tasks, proceed only after all shards have responded.
Keep store dumb. All ops on store as atomic as possible.
. Add complexity to IOLayer / Coordinator in future if optimisation are needed in future.
Next step task
- Remove locks and provide atomic operations.
- Address review comments and merge Store abstraction refactor Yash
- Watch to move out of store / Or implement a scan operator in store Jyotinder
- Limit Store operations to Put / Get / Delete & Scan Pratik
- We are using unsafe pointer, should we move to empty struct ot interface. Move to generics. Ashwin Kulkarni
- Removing all Mutex from Store after the shards and channels are created for multi threads. soumya
- Skeleton for multithreaded mode Yash
- Extract Coordinator from IO Tasks. Gaurav
- Qwatch: each io thread launches a qwatch command, this fans out to every shard. Each shard now maintains records for which io threads are listening to which queries . Jyotinder
- moar thoughts - the watchlist can be maintained at the said coordinator level
- Fanout commands that require consistency soumya