rocksdb/utilities/transactions
Levi Tamasi 1f96e652b3 Support using secondary indices with write-committed transactions (#13180)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13180

The patch adds initial support for secondary indices using write-committed transactions. Currently, only the `PutEntity` API is supported; other APIs like `Put` and `Delete` will be added separately. Applications can set up secondary indices using the new configuration option `TransactionDBOptions::secondary_indices`. When secondary indices are enabled, calling `PutEntity` via a (n explicit or implicit) transaction performs the following steps:
1) It retrieves the current value (if any) of the primary key using `GetEntityForUpdate`.
2) If there is an existing primary key-value, it removes any existing secondary index entries using `SingleDelete`. (Note: as a later optimization, we can avoid removing and recreating secondary index entries when neither the secondary key nor the value changes during an update.)
3) It invokes `UpdatePrimaryColumnValue` for all applicable `SecondaryIndex` objects, that is, those for which the primary column family matches the column family from the `PutEntity` call and for which the primary column appears in the new wide-column structure.
4) It writes the new primary key-value. Note that the values of the indexing columns might have been changed in step 3 above.
5) It builds the secondary key-value for each applicable secondary index using `GetSecondaryKeyPrefix` and `GetSecondaryValue`, and writes it to the appropriate secondary column family.

All the above operations are performed as part of the same transaction. The logic uses `SavePoint`s to roll back any earlier operations related to a primary key if a subsequent step fails.

Implementation-wise, the code uses a mixin template `SecondaryIndexMixin` that can inherit from any kind of transaction and use the write APIs and concurrency control mechanisms of the base class to implement the index maintenance logic. The mixin will enable us to later extend secondary indices to optimistic or write-prepared/write-unprepared pessimistic transactions as well.

Reviewed By: jowlyzhang

Differential Revision: D66672931

fbshipit-source-id: cdf6ef9c40dec46d928156bad0a3cc546aa8b887
2024-12-05 19:05:56 -08:00
..
lock Fix deprecated use of 0/NULL in internal_repo_rocksdb/repo/util/xxhash.h + 5 2024-04-01 21:20:51 -07:00
optimistic_transaction.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
optimistic_transaction.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
optimistic_transaction_db_impl.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
optimistic_transaction_db_impl.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
optimistic_transaction_test.cc Add a new API Transaction::GetCoalescingIterator (#13128) 2024-11-08 14:14:39 -08:00
pessimistic_transaction.cc Introduce a transaction option to skip memtable write during commit (#13144) 2024-12-05 15:00:17 -08:00
pessimistic_transaction.h Introduce a transaction option to skip memtable write during commit (#13144) 2024-12-05 15:00:17 -08:00
pessimistic_transaction_db.cc Support using secondary indices with write-committed transactions (#13180) 2024-12-05 19:05:56 -08:00
pessimistic_transaction_db.h Make transaction name conflict check more robust (#12895) 2024-07-30 12:31:02 -07:00
snapshot_checker.cc Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
timestamped_snapshot_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
transaction_base.cc Add a new API Transaction::GetCoalescingIterator (#13128) 2024-11-08 14:14:39 -08:00
transaction_base.h Support using secondary indices with write-committed transactions (#13180) 2024-12-05 19:05:56 -08:00
transaction_db_mutex_impl.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
transaction_db_mutex_impl.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
transaction_test.cc Support using secondary indices with write-committed transactions (#13180) 2024-12-05 19:05:56 -08:00
transaction_test.h Rename, deprecate LogFile and VectorLogPtr (#12695) 2024-05-28 09:24:49 -07:00
transaction_util.cc Add an option to toggle timestamp based validation for the whole DB (#12857) 2024-07-29 13:54:37 -07:00
transaction_util.h Add an option to toggle timestamp based validation for the whole DB (#12857) 2024-07-29 13:54:37 -07:00
write_committed_transaction_ts_test.cc Fix write committed transactions replay when UDT setting toggles (#13121) 2024-11-06 17:32:03 -08:00
write_prepared_transaction_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
write_prepared_txn.cc Add a new API Transaction::GetCoalescingIterator (#13128) 2024-11-08 14:14:39 -08:00
write_prepared_txn.h Add a new API Transaction::GetCoalescingIterator (#13128) 2024-11-08 14:14:39 -08:00
write_prepared_txn_db.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_prepared_txn_db.h Properly disable MultiCFIterator in WritePrepared/UnPreparedTxnDBs (#12883) 2024-07-24 16:50:12 -07:00
write_unprepared_transaction_test.cc Refactor WriteUnpreparedStressTest to be a unit test (#11424) 2023-05-22 12:31:52 -07:00
write_unprepared_txn.cc Do not add unprep_seqs when WriteImpl() fails in unprepared txn (#12927) 2024-08-15 09:16:29 -07:00
write_unprepared_txn.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
write_unprepared_txn_db.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_unprepared_txn_db.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00