Transaction Support in Windows Transaction Support in Windows NTFS NTFS Surendra Verma Surendra Verma Development Manager Development Manager Windows File Systems Windows File Systems Microsoft Microsoft
Transactional NTFS (TxF) Transactional NTFS (TxF) _ Adds transaction support for all NTFS file Adds transaction support for all NTFS file _ operations: operations: Full Atomicity, Consistency, Isolation, Durability Full Atomicity, Consistency, Isolation, Durability Allows arbitrary number of file system Allows arbitrary number of file system operations to be treated as an atomic unit operations to be treated as an atomic unit Reads, writes, file creations, deletions, renames, Reads, writes, file creations, deletions, renames, security, object-id, named-streams, quota etc. security, object-id, named-streams, quota etc.
Semantics - isolation Semantics - isolation Committed Read without blocking Reader for Writers _ Transaction 1: Transaction 1: _ Transaction 2: Transaction 2: _ _ File 1 File 1 File 1 File 1 File 2 <- New File File 2 <- New File File 3 File 3 File 3 -< Deleted File File 3 -< Deleted File Transactions don’t see changes made by other transactions Same for Non-Transactions Contemplating Dirty Reader as an isolation level
Semantics - locking Semantics - locking _ File is the unit of locking File is the unit of locking _ _ File locked for update for the duration of the File locked for update for the duration of the _ transaction transaction _ Other handles in the same transaction Other handles in the same transaction _ allowed to update allowed to update _ Can be read concurrently (& consistently) by Can be read concurrently (& consistently) by _ the others the others
Demo _ Demo _
Volumes and RMs Volumes and RMs _ Each volume comes with a Transactional Each volume comes with a Transactional Resource Resource _ Manager (RM) by default (RM) by default Manager _ Its log is resident on the volume. Recovery Its log is resident on the volume. Recovery _ automatic. Admin-free. automatic. Admin-free. _ Many Many secondary secondary RMs may be created in various RMs may be created in various _ places within the volume places within the volume _ Their logs can be anywhere on the machine Their logs can be anywhere on the machine _ _ Their admin/recovery is user-controlled via APIs Their admin/recovery is user-controlled via APIs _ _ Designed to be embedded in other Designed to be embedded in other _ stores/applications stores/applications
Logging Modes Logging Modes _ Undo-Only Undo-Only logging: Minimizes logging and logging: Minimizes logging and _ supports ACID semantics supports ACID semantics _ Redo-Undo Redo-Undo logging: logging: “ “redo redo” ” is logged as well is logged as well _ => log contains complete complete history of history of => log contains changes changes _ Allows playback from a backup to achieve Allows playback from a backup to achieve _ consistency at a chosen point in time consistency at a chosen point in time _ Logging Mode can be set for secondary RMs Logging Mode can be set for secondary RMs _ and toggled live and toggled live
Implementation Details Implementation Details
Other Resources like SQL DTC WinFX APIs Win32 Log APIs Win32 I/O OLE APIs APIs Win32 Kernel Mode TM Kernel Mode APIs Trans Mgr Transactional Kernel Kernel Mode Mode Log NTFS Log APIs APIs Common Log Manager IRP Based
TxF Recovery TxF Recovery TxF builds Two types of content treated differently. builds Two types of content treated differently. TxF _ Metadata Metadata – – Names, attributes, security etc. Names, attributes, security etc. _ upon Ntfs recovery. upon Ntfs recovery. _ Data Data – – _ Built from scratch. Built from scratch. _ Both use Write Ahead Logging Technique. Both use Write Ahead Logging Technique. _
Namespace Isolation Namespace Isolation _ Main-memory balanced binary trees used. Main-memory balanced binary trees used. _
TxF Data Recovery TxF Data Recovery TOPS Undo Page Stream TxF Log Write Log Undo record Image of Page
TxF Data Recovery TxF Data Recovery _ For undo only logging mode For undo only logging mode – – files flushed files flushed _ on commit. on commit. _ TOPS stream pages and log written TOPS stream pages and log written _ independently of each other. independently of each other. _ A page changed multiple times in a A page changed multiple times in a _ transaction gets logged only once. transaction gets logged only once.
Questions _ Questions _
Transaction Management (KTM) Transaction Management (KTM) _ Coordinates commit/abort processing Coordinates commit/abort processing _ between the various actors: between the various actors: Resource Managers (RMs) (eg, SQL, TxF) Resource Managers (RMs) (eg, SQL, TxF) Applications Applications _ Persistently maintains outcome of Persistently maintains outcome of _ transactions using the common-log transactions using the common-log _ Lives in the kernel with Win32 and kernel Lives in the kernel with Win32 and kernel _ interfaces interfaces
Why Common Logging Why Common Logging Group log writes from multiple clients into a Group log writes from multiple clients into a single physical Disk I/O single physical Disk I/O Single logical view log for tightly coupled Single logical view log for tightly coupled but distinct resources but distinct resources Ease of configuration, archival, and media Ease of configuration, archival, and media recovery, and administration recovery, and administration Single paradigm for high-bandwidth logging Single paradigm for high-bandwidth logging on Windows on Windows
Common Logging Common Logging Multiple clients sharing a single logical log Multiple clients sharing a single logical log stream stream Each client has exclusive use of a virtual Each client has exclusive use of a virtual log stream log stream Common log manager multiplexes Common log manager multiplexes multiple client streams to single logical log multiple client streams to single logical log stream stream Multiplexing separated from I/O Multiplexing separated from I/O
Example: two file updates Example: two file updates _ Program writes to file1, then to file2 Program writes to file1, then to file2 _ _ System/application crashes System/application crashes _ _ Are the updates on disk? Are the updates on disk? _ _ Both on disk? Some? None? Both on disk? Some? None? _ _ Do others see updates as they occur? Do others see updates as they occur? _ _ What if the system showed the previous What if the system showed the previous _ (consistent) state until app ready to expose them? (consistent) state until app ready to expose them? _ Same issues if only Same issues if only one one file is involved! file is involved! _
Scenario: Update a web-site Scenario: Update a web-site _ Hide temporary inconsistencies Hide temporary inconsistencies _ _ System handles data recovery on app failure System handles data recovery on app failure _ or system crash or system crash _ System guarantees that updates survive System guarantees that updates survive _ crash once committed crash once committed _ On high-end, archive transaction logs for On high-end, archive transaction logs for _ shipping or media recovery shipping or media recovery
Scenario: Remote file Copy Scenario: Remote file Copy _ Reliable copy of file over the network Reliable copy of file over the network _ _ Cheap, low-level message transfer Cheap, low-level message transfer _ coordinated with other transaction work. coordinated with other transaction work. _ Pass data between branch office and central Pass data between branch office and central _ office (financial institutions, retail) office (financial institutions, retail) _ Frequently mentioned scenario by our Frequently mentioned scenario by our _ customers customers
Document Management Document Management _ Files in the file-system, file-attributes in a Files in the file-system, file-attributes in a _ relational database relational database _ Transaction maintains consistency between Transaction maintains consistency between _ the two the two _ Makes it possible to integrate administration Makes it possible to integrate administration _ utilities between the two stores utilities between the two stores
Recommend
More recommend
