A Concurrent Specification for POSIX File Systems Gian Ntzik, Pedro da Rocha Pinto and Philippa Gardner Imperial College London gian.ntzik08@imperial.ac.uk pedro.da-rocha-pinto09@imperial.ac.uk p.gardner@imperial.ac.uk Dagstuhl 2016 1/22
POSIX: Portable Operating System Interface ◮ An operating system standard ◮ A large part is devoted to the file system > 20 base definitions, > 50 system interfaces ◮ It is written in English 2/22
Formal Sequential Specifications ◮ Based on first-order logic ◮ Z Notation: specification language with refinement to implementation ◮ Forest: Haskell DSL for type-safe file-system interactions These approaches lack modularity for client reasoning ◮ Based on separation logics ◮ SSL [ESOP’14], Fusion Logic [OOPSLA’15] These approaches give modular client reasoning. The second has been used to find bugs in implementations of e.g. rm − r . 3/22
Concurrency in the POSIX File System ◮ Understanding the intended POSIX concurrent behaviour is difficult ◮ It contains ambiguities Example: I/O atomicity I/O is intended to be atomic to ordinary files and pipes and FIFOs. This volume of POSIX.1-2008 does not specify behavior of concurrent writes to a file from multiple processes. Applications should use some form of concurrency control. 4/22
Concurrency in the POSIX File System ◮ Understanding the intended POSIX concurrent behaviour is difficult ◮ It contains ambiguities ◮ Information is fragmented throughout the whole standard Example: unlink ( p ) atomically removes the file at path p . But resolving the path is not atomic. Any part of the path of a file could be changed in parallel to a call to unlink(), resulting in unspecified behavior. 5/22
Example: unlink ( / usr / bin / git ) The operation takes a sequence of atomic actions 0 tmp usr 1 2 share local .X0-lock bin 3 8 0101111011... 5 4 lib git 7 9 0110011011... File-system notation: FS [2 �→ S ] and FS (2) = S where S = { (share , 3) , (bin , 4) , (local , 5) } 6/22
Sequence of Atomic Actions unlink(/usr/bin/git) usr bin git FS 1 FS 1 FS 2 FS 2 FS 3 FS 4 Environment: mupltiple atomic updates Thread: single atomic read in path traversal Thread: single atomic update 7/22
Atomic Specification: atomic Hoare triple Atomic Hoare triples introduced in TaDA, encoded in Iris: ℂ P Q x ∈ X . � P ( x ) � C � Q ( x ) � A 8/22
Atomic Specification: refinement ℂ P Q � � C ⊑ A x ∈ X . P ( x ) , Q ( x ) 9/22
Sequence of Atomic Actions: refinement ℂ ... P 1 P 2 P 3 P 4 P n − 1 P n C ⊑ x 1 ∈ X 1 . � P 1 ( x 1 ) , P 2 ( x 1 ) � ; x 2 ∈ X 2 . � P 3 ( x 2 ) , P 4 ( x 2 ) � ; . . . ; x n − 1 ∈ X n − 1 . � P n − 1 ( x n − 1 ) , P n ( x n − 1 ) � A A A 10/22
Concurrent unlink Specification unlink ( p / a , ret ) ⊑ let r = resolve ( ι 0 , p ) in if ¬ iserr( r ) then ret = remlink( r , a ) else ret = r fi 11/22
Concurrent unlink Specification unlink ( p / a , ret ) ⊑ ∃ r . resolve ( ι 0 , p , r ); if ¬ iserr( r ) then ret = remlink( r , a ) else ret = r fi 11/22
Concurrent unlink Specification unlink ( p / a , ret ) ⊑ ∃ r . resolve ( ι 0 , p , r ); [ ¬ iserr( r )] ; remlink( r , a , ret ) ⊔ [iserr( r )] ; [ ret = r ] 11/22
Concurrent unlink Specification unlink ( p / a , ret ) ⊑ ∃ r . resolve ( ι 0 , p , r ); { true , ¬ iserr( r ) } ; remlink( r , a , ret ) ⊔ { true , iserr( r ) } ; { true , ret = r } 11/22
Concurrent unlink Specification unlink ( p / a , ret ) ⊑ ∃ r . resolve ( ι 0 , p , r ); { true , ¬ iserr( r ) } ; remlink( r , a , ret ) ⊔ { true , iserr( r ) } ; { true , ret = r } remlink( r , a , ret ) � � fs ( FS ) ∧ r ∈ dom( FS ) , � . FS ∈ FS . A ∃ S . fs ( FS [ r �→ S ]) ∧ FS ( r ) = S ∪ { ( a , − ) } = ⇒ ret = 0 � fs ( FS ) ∧ r ∈ dom( FS ) , � ⊓ FS ∈ FS . A fs ( FS ) ∧ { ( a , − ) } �∈ FS ( r ) = ⇒ ret = − 1 ∗ errno = ENOENT 11/22
Concurrent unlink Specification letrec resolve ( ι, p , ret ) � ∃ b , a ′ , p ′ . if p = a ′ / p ′ then � fs ( FS ) ∧ ι ∈ dom( FS ) , � ∃ ι ′ . A FS ∈ FS . fs ( FS ) ∧ (( a ′ , ι ′ ) ∈ FS ( ι ) ∧ b = 0) ; ∨ (( a ′ , − ) �∈ FS ( ι ) ∧ b = − 1) if b = 0 then resolve ( ι ′ , p ′ , ret ) else . . . fi else . . . 12/22
Concurrent rename Specification rename ( p / a , p ′ / b , ret ) ⊑ ∃ r 1 , r 2 . resolve ( ι root , p , r 1 ) � resolve ( ι root , p ′ , r 2 ); if ¬ iserr( r 1 ) ∧ ¬ iserr( r 2 ) then // success cases: rename file ( r 1 , r 2 , a , b ) ⊓ rename dir ( r 1 , r 2 , a , b ) // error cases: ⊓ err file and dir ( r 1 , r 2 , a , b ) ⊓ err target notempty dir ( r 1 , r 2 , b ) ⊓ err source ancestor ( r 1 , r 2 ) ⊓ err dir and file ( r 1 , r 2 , a , b ) else if iserr( r 1 ) ∧ ¬ iserr( r 2 ) then // error in resolve of source [ ret = − 1 ∗ errno = r 1 ] else if ¬ iserr( r 1 ) ∧ iserr( r 2 ) then // error in resolve of target [ ret = − 1 ∗ errno = r 2 ] else if iserr( r 1 ) ∧ iserr( r 2 ) then // error in both resolves; either error code is set [ ret = − 1 ∗ errno ∈ { r 1 , r 2 } ] fi 13/22
Concurrent rename Specification // rename file, if target exists it must be a file and it is replaced let rename file ( r 1 , r 2 , a , b ) � ∃ ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS [ r 1 �→ S ][ r 2 �→ ( FS ( r 2 ) \ { ( b , − ) } ) ∪ { ( b , ι a ) } ]) ∧ FS ( r 1 ) = S ∪ { ( a , ι a } A ∧ isfile( FS , ι a ) ∧ ( ∃ ι b . ( b , ι b ) ∈ FS ( r 2 ) = ⇒ isfile( FS , ι b )) = ⇒ ret = 0 14/22
Concurrent rename Specification // rename file, if target exists it must be a file and it is replaced let rename file ( r 1 , r 2 , a , b ) � ∃ ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS [ r 1 �→ S ][ r 2 �→ ( FS ( r 2 ) \ { ( b , − ) } ) ∪ { ( b , ι a ) } ]) ∧ FS ( r 1 ) = S ∪ { ( a , ι a } A ∧ isfile( FS , ι a ) ∧ ( ∃ ι b . ( b , ι b ) ∈ FS ( r 2 ) = ⇒ isfile( FS , ι b )) = ⇒ ret = 0 // rename directory, if target exists it must be an empty directory and it is replaced let rename dir ( r 1 , r 2 , a , b ) � ∃ S , ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS [ r 1 �→ S ][ r 2 �→ ( FS ( r 2 ) \ { ( b , − ) } ) ∪ { ( b , ι a ) } ]) ∧ FS ( r 1 ) = S ∪ { a , ι a } ∧ isdir( FS , ι a ) A ∧ r 2 �∈ descendants( FS , r 1 ) ∧ ( ∃ ι b . ( b , ι b ) ∈ FS ( r 2 ) = ⇒ isempdir( FS , ι b )) = ⇒ ret = 0 14/22
Concurrent rename Specification // rename file, if target exists it must be a file and it is replaced let rename file ( r 1 , r 2 , a , b ) � ∃ ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS [ r 1 �→ S ][ r 2 �→ ( FS ( r 2 ) \ { ( b , − ) } ) ∪ { ( b , ι a ) } ]) ∧ FS ( r 1 ) = S ∪ { ( a , ι a } A ∧ isfile( FS , ι a ) ∧ ( ∃ ι b . ( b , ι b ) ∈ FS ( r 2 ) = ⇒ isfile( FS , ι b )) = ⇒ ret = 0 // rename directory, if target exists it must be an empty directory and it is replaced let rename dir ( r 1 , r 2 , a , b ) � ∃ S , ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS [ r 1 �→ S ][ r 2 �→ ( FS ( r 2 ) \ { ( b , − ) } ) ∪ { ( b , ι a ) } ]) ∧ FS ( r 1 ) = S ∪ { a , ι a } ∧ isdir( FS , ι a ) A ∧ r 2 �∈ descendants( FS , r 1 ) ∧ ( ∃ ι b . ( b , ι b ) ∈ FS ( r 2 ) = ⇒ isempdir( FS , ι b )) = ⇒ ret = 0 // error: source is file, target is an existing directory let err file and dir ( r 1 , r 2 , a , b ) � ∃ ι a , ι b . � fs( FS ) ∧ r 1 , r 2 ∈ dom( FS ) , � FS ∈ FS . fs( FS ) ∧ ( a , ι a ) ∈ FS ( r 1 ) ∧ isfile( FS , ι a ) ∧ ( b , ι b ) ∈ FS ( r 2 ) ∧ isdir( FS , ι b ) A = ⇒ ret = − 1 ∗ errno = EISDIR 14/22
Client Reasoning ◮ Lock Files ◮ Named Pipes ◮ Advisory record locks (on-going) 15/22
Example: lock-file /tmp/ .X0-lock is locked 0 tmp usr 1 2 share .X0-lock local bin 3 8 5 4 lib git 7 0110011011... 9 16/22
Lock Files ◮ lock ( path ): atomically create a non-existing lock file at path ◮ unlock ( path ): remove the lock file identified by path ◮ Implemented similarly to spin locks ◮ open ( path , O CREAT | O EXCL ) to try to lock ◮ unlink to unlock 17/22
Lock-file Implementation letrec lock ( path ) � let fd = open ( path , O CREAT | O EXCL ); if fd = − 1 then lock ( path ) else close ( fd ) fi let unlock ( path ) � unlink ( path ) 18/22
Lock-file Specification: a first try ◮ We want clients to observe lock and unlock operations as single atomic actions � � lock ( path ) ⊑ A v ∈ { 0 , 1 } . Lock( path , v ) , Lock( path , 1) ∗ v = 0 � � unlock ( path ) ⊑ Lock( path , 1) , Lock( path , 0) where Lock( path , v ) is an abstract predicate. ◮ This specification cannot be guaranteed only by the module ◮ The file system is globally shared ◮ The module cannot enforce ownership of the path ◮ The module and the environment must agree a priori on a protocol for the use of path 19/22
Lock-file Protocol Agreement We want the module and the environment to agree on a boundary 0 tmp usr LF 1 2 share .X0-lock local bin 3 8 5 4 lib git 7 0110011011... 9 20/22
Recommend
More recommend
