Background Dist r ibut ed f ile syst em (DFS) a dist r ibut ed - - PDF document

SLIDE 1

1

• 1

20: Dist ribut ed File Syst ems

Last Modif ied: 7/ 3/ 2004 1:51:15 PM

• 2

Background

Dist r ibut ed f ile syst em (DFS) – a dist r ibut ed

implement at ion of t he classical t ime-shar ing model

• f a f ile syst em, wher e mult iple user s shar e f iles

and st or age r esour ces.

A DFS manages set of disper sed st or age devices Over all st or age space managed by a DFS is

composed of dif f er ent , r emot ely locat ed, smaller st or age spaces.

Ther e is usually a cor r espondence bet ween

const it uent st or age spaces and set s of f iles.

• 3

DFS St ruct ure

Service – sof t ware ent it y running on one or

more machines and providing a part icular t ype of f unct ion t o a priori unknown client s.

Server – ser vice sof t war e r unning on a

single machine.

Client – process t hat can invoke a service

using a set of operat ions t hat f orms it s client int erf ace.

A client int er f ace f or a f ile ser vice is f or med

by a set of pr imit ive f ile oper at ions (cr eat e, delet e, read, writ e).

• 4

Naming and Transparency

Naming – mapping bet ween logical and physical

• bj ect s.

Mult ilevel mapping – abst ract ion of a f ile t hat

hides t he det ails of how and wher e on t he disk t he f ile is act ually st ored.

A t r anspar ent DFS hides t he locat ion wher e in t he

net wor k t he f ile is st or ed.

For a f ile being r eplicat ed in sever al sit es, t he

mapping r et ur ns a set of t he locat ions of t his f ile’s r eplicas; bot h t he exist ence of mult iple copies and t heir locat ion ar e hidden.

• 5

Naming St ruct ures

Locat ion t ransparency – f ile name does not r eveal

t he f ile’s physical st or age locat ion.

File name st ill denot es a specif ic, alt hough hidden, set of

physical disk blocks.

Convenient way t o share dat a. Can expose correspondence bet ween component unit s and

machines. Locat ion independence – f ile name does not need

t o be changed when t he f ile’s physical st or age locat ion changes.

Bet t er f ile abst ract ion. P

romot es sharing t he st orage space it self .

Separat es t he naming hierarchy f rom t he st orage-

devices hierarchy.

• 6

Naming Schemes —Thr ee Main Approaches

1) Files named by combinat ion of t heir host name

and local name; guar ant ees a unique syst emwide name.

2) At t ach r emot e dir ect or ies t o local dir ect or ies,

giving t he appear ance of a coher ent dir ect or y t r ee; only pr eviously mount ed r emot e dir ect or ies can be accessed t r anspar ent ly.

3)Tot al int egr at ion of t he component f ile syst ems.

A single global name st ruct ure spans all t he f iles in t he

syst em.

I f a server is unavailable, some arbit rary set of

direct ories on dif f erent machines also becomes unavailable.

SLIDE 2

2

• 7

Client Caching

Reduce net wor k t r af f ic by r et aining r ecent ly

accessed disk blocks in a cache, so t hat r epeat ed accesses t o t he same inf or mat ion can be handled locally.

I f needed dat a not already cached, a copy of dat a is

brought f rom t he server t o t he user.

Accesses are perf ormed on t he cached copy. Files ident if ied wit h one mast er copy residing at t he

server machine, but copies of (part s of ) t he f ile are scat t ered in dif f erent caches.

Cache- consist ency problem – keeping t he cached copies

consist ent wit h t he mast er f ile

Where is cache? Client disk, Client memory, Bot h?

• 8

Benef it s of Caching

Fast Access

many remot e accesses handled ef f icient ly by t he local

cache; most remot e accesses will be served as f ast as local ones. Decr ease Load on Ser ver s

Servers are cont ract ed only occasionally in caching

(rat her t han f or each access).

Enhances pot ent ial f or scalabilit y.

Ef f icient Use of Net wor k

Reduces net work t raf f ic. Tot al net work overhead in t ransmit t ing big chunks of

dat a (caching) is lower t han a series of responses t o specif ic request s (remot e- service).

• 9

P roblems of Caching

Cache Consist ency

Wit h f r equent wr it es, subst ant ial over head

incur r ed t o over come cache-consist ency pr oblem

Mor e lat er ..

New I nt er f ace

I n caching, t he lower int er machine int er f ace is

dif f er ent f or m t he upper user int er f ace.

I n r emot e-ser vice (no caching), t he

int er machine int er f ace mir r or s t he local user - f ile-syst em int er f ace.

• 10

Cache Updat e P

• licy

Write- t hrough – wr it e dat a t hr ough t o disk as

soon as t hey ar e placed on any cache. Reliable, but poor per f or mance.

Delayed- write – modif icat ions wr it t en t o t he

cache and t hen wr it t en t hr ough t o t he ser ver lat er . Wr it e accesses complet e quickly; some dat a may be over wr it t en bef or e t hey ar e wr it t en back, and so need never be wr it t en at all.

P

• or reliabilit y; unwrit t en dat a will be lost whenever a

user machine crashes.

Variat ion – scan cache at regular int ervals and f lush

blocks t hat have been modif ied since t he last scan.

Variat ion – wr it e

• on- close, writ es dat a back t o t he server

when t he f ile is closed. Best f or f iles t hat are open f or long periods and f requent ly modif ied.

• 11

Consist ency

I s locally cached copy of t he dat a consist ent wit h

t he mast er copy?

Client-init iat ed appr oach

Client init iat es a validit y check. Server checks whet her t he local dat a are consist ent wit h

t he mast er copy. Server-init iat ed appr oach

Server records, f or each client , t he (part s of ) f iles it

caches.

When server det ect s a pot ent ial inconsist ency, it must

react .

• 12

St at ef ul File Service

Mechanism.

Client opens a f ile. Server f et ches inf ormat ion about t he f ile f rom it s disk,

st ores it in it s memory, and gives t he client a connect ion ident if ier unique t o t he client and t he open f ile.

I dent if ier is used f or subsequent accesses unt il t he

session ends.

Server must reclaim t he main- memory space used by

client s who are no longer act ive. I ncr eased per f or mance.

Fewer disk accesses. St at ef ul server knows if a f ile was opened f or sequent ial

access and can t hus read ahead t he next blocks.

SLIDE 3

3

• 13

St at eless File Server

Avoids st at e inf ormat ion by making each

request self -cont ained.

Each request ident if ies t he f ile and

posit ion in t he f ile.

No need t o est ablish and t erminat e a

connect ion by open and close operat ions.

• 14

Dist inct ions Bet ween St at ef ul & St at eless Service

Failure Recovery.

A st at ef ul ser ver loses all it s volat ile st at e in a

crash.

• Rest ore st at e by recovery prot ocol based on a dialog

wit h client s, or abort operat ions t hat were underway when t he crash occurred.

• Server needs t o be aware of client f ailures in order

t o reclaim space allocat ed t o record t he st at e of crashed client processes (orphan det ect ion and eliminat ion).

Wit h st at eless ser ver , t he ef f ect s of ser ver

f ailur e sand r ecover y ar e almost unnot iceable. A newly r eincar nat ed ser ver can r espond t o a self-cont ained r equest wit hout any dif f icult y.

• 15

Dist inct ions (Cont .)

Penalt ies f or using t he r obust st at eless ser vice:

longer request messages slower request processing addit ional const raint s imposed on DFS design

Some envir onment s r equir e st at ef ul ser vice.

A server employing server - init iat ed cache validat ion

cannot provide st at eless service, since it maint ains a record of which f iles are cached by which client s.

UNI X use of f ile descript ors and implicit of f set s is

inherent ly st at ef ul; must maint ain t ables t o map t he f ile descript ors t o inodes, and st ore t he current of f set wit hin a f ile. (Can do on t he client side t hough)

• 16

File Replicat ion

Replicas of t he same f ile r eside on f ailur e-

independent machines.

I mpr oves availabilit y and can shor t en ser vice t ime. Naming scheme maps a r eplicat ed f ile name t o a

par t icular r eplica.

Exist ence of replicas should be invisible t o higher levels. Replicas must be dist inguished f rom one anot her by

dif f erent lower - level names. Updat es – r eplicas of a f ile denot e t he same

logical ent it y, and t hus an updat e t o any r eplica must be ref lect ed on all ot her replicas.

Demand r eplicat ion – r eading a nonlocal r eplica

causes it t o be cached locally, t her eby gener at ing a new nonpr imar y r eplica.

• 17

AFS – Andrew File Syst em

A dist r ibut ed comput ing envir onment under

development since 1983 at Car negie-Mellon Univer sit y.

Andr ew is highly scalable; t he syst em is t ar get ed

t o span over 5000 wor kst at ions.

Andr ew dist inguishes bet ween client machines

(wor kst at ions) and dedicat ed ser ver machines. Ser ver s and client s r un t he 4.2BSD UNI X OS and ar e int er connect ed by an int er net of LANs.

• 18

AFS (Cont .)

Client s ar e pr esent ed wit h a par t it ioned space of

f ile names: a local name space and a shared name space.

Servers collect ively are responsible f or t he

st or age and management of t he shar ed name space.

Dedicat ed servers, called Vice, present t he shared name

space t o t he client s as an homogeneous, ident ical, and locat ion t ransparent f ile hierarchy. The local name space is t he r oot f ile syst em of a

wor kst at ion, f r om which t he shar ed name space descends.

Workst at ions run t he Virt ue prot ocol t o communicat e

wit h Vice, and are required t o have local disks where t hey st ore t heir local name space.

SLIDE 4

4

• 19

AFS Shared Name Space

Andr ew’s volumes ar e small component unit s

associat ed wit h t he f iles of a single client .

A f id ident if ies a Vice f ile or dir ect or y. A f id is

96 bit s long and has t hr ee equal-lengt h component s:

volume number vnode number – index int o an array cont aining t he inodes

• f f iles in a single volume.

uniquif ier – allows reuse of vnode numbers, t hereby

keeping cert ain dat a st ruct ures, compact . Fids ar e locat ion t r anspar ent ; t her ef or e, f ile

movement s f r om ser ver t o ser ver do not invalidat e cached dir ect or y cont ent s.

Locat ion inf or mat ion is kept on a volume basis, and

t he inf or mat ion is r eplicat ed on each ser ver .

• 20

AFS File Operat ions

A key mechanism select ed f or r emot e f ile

• per at ions is whole f ile caching.

Opening a f ile causes it t o be cached, in it s ent iret y, on

t he local disk.

Reading and writ ing byt es of a f ile are done by t he kernel

wit hout Venus int ervent ion on t he cached copy.

Changes writ t en back t o server on close

Except ions t o t he caching policy ar e modif icat ions

t o dir ect or ies t hat ar e made dir ect ly on t he ser ver r esponsibilit y f or t hat dir ect or y

Ser ver s ar e st at ef ul

Servers hold callbacks on f iles client s are caching so can

not if y client if anot her one writ es back t he f ile wit h changes

• 21

NFS – Net work File Syst em

Complet ely dif f er ent design point f r om AFS I mplement ed by Sun in ear ly 80s Wr it e-t hr ough caching

Servers involved on every writ e I f t wo client s act ively writ ing, writ es will be int erleaved

semi- randomly as t hey reach t he server St at eless pr ot ocol

Each operat ion requires complet e inf ormat ion about t he

request

Operat ions must be idempot ent (ok t o repeat as many

t imes as you want )

• Ok f or r ead/ wr it e; r emove? May r emove wr ong ver sion of a

f ile if removed t hen recreat ed t hen remove is ret ried

Failed server j ust looks really slow t o client s

• 22

NFS (con’t)

Reads allowed t o proceed out of client

cache

Client periodically polls server t o see if f ile

has changed (been writ t en by someone else while t hey were reading it )

Hard t o reason about who will see a change

and when

• 23

Out t akes

• 24

Cache Locat ion – Disk vs. Main Memor y

Advant ages of disk caches

More reliable. Cached dat a kept on disk are st ill t here during recovery

and don’t need t o be f et ched again. Advant ages of main-memory caches:

P

ermit workst at ions t o be diskless.

Dat a can be accessed more quickly. P

erf ormance speedup in bigger memories.

Server caches (used t o speed up disk I / O) are in main

memory regardless of where user caches are locat ed; using main- memory caches on t he user machine permit s a single caching mechanism f or servers and users.

SLIDE 5

5

• 25

ANDREW I mplement at ion

Client pr ocesses ar e int er f aced t o a UNI X ker nel

wit h t he usual set of syst em calls.

Venus car r ies out pat h-name t r anslat ion

component by component .

The UNI X f ile syst em is used as a low-level

st or age syst em f or bot h ser ver s and client s. The client cache is a local dir ect or y on t he wor kst at ion’s disk.

Bot h Venus and ser ver pr ocesses access UNI X

f iles dir ect ly by t heir inodes t o avoid t he expensive pat h name- to-inode t r anslat ion r out ine.

• 26

ANDREW I mplement at ion (Cont .)

Venus manages t wo separ at e caches:

• ne f or st at us
• ne f or dat a

LRU algor it hm used t o keep each of t hem bounded

in size.

The st at us cache is kept in vir t ual memor y t o allow

r apid ser vicing of st at (f ile st at us r et ur ning) syst em calls.

The dat a cache is r esident on t he local disk, but

t he UNI X I / O buf f er ing mechanism does some caching of t he disk blocks in memor y t hat ar e t r anspar ent t o Venus.