changelog
play

Changelog Changes not seen in fjrst lecture: 19 March 2020: move - PowerPoint PPT Presentation

Mar 08, 2023 •131 likes •1.5k views

Changelog Changes not seen in fjrst lecture: 19 March 2020: move page usage slides later 19 March 2020: adjust PF counting exercise to specify addreses, not ofgsets 19 March 2020: Linux maps: correct shown mmap call for 0x400000 0 virtual

  1. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 16 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  2. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 16 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  3. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 16 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  4. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 16 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  5. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 16 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  6. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 17 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  7. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  8. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  9. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  10. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  11. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  12. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  13. mapped pages (read-only) update page table, retry now point to page PF handler: read in page fjrst read in page PF handler: no cached page page fault read from fjrst page? PF handler: fjnd cached page virtual pages mapped to fjle page fault read from second page? (could also prefjll entries…) initially — all invalid? fjle data on disk/SSD fjle data, cached in memory page table (part) 18

  14. shared mmap int fd = open("/tmp/somefile.dat", O_RDWR); MAP_SHARED, fd, 0); from /proc/PID/maps for this program: 7f93ad877000-7f93ad887000 rw-s 00000000 08:01 1839758 /tmp/somefile.dat 19 mmap(0, 64 * 1024, PROT_READ | PROT_WRITE,

  15. mapped pages (read/write, shared) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD write to page? update cached fjle data data on disk out of date eventually free memory… write update to disk 20

  16. mapped pages (read/write, shared) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD write to page? update cached fjle data data on disk out of date eventually free memory… write update to disk 20

  17. mapped pages (read/write, shared) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD write to page? update cached fjle data data on disk out of date eventually free memory… write update to disk 20

  18. mapped pages (read/write, shared) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD write to page? update cached fjle data data on disk out of date eventually free memory… write update to disk 20

  19. minor and major faults minor page fault page is already in memory (“page cache”) just fjll in page table entry major page fault page not already in memory (“page cache”) need to allocate space possibly need to read data from disk/etc. 21

  20. Linux: reporting minor/major faults Major (requiring I/O) page faults: 0 Exit status: 0 ... Swaps: 0 Involuntary context switches: 53 Voluntary context switches: 1423 Minor (reclaiming a frame) page faults: 230166 Average resident set size (kbytes): 0 $ /usr/bin/time --verbose some-command Maximum resident set size (kbytes): 749820 ... Elapsed (wall clock) time (h:mm:ss or m:ss): 0:19.57 Percent of CPU this job got: 94% System time (seconds): 0.35 User time (seconds): 18.15 Command being timed: "some-command" 22

  21. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 23 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  22. protection fault mapped pages (copy-on-write) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD reads like before write to second page? page table entry says read-only fault handler: make copy, update page table copies of fjle data, modifjed 24

  23. mapped pages (copy-on-write) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD reads like before write to second page? page table entry says read-only fault handler: make copy, update page table copies of fjle data, modifjed 24 protection fault

  24. protection fault mapped pages (copy-on-write) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD reads like before write to second page? page table entry says read-only fault handler: make copy, update page table copies of fjle data, modifjed 24

  25. protection fault mapped pages (copy-on-write) virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD reads like before write to second page? page table entry says read-only fault handler: make copy, update page table copies of fjle data, modifjed 24

  26. maps counting 4KB ( 0x1000 byte) pages 0-0x0FFFF map setup private (copy-on-write) bytes 0-0x3FFF and 0x5000-0x6FFF cached in memory program reads addresses 0x13800 – 0x15800 then, program overwrites addresses 0x14800 – 0x15100 assume: program page table fjlled in on demand only smarter OS would probably proactively fjll in multiple pages question: how much page/protection faults? 1: set PTE for ofgset 0x3000-0x3FFF (use cached version) 2,3: read from disk + set PTE for 0x4000-0x4FFF; set PTE for 0x5000-0x5FFF 4,5: copy for 0x4000-0x4FFF, 0x5000-0x5FFF 25 virtual 0x10000-0x1FFFF (64KB) → “foo.dat” bytes

  27. maps counting 4KB ( 0x1000 byte) pages 0-0x0FFFF map setup private (copy-on-write) bytes 0-0x3FFF and 0x5000-0x6FFF cached in memory program reads addresses 0x13800 – 0x15800 then, program overwrites addresses 0x14800 – 0x15100 assume: program page table fjlled in on demand only smarter OS would probably proactively fjll in multiple pages question: how much page/protection faults? 1: set PTE for ofgset 0x3000-0x3FFF (use cached version) 2,3: read from disk + set PTE for 0x4000-0x4FFF; set PTE for 0x5000-0x5FFF 4,5: copy for 0x4000-0x4FFF, 0x5000-0x5FFF 25 virtual 0x10000-0x1FFFF (64KB) → “foo.dat” bytes

  28. maps counting 4KB ( 0x1000 byte) pages 0-0x0FFFF map setup private (copy-on-write) bytes 0-0x3FFF and 0x5000-0x6FFF cached in memory program reads addresses 0x13800 – 0x15800 then, program overwrites addresses 0x14800 – 0x15100 assume: program page table fjlled in on demand only smarter OS would probably proactively fjll in multiple pages question: how much page/protection faults? 1: set PTE for ofgset 0x3000-0x3FFF (use cached version) 2,3: read from disk + set PTE for 0x4000-0x4FFF; set PTE for 0x5000-0x5FFF 4,5: copy for 0x4000-0x4FFF, 0x5000-0x5FFF 26 virtual 0x10000-0x1FFFF (64KB) → “foo.dat” bytes

  29. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 27 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  30. Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 28 [ heap ] / bin / cat / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  31. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  32. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  33. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  34. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  35. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  36. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  37. mapped pages (no backing fjle) virtual pages w/o backing fjle page table (part) data in memory data on disk (if any) “swapped out” access new page page fault handler allocates on demand need more memory? save page to disk AKA “swap out” data in memory data in memory 29

  38. MAP_PRIVATE | MAP_ANONYMOUS /* = no file */ , ...); Linux maps allocated using sbrk() [ vvar ] [ vdso ] at virtual addresses 0x400000 – 0x40b000 read, not write, execute, private private = copy-on-write (if writeable) starting at ofgset 0 of the fjle /bin/cat device major number 8 device minor number 1 inode 48328831 more on what this means when we talk about fjlesystems heap — no corresponding fjle but can get same efgect with mmap() call $ cat / proc / self / maps read/write, copy-on-write (private) mapping int fd = open("/bin/cat", O_RDONLY); mmap(0x60b000, 0x1000, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0xb000); (aside: probably used for global variables) as if: int fd = open("/bin/cat", O_RDONLY); mmap(0x400000, 0xb000, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0x0); as if: mmap(..., 0x5000, PROT_READ | PROT_WRITE, [ stack ] 30 / bin / cat [ heap ] / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060b000 − 0060c000 rw − p 0000b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7f60c7854000 − 7f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  39. can move copied data to disk swapping with copy-on-write virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD copies of fjle data, modifjed free up space by removing cached copies of fjle need to free up more space? “swapped out” modifjed data ‘swapped out’ modifjed data 31

  40. can move copied data to disk swapping with copy-on-write virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD copies of fjle data, modifjed free up space by removing cached copies of fjle need to free up more space? “swapped out” modifjed data ‘swapped out’ modifjed data 31

  41. swapping with copy-on-write virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD copies of fjle data, modifjed free up space by removing cached copies of fjle need to free up more space? “swapped out” modifjed data ‘swapped out’ modifjed data 31 can move copied data to disk

  42. can move copied data to disk swapping with copy-on-write virtual pages mapped to fjle page table (part) fjle data, cached in memory fjle data on disk/SSD copies of fjle data, modifjed free up space by removing cached copies of fjle need to free up more space? “swapped out” modifjed data ‘swapped out’ modifjed data 31

  43. swapping historical major use of virtual memory is supporting “swapping” using disk (or SSD, …) as the next level of the memory hierarchy process is allocated space on disk/SSD memory is a cache for disk/SSD only need keep ‘currently active’ pages in physical memory swapping mmap with “default” fjles to use 32

  44. swapping historical major use of virtual memory is supporting “swapping” using disk (or SSD, …) as the next level of the memory hierarchy process is allocated space on disk/SSD memory is a cache for disk/SSD only need keep ‘currently active’ pages in physical memory 32 swapping ≈ mmap with “default” fjles to use

  45. HDD/SDDs are slow HDD reads and writes: milliseconds to tens of milliseconds minimum size: 512 bytes writing tens of kilobytes basically as fast as writing 512 bytes SSD writes and writes: hundreds of microseconds designed for writes/reads of kilobytes (not much smaller) page fault handler is going switch to another program 33

  46. HDD/SDDs are slow HDD reads and writes: milliseconds to tens of milliseconds minimum size: 512 bytes writing tens of kilobytes basically as fast as writing 512 bytes designed for writes/reads of kilobytes (not much smaller) page fault handler is going switch to another program 33 SSD writes and writes: hundreds of microseconds

  47. HDD/SDDs are slow HDD reads and writes: milliseconds to tens of milliseconds minimum size: 512 bytes writing tens of kilobytes basically as fast as writing 512 bytes SSD writes and writes: hundreds of microseconds page fault handler is going switch to another program 33 designed for writes/reads of kilobytes (not much smaller)

  48. the page cache memory is a cache for disk fjles and program memory has a place on disk running low on memory? always have room on disk assumption: disk space approximately infjnite physical memory pages: disk ‘temporarily’ kept in faster storage possibly being used by one or more processes? possibly part of a fjle on disk being read/written? possibly both goal: manage this cache intelligently 34

  49. the page cache memory is a cache for disk fjles and program memory has a place on disk running low on memory? always have room on disk assumption: disk space approximately infjnite physical memory pages: disk ‘temporarily’ kept in faster storage possibly being used by one or more processes? possibly part of a fjle on disk being read/written? possibly both goal: manage this cache intelligently 34

  50. the page cache memory is a cache for disk fjles and program memory has a place on disk running low on memory? always have room on disk assumption: disk space approximately infjnite physical memory pages: disk ‘temporarily’ kept in faster storage possibly being used by one or more processes? possibly part of a fjle on disk being read/written? possibly both goal: manage this cache intelligently 34

  51. the page cache memory is a cache for disk fjles and program memory has a place on disk running low on memory? always have room on disk assumption: disk space approximately infjnite physical memory pages: disk ‘temporarily’ kept in faster storage possibly being used by one or more processes? possibly part of a fjle on disk being read/written? possibly both goal: manage this cache intelligently 34

  52. page cache components [text] handle cache hits fjnd backing location based on virtual address/fjle+ofgset handle cache misses track information about each physical page handle page allocation handle cache eviction 35 mapping: virtual address or fjle+ofgset → physical page

  53. page cache components (recently used? etc.) pointers to remove need reverse mappings to fjnd requires removing pointers to it might need to evict used page choose page that’s not being used much allocating a physical page cache miss : OS looks up location on disk CPU lookup in page table OS lookup for read()/write() cache hit page usage virtual address OS datastructure OS datastructure? OS datastructure page table OS datastructure disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 37

  54. page cache components (recently used? etc.) pointers to remove need reverse mappings to fjnd requires removing pointers to it might need to evict used page choose page that’s not being used much allocating a physical page cache miss : OS looks up location on disk CPU lookup in page table OS lookup for read()/write() cache hit page usage virtual address OS datastructure OS datastructure? OS datastructure page table OS datastructure disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 38

  55. virtual addr/fjle ofgset to physical page for cache hit on memory access multiple designs; one idea: balanced tree (or page fault for mmap’d memory) for cache hit on read/write kernel data structure OS datastructure structure determined by hardware! page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 39

  56. virtual addr/fjle ofgset to physical page for cache hit on memory access multiple designs; one idea: balanced tree (or page fault for mmap’d memory) for cache hit on read/write kernel data structure OS datastructure structure determined by hardware! page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 39

  57. virtual addr/fjle ofgset to physical page for cache hit on memory access multiple designs; one idea: balanced tree (or page fault for mmap’d memory) for cache hit on read/write kernel data structure OS datastructure structure determined by hardware! page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 39

  58. Linux: forward mapping process control block ( task_struct ) mmap region info ( vm_area_struct ) open fjle info ( struct file ) fjle on disk info ( struct inode ) cached physical pages for fjle ( address_space ) page table used to fjll (for mmap) read()/write() 40

  59. Linux: forward mapping process control block ( task_struct ) mmap region info ( vm_area_struct ) open fjle info ( struct file ) fjle on disk info ( struct inode ) cached physical pages for fjle ( address_space ) page table used to fjll (for mmap) read()/write() 41

  60. Linux: forward mapping process control block ( task_struct ) mmap region info ( vm_area_struct ) open fjle info ( struct file ) fjle on disk info ( struct inode ) cached physical pages for fjle ( address_space ) page table used to fjll (for mmap) read()/write() 42

  61. Linux: forward mapping process control block ( task_struct ) mmap region info ( vm_area_struct ) open fjle info ( struct file ) fjle on disk info ( struct inode ) cached physical pages for fjle ( address_space ) page table used to fjll (for mmap) read()/write() 43

  62. Linux: forward mapping process control block ( task_struct ) mmap region info ( vm_area_struct ) open fjle info ( struct file ) fjle on disk info ( struct inode ) cached physical pages for fjle ( address_space ) page table used to fjll (for mmap) read()/write() 44

  63. mapped pages (read/write, shared) fjle data, cached in memory fjle data on disk/SSD 45

  64. page replacement step 1: evict a page to free a physical page case 1: there’s an unused page, just use that (easy) case 2: need to remove whatever what’s in that page (more work) step 2: load new, more important in its place needs some way of knowing location of data 47

  65. page replacement step 1: evict a page to free a physical page case 1: there’s an unused page, just use that (easy) case 2: need to remove whatever what’s in that page (more work) step 2: load new, more important in its place needs some way of knowing location of data 48

  66. page cache components (recently used? etc.) pointers to remove need reverse mappings to fjnd requires removing pointers to it might need to evict used page choose page that’s not being used much allocating a physical page cache miss : OS looks up location on disk CPU lookup in page table OS lookup for read()/write() cache hit page usage virtual address OS datastructure OS datastructure? OS datastructure page table OS datastructure disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 49

  67. 50 OS datastructure swapped out non-fjle: trick: unused PTEs part of fjle: track mmap ‘regions’ (Linux) based on fjlesystem — later topic OS datastructure OS datastructure page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) virtual address/fjle ofgset → location on disk

  68. 50 OS datastructure swapped out non-fjle: trick: unused PTEs part of fjle: track mmap ‘regions’ (Linux) based on fjlesystem — later topic OS datastructure OS datastructure page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) virtual address/fjle ofgset → location on disk

  69. 50 OS datastructure swapped out non-fjle: trick: unused PTEs part of fjle: track mmap ‘regions’ (Linux) based on fjlesystem — later topic OS datastructure OS datastructure page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) virtual address/fjle ofgset → location on disk

  70. 51 OS datastructure swapped out non-fjle: trick: unused PTEs part of fjle: track mmap ‘regions’ (Linux) based on fjlesystem — later topic OS datastructure OS datastructure page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) virtual address/fjle ofgset → location on disk

  71. Linux maps: list of maps $ cat / proc / self / maps … info about sharing of non-fjle data (e.g. heap after fork) (not shown): pointer to backing fjle (if any) ofgset in backing fjle (if any) permissions virtual address start, end (shown in this output): PCB contains list of struct vm_area_struct with: [ vdso ] [ vvar ] [ stack ] 52 [ heap ] / bin / cat / bin / cat / bin / cat 00400000 − 0040b000 r − xp 00000000 08:01 48328831 0060 a000 − 0060 b000 r − − p 0000 a000 08:01 48328831 0060 b000 − 0060 c000 rw − p 0000 b000 08:01 48328831 01974000 − 01995000 rw − p 00000000 00:00 0 7 f60c718b000 − 7 f60c7490000 r − − p 00000000 08:01 77483660 / usr / lib / locale / locale − archive 7 f60c7490000 − 7 f60c764e000 r − xp 00000000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c764e000 − 7 f60c784e000 − − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c784e000 − 7 f60c7852000 r − − p 001 be000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7852000 − 7 f60c7854000 rw − p 001 c2000 08:01 96659129 / lib / x86_64 − linux − gnu / libc − 2.19. so 7 f60c7854000 − 7 f60c7859000 rw − p 00000000 00:00 0 7 f60c7859000 − 7 f60c787c000 r − xp 00000000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a39000 − 7 f60c7a3b000 rw − p 00000000 00:00 0 7 f60c7a7a000 − 7 f60c7a7b000 rw − p 00000000 00:00 0 7 f60c7a7b000 − 7 f60c7a7c000 r − − p 00022000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7c000 − 7 f60c7a7d000 rw − p 00023000 08:01 96659109 / lib / x86_64 − linux − gnu / ld − 2.19. so 7 f60c7a7d000 − 7 f60c7a7e000 rw − p 00000000 00:00 0 7 ffc5d2b2000 − 7 ffc5d2d3000 rw − p 00000000 00:00 0 7 ffc5d3b0000 − 7 ffc5d3b3000 r − − p 00000000 00:00 0 7 ffc5d3b3000 − 7 ffc5d3b5000 r − xp 00000000 00:00 0 ffffffffff600000 − ffffffffff601000 r − xp 00000000 00:00 0 [ vsyscall ]

  72. 53 OS datastructure swapped out non-fjle: trick: unused PTEs part of fjle: track mmap ‘regions’ (Linux) based on fjlesystem — later topic OS datastructure OS datastructure page table virtual address disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) virtual address/fjle ofgset → location on disk

  73. Linux: tracking swapped out pages need to lookup location on disk potentially one location for every virtual page trick: store location in “ignored” part of page table entry instead of physical page #, permission bits, etc., store ofgset on disk 54

  74. page replacement step 1: evict a page to free a physical page case 1: there’s an unused page, just use that (easy) case 2: need to remove whatever what’s in that page (more work) step 2: load new, more important in its place needs some way of knowing location of data 55

  75. evicting a page remove victim page from page table, etc. every page table it is referenced by every list of fjle pages … if needed, save victim page to disk going to require: way to fjnd page tables, etc. using page way to detect whether it needs to be saved to disk 56

  76. page cache components (recently used? etc.) pointers to remove need reverse mappings to fjnd requires removing pointers to it might need to evict used page choose page that’s not being used much allocating a physical page cache miss : OS looks up location on disk CPU lookup in page table OS lookup for read()/write() cache hit page usage virtual address OS datastructure OS datastructure? OS datastructure page table OS datastructure disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 57

  77. page cache components (recently used? etc.) pointers to remove need reverse mappings to fjnd requires removing pointers to it might need to evict used page choose page that’s not being used much allocating a physical page cache miss : OS looks up location on disk CPU lookup in page table OS lookup for read()/write() cache hit page usage virtual address OS datastructure OS datastructure? OS datastructure page table OS datastructure disk location (if cached) physical page (for read()/write()) fjle + ofgset (used by program) 57

  78. tracking physical pages: fjnding mappings want to evict a page? remove from page tables, etc. need to track where every page is used! common solution: structure for every physical page with info about every cached fjle/page table using page 58

  79. Linux: reverse mapping (fjle pages) page table (e.g. to remove/change them) fjnd references to that page given page number page number ( struct page ) per-physical page info ( address_space ) process control block ( task_struct ) cached physical pages for fjle ( struct inode ) fjle on disk info ( struct file ) open fjle info ( vm_area_struct ) mmap region info 59

Recommend

Changelog f(i, j); // use A[i*N+j] and B[j*N+i] } f(i + 1, j); for ( int j = ...) f(i, j); for (

Changelog f(i, j); // use A[i*N+j] and B[j*N+i] } f(i + 1, j); for ( int j = ...) f(i, j); for (

Changelog f(i, j); // use A[i*N+j] and B[j*N+i] } f(i + 1, j); for ( int j = ...) f(i, j); for ( int j = ...) for ( int i = ...; i += 2) { // (probably not very helpful) // unroll loop in i only } f(i + 1, j); f(i + 0, j); for ( int j = ...)

764 views • 22 slides
Changelog Changes made in this version not seen in fjrst lecture: sz is one past the end of the

Changelog Changes made in this version not seen in fjrst lecture: sz is one past the end of the

Changelog Changes made in this version not seen in fjrst lecture: sz is one past the end of the heap 0 2 November: Correct space on demand from to < and > to since Virtual Memory 1 1 last time deadlock thread A holding a

2.01k views • 162 slides
I/O 2 / Filesystems 1 1 Changelog Changes made in this version not seen in fjrst lecture: 13

I/O 2 / Filesystems 1 1 Changelog Changes made in this version not seen in fjrst lecture: 13

I/O 2 / Filesystems 1 1 Changelog Changes made in this version not seen in fjrst lecture: 13 November: Correct cluster number on FAT directory entry slide. 1 last time page replacement modifjcations for scanning Linux: guess fjle pages used

2.03k views • 95 slides
Distributed 3: Network FS (fjnish) / Failure 1 Changelog Changes made in this version not seen

Distributed 3: Network FS (fjnish) / Failure 1 Changelog Changes made in this version not seen

Distributed 3: Network FS (fjnish) / Failure 1 Changelog Changes made in this version not seen in fjrst lecture: 16 April 2019: move and relocate Coda/disconnected operation slides to better explain connection to last-writer-wins being a

1.45k views • 123 slides
The GNU Radio Companion Changelog Communicatjons Engineering Lab Prof. i.R. Dr.rer.nat. Friedrich

The GNU Radio Companion Changelog Communicatjons Engineering Lab Prof. i.R. Dr.rer.nat. Friedrich

The GNU Radio Companion Changelog Communicatjons Engineering Lab Prof. i.R. Dr.rer.nat. Friedrich K. Jondral KIT Universitt des Landes Baden-Wrttemberg und www.kit.edu nationales Forschungszentrum in der Helmholtz-Gemeinschaft Overview

448 views • 23 slides
bitwise (fjnish) / SEQ part 1 1 Changelog Changes made in this version not seen in fjrst

bitwise (fjnish) / SEQ part 1 1 Changelog Changes made in this version not seen in fjrst

bitwise (fjnish) / SEQ part 1 1 Changelog Changes made in this version not seen in fjrst lecture: 14 September 2017: slide 16-17: the x86 arithmetic shift instruction is sar , not sra 1 last time bitwise strategies: construct/apply mask =

890 views • 86 slides
capabilities / virtual machines 1 Changelog Changes not seen in fjrst lecture: 23 April 2020:

capabilities / virtual machines 1 Changelog Changes not seen in fjrst lecture: 23 April 2020:

capabilities / virtual machines 1 Changelog Changes not seen in fjrst lecture: 23 April 2020: add index slides re: cases for priviliged instruction handling and introduction explaining syscalls as special case 1 last time (1) network fjle

1.71k views • 120 slides
Optimization part 1 1 Changelog Changes made in this version not seen in fjrst lecture: 29 Feb

Optimization part 1 1 Changelog Changes made in this version not seen in fjrst lecture: 29 Feb

Optimization part 1 1 Changelog Changes made in this version not seen in fjrst lecture: 29 Feb 2018: loop unrolling performance: remove bogus instruction cache overhead remark 1 29 Feb 2018: spatial locality in Akj: correct reference to B k +1

1.03k views • 63 slides
web security (part 2) 1 Changelog Corrections made in this version not in fjrst posting: 25

web security (part 2) 1 Changelog Corrections made in this version not in fjrst posting: 25

web security (part 2) 1 Changelog Corrections made in this version not in fjrst posting: 25 April 2017: removed text about reading contents without sending cokoies from operations not requiring same origin slide. (This can be done with

998 views • 77 slides
Changelog Changes made in this version not seen in fjrst lecture: 5 Feb 2019: precedence note:

Changelog Changes made in this version not seen in fjrst lecture: 5 Feb 2019: precedence note:

Changelog Changes made in this version not seen in fjrst lecture: 5 Feb 2019: precedence note: replace array subscripts with 42 to avoid using special numbers like 0 and 1 that make examples less clear 0 assembly 2 / C 1 last time object

1.66k views • 134 slides
Virtual Memory 2 1 Changelog Changes made in this version not seen in fjrst lecture: 23

Virtual Memory 2 1 Changelog Changes made in this version not seen in fjrst lecture: 23

Virtual Memory 2 1 Changelog Changes made in this version not seen in fjrst lecture: 23 October: mapped pages (no backing fjle): fjx end of animation to have page on disk 23 October: separate out discussion of readahead from other reasons why

1.79k views • 152 slides
Changelog Changes made in this version not seen in fjrst lecture: 6 September: fjx stray @s on

Changelog Changes made in this version not seen in fjrst lecture: 6 September: fjx stray @s on

Changelog Changes made in this version not seen in fjrst lecture: 6 September: fjx stray @s on implementing fjle descriptors in xv6 slide 6 September: typical pattern with redirection: hilite parts of code more sensibly 6 September: exec

912 views • 77 slides
context switches/POSIX API 1 Changelog Changes made in this version not seen in fjrst lecture:

context switches/POSIX API 1 Changelog Changes made in this version not seen in fjrst lecture:

context switches/POSIX API 1 Changelog Changes made in this version not seen in fjrst lecture: 24 Jan 2019: wait: return value for WNOHANG when process not done is 0, not -1 1 last time system call fmow in xv6 other exceptions in xv6

1.7k views • 138 slides
1 Changelog Changes made in this version not seen in fjrst lecture: 25 September: add back

1 Changelog Changes made in this version not seen in fjrst lecture: 25 September: add back

1 Changelog Changes made in this version not seen in fjrst lecture: 25 September: add back stages walkthrough slides 1 last time mov CPU build incrementally difgerent things for difgerent instructions add MUX MUX controls = function of

1.22k views • 107 slides
synchronization 3: rwlocks / deadlock 1 Changelog Changes not seen in fjrst lecture: 20 Feb

synchronization 3: rwlocks / deadlock 1 Changelog Changes not seen in fjrst lecture: 20 Feb

synchronization 3: rwlocks / deadlock 1 Changelog Changes not seen in fjrst lecture: 20 Feb 2020: moving two fjles graphs: make directory names consistent with code 20 Feb 2020: is deadlock exercise: correct option lettering 20 Feb 2020:

1.52k views • 120 slides
Virtual Memory 1 Changelog Changes made in this version not seen in fjrst lecture: 16 November

Virtual Memory 1 Changelog Changes made in this version not seen in fjrst lecture: 16 November

Virtual Memory 1 Changelog Changes made in this version not seen in fjrst lecture: 16 November 2017: shared libraries: clarifjed that its multiple read-only virtual copies backed by one physical copy 1 SIMD notes (1) sorry for two typos:

1.42k views • 99 slides
two-phase commit / security (start) 1 Changelog Changes made in this version not seen in fjrst

two-phase commit / security (start) 1 Changelog Changes made in this version not seen in fjrst

two-phase commit / security (start) 1 Changelog Changes made in this version not seen in fjrst lecture: quorum: add note that part of voting is updating other nodes to latest version 1 last time (1) RPC: remote function calls like local

1.05k views • 78 slides
scheduling 3 1 Changelog Changes not seen in fjrst lecture: 4 Feb 2020: MLFQ example: number

scheduling 3 1 Changelog Changes not seen in fjrst lecture: 4 Feb 2020: MLFQ example: number

scheduling 3 1 Changelog Changes not seen in fjrst lecture: 4 Feb 2020: MLFQ example: number priorities so largest number is highest priority 4 Feb 2020: lottery scheduler assignment: replace how long processes run with how often

839 views • 82 slides
system calls/excpetions/context switches 1 Changelog Changes made in this version not seen in

system calls/excpetions/context switches 1 Changelog Changes made in this version not seen in

system calls/excpetions/context switches 1 Changelog Changes made in this version not seen in fjrst lecture: contexts (B running) adjust remark about where values are saved to note that As user regs are saved on As kernel stack rather

1.74k views • 106 slides
injection vulnerabilities 1 Changelog Corrections made in this version not in fjrst posting: 17

injection vulnerabilities 1 Changelog Corrections made in this version not in fjrst posting: 17

injection vulnerabilities 1 Changelog Corrections made in this version not in fjrst posting: 17 April 2017: slide 35: make note on slide of second escapings misinterpretation 1 Last time static analysis pattern matching for

861 views • 74 slides
1 Changelog Corrections made in this version not in fjrst posting: 1 April 2017: slide 13: a few

1 Changelog Corrections made in this version not in fjrst posting: 1 April 2017: slide 13: a few

1 Changelog Corrections made in this version not in fjrst posting: 1 April 2017: slide 13: a few more %cs would be needed to skip format string part 1 OVER questions? 2 last time memory management problems two objects end up at same

965 views • 63 slides
Changelog Changes made in this version not seen in fjrst lecture: 24 Jan 2019: wait: return value

Changelog Changes made in this version not seen in fjrst lecture: 24 Jan 2019: wait: return value

Changelog Changes made in this version not seen in fjrst lecture: 24 Jan 2019: wait: return value for WNOHANG when process not done is 0, not -1 24 Jan 2019: reading a fjxed amount: add check for amount_read = 0 0 POSIX API 2 1 last time

953 views • 93 slides
Scheduling 1 1 Changelog Changes made in this version not seen in fjrst lecture: 10 September:

Scheduling 1 1 Changelog Changes made in this version not seen in fjrst lecture: 10 September:

Scheduling 1 1 Changelog Changes made in this version not seen in fjrst lecture: 10 September: RR varying quantum examples: fjx calculation of response/wait time on Q=2 10 September: add priority scheduling and preemption slide 10 September:

1.79k views • 115 slides
Changelog Changes made in this version not seen in fjrst lecture: 18 Feb 2019: counting to binary

Changelog Changes made in this version not seen in fjrst lecture: 18 Feb 2019: counting to binary

Changelog Changes made in this version not seen in fjrst lecture: 18 Feb 2019: counting to binary semaphores: really correct implementation (after some failed attempts) 0 Locks part 2 1 last time disabling interrupts for locks (fjnish)

1.13k views • 82 slides

More recommend