shifting media app development into high gear
play

Shifting Media app development into high gear Using virtual drivers - PowerPoint PPT Presentation

Shifting Media app development into high gear Using virtual drivers to speed up development Helen Koike helen.koike@collabora.com Open First Creative Commons Attribution-ShareAlike 4.0 Summary Classic V4L2 API Vivid Driver Media


  1. Shifting Media app development into high gear Using virtual drivers to speed up development Helen Koike helen.koike@collabora.com Open First Creative Commons Attribution-ShareAlike 4.0

  2. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 2

  3. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 3

  4. ELC 2018 4

  5. Sepia Sensor Composer Scaler Greyscale ELC 2018 5

  6. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 6

  7. APP Read/Write/MMAP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Video stream Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 7

  8. APP Read/Write/MMAP User IOCTLs space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Standard Custom Video stream General confgs Controls Controls * Contrast * DRV DEF 1 * Img fmt Kernel * Brightness * DRV DEF 2 * Bufers space * Gamma * DRV DEF 3 * Video std . . * Frame rate . . . . . . ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 8

  9. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 9

  10. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 10

  11. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- The Virtual Video T est Driver (VIVID) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 11

  12. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- The Virtual Video T est Driver (VIVID) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 12

  13. Vivid driver ELC 2018 13

  14. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 14

  15. Media API Motivation ● Embedded systems: great variety of devices ● Hardware complexity ● Similar confguration for diferent entities ELC 2018 15

  16. contrast Sepia Sensor Composer Scaler Greyscale contrast ELC 2018 16

  17. APP Read/Write/MMAP User IOCTLs space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Standard Custom Video stream General confgs Controls Controls * Contrast * DRV DEF 1 * Img fmt Kernel * Brightness * DRV DEF 2 * Bufers space * Gamma * DRV DEF 3 * Video std . . * Frame rate . . . . . . ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 17

  18. Sepia Sensor Composer Scaler Greyscale ELC 2018 18

  19. Sepia Sensor Composer Scaler Greyscale ELC 2018 19

  20. /dev/video0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 20

  21. APP Subdev API contrast IOCTLs Sepia /dev/v4l- subdev0 APP APP R/W/MMAP IOCTLs Scaler IOCTLs /dev/v4l- /dev/video0 subdev2 APP contrast IOCTLs Greyscale /dev/v4l- subdev1 ELC 2018 21

  22. What is part of the device? What is the topology? APP ? ? ? ? ? ? ? ? ? ? ? ? /dev/v4l-subdev0 /dev/v4l-subdev98 /dev/v4l-subdev11 /dev/v4l-subdev14 /dev/video2 /dev/video0 /dev/v4l-subdev6 /dev/video42 /dev/v4l-subdev21 /dev/v4l-subdev31 ELC 2018 22

  23. /dev/video0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 23

  24. /dev/video0 out Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 24

  25. APP Media API contrast IOCTLs Sepia /dev/v4l- subdev0 APP APP APP R/W/MMAP IOCTLs IOCTLs Scaler IOCTLs /dev/media0 /dev/v4l- /dev/video0 subdev2 APP contrast IOCTLs Greyscale /dev/v4l- subdev1 ELC 2018 25

  26. APP /dev/video0 IOCTLs out /dev/media0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 26

  27. APP /dev/video0 IOCTLs out /dev/media0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 27

  28. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 28

  29. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 29

  30. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- The Virtual Media Controller Driver (VIMC) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 30

  31. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- The Virtual Media Controller Driver (VIMC) Kernel space ? ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 31

  32. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 32

  33. APP Developer User space --------------- VIMC Confgfs API --------------------- /dev/media0 --------------------- /dev/media1 ----------- Kernel create space create ELC 2018 33

  34. Vimc: submodules Subdev1.ko Implementation of subdevice Subdev2.ko drivers are modularized and doesn’t need to alter Vimc’s core code. SubdevN.ko ELC 2018 34

  35. vimc-sensor.ko vimc-scaler.ko APP Developer Standard vimc-debayer.ko vimc-capture.ko ... Emulate diferent topologies to test the app in several scenarios custom-sd1.ko custom-sd2-ko Manufacturer custom-sd3.ko ... ELC 2018 35

  36. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 36

  37. Vimc Driver ● Proposed by Laurent Pinchart for Outreachy in 2015 ● Merged in Kernel 4.12 ● Moving slowly ELC 2018 37

  38. Vimc Driver: current state ● Basic standard submodules: – vimc-capture.ko – vimc-sensor.ko – vimc-debayer.ko – vimc-scaler.ko ● Hard-coded topology (re-compilation required) ELC 2018 38

  39. ELC 2018 39

  40. Vimc Driver ELC 2018 40

  41. Userspace tools ● v4l-utils media-ctl -d /dev/media0 --print-dot > /tmp/out.dot && xdot /tmp/out.dot media-ctl -d /dev/media0 -V " 'Sensor A':0 [ fmt: RGB888_1X24/600x600]" media-ctl -v -d /dev/media0 --links "'Debayer A':1 -> 'Scaler':0 [0] " ● Yavta (Yet Another V4L2 T est Application) yavta --format RGB24 --size 600x600 /dev/video0 ELC 2018 41

  42. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 42

  43. Confgfs API: Topology ● Entities ‘Sensor A’ vimc-sensor.ko – Name 0 – Submodule ● Pads 0 ‘Debayer A’ vimc-debayer.ko – Source 1 – Sink ● Links 0 ‘Raw Capture 0’ vimc-capture.ko ELC 2018 43

  44. mkdir “MEDIA_NAME” ELC 2018 44

Recommend


More recommend