High complexity GStreamer pipelines Buzztard / Audio / Gstreamer Stefan Sauer <ensonic@hora-obscura.de>, 24.10.2011
What is buzztard? What is buzztard? ?
NO
NO
We're getting We're getting there there
Short history Short history ● Buzz: free, but closed source and windows only ● Dead in 2002, continued in 2009
● Buzztard: open source, focuses on Linux ● Since 2003 ● Uses GStreamer :)
Foundation: Registry Foundation: Registry ● Book keeping of features ● Fast startup (cache) ● Safe plugin probing (out-of-process)
Foundation: Scalability Foundation: Scalability ● Huge pipelines ● >500 elements ● >50 threads ● Example buzztard song of this size ● ~ 50 CPU use on a 1.6 GHz Intel Atom ● on a gstreamer debug enabled build
Foundation: Sequencing Foundation: Sequencing ● GstController ● Sync control-changes with rendering ● Outside of ui threads
Foundation: Sparse streams Foundation: Sparse streams ● Gap flag on buffers ● elements take shortcuts on silence
Foundation: DecodeBin(2) Foundation: DecodeBin(2) ● Handling of different audio formats is a NO-OP VORBIS MP3 WAV AIFF AAC AU
Foundation : Audio IO Foundation : Audio IO ● Everything is there! ● alsa, jack, pulseaudio, ...
Foundation: Elements Foundation: Elements ● Lots of utility elements available ● data conversion, level meter, spectrum analyzer, ...
Foundation: Introspection Foundation: Introspection ● Lots of GObject goodness ● Classify elements ● Generate dialogs
Tricky: trick modes Tricky: trick modes ● Jog dial like scrubbing ● All elements need to play to the rules ● Adder fixed in git HEAD ● Still occational lockups in reverse mode
Tricky: seamless loops Tricky: seamless loops ● Don't miss a beat ● Adder fixed in git HEAD
Tricky: dynamic linking Tricky: dynamic linking ● Add/remove elements while playing ● Fragile, complicated and under-documented
Tricky: dynamic linking Tricky: dynamic linking ● data flow on disconnected pads is fatal => pad-blocking ● pad-blocking on inactive pads hangs ● Pad-blocking is async ● Happens on data-flow over the pad ● Mind the direction: src for push, sink for pull ● Block the the the last src pad for push, first sink pad for pull ● Need to emulate startup/shutdown
Tricky: dynamic linking Tricky: dynamic linking ● What to do with conditional elements ● un-parent from the pipeline ● Keep in the pipeline and lock the state
Tricky: dynamic linking Tricky: dynamic linking ● Add elements ● Activate pads ● Block pads ● Change state to PAUSED ● Link ● Seek (the flush unblocks) ● Unlock the state ● Change state to PLAYING
Tricky: dynamic linking : add 1 Tricky: dynamic linking : add 1
Tricky: dynamic linking : add 2 Tricky: dynamic linking : add 2
Tricky: dynamic linking : add 3 Tricky: dynamic linking : add 3
Tricky: dynamic linking Tricky: dynamic linking ● Remove elements ● Block pads ● change state to NULL ● Lock the state ● Unlink ● Unblock pads
Tricky: dynamic linking : remove Tricky: dynamic linking : remove
Tricky: dynamic linking Tricky: dynamic linking ● Buzztard svn repo: design/gst/dynlink{2,3}.c
WIP: plugin wrappers WIP: plugin wrappers ● Ladspa and lv2 still have issues (sources) ● More wrappers needed (dssi, vst)
WIP: low latency audio WIP: low latency audio ● Determine effective latency in push mode ● Make pull mode work for audio ● It does not lockup anymore ● Queues need changes to not buffer
WIP: push mode latency WIP: push mode latency ● Latency related to bpm/tpb and sampling rate ● e.g.: chunk-size for sink 120000 µs = 120 ms
WIP: push mode latency WIP: push mode latency ● 140 ms delay ~ 46ms per queue
Future Future ● GStreamer 1.0 :) ● EncodeBin for recording ● discoverer in the wave-table browser ● if the gtk+filechooser could be populated with disco- info asynchronously … ● tempo information as sticky events on the pads ● beats-per-minute + ticks-per-beat
Thanks! Questions?
Thanks Thanks sheetmusic: by 'starrise' Trey Jones ● sequencer: shows propellerheads reason ● icons are from gnome and tango icon themes ●
Recommend
More recommend
