V, pensiero! - Fly, thought! Experiment for interactive internet - PowerPoint PPT Presentation

„Và, pensiero!“ - Fly, thought! Experiment for interactive internet based piece using Csound6 Tarmo Johannes Kita Toshihiro 2013

WHY? ● Csound community is strong and active, spread over the world ● Chances to meet physically are rear ● Are there ways to meet musically over distance? ● Csound6 run-time compilation of orchestra and score offer vast possibilities for internet based interactive music making

„Và pensiero“- Fly, thought! ● Dedicated to developers of Csound6 and all bright Csounders ● People join to „choir“ - writing everybody their voice as an Csound instrument ● One player plays on a MIDI keyboard that triggers the „voices“ ● Participants can change their instruments in real time, depending on the current situation. Nobody knows exactly how the choir will sound in the performance :) ● The result (should be) streamed into internet, so the players and public can hear it

Software ● pensiero-server – written in C++, Qt framework: – sets up listener for incoming messages, loads csd file and plays it, compiles Csound code from the messages, displays output ● va_pensiero_orc.csd – collection of the „voices“, triggering midi instrument, definitions of chords (the melody is orchestrated automatically) etc ● Clients for sending TCP messages: – pensiero-client.csd – works in CsoundQt with PythonQt support (sends selected text) – pensiero_client.ipynb – iPython notebook (requires iPython works in any web browser, send Csound code from iPython cells.)

Software ● pensiero-server – written in C++, Qt framework: – sets up listener for incoming messages, loads csd file and plays it, compiles Csound code from the messages, displays output ● va_pensiero_orc.csd – collection of the „voices“, triggering midi instrument, definitions of chords (the melody is orchestrated automatically) etc ● Clients for sending TCP messages: – pensiero-client.csd – works in CsoundQt with PythonQt support (sends selected text) – pensiero_client.ipynb – iPython notebook (requires iPython works in any web browser, send Csound code from iPython cells.)

pensiero-server ● Incoming connections are handled in separate threads ● Received messages are forwarded to Csound in a separate thread to compile and play, result is sent back to client ● The messages are displayed as chat conversation between users and Csound

Networking ● pensiero-server must be visible from network (open TCP port) ● The public server handling/forwarding the messages is now 54.228.204.118 (in Ireland) ● The messages can be tunneled via ssh to another computer even if it is behind a firewall (like in Boston) – the actual performance place can be anywhere with internet connection

Streaming the result ● Needs more work and testing ● We tried vlc ( VideoLAN media player) for streaming – works otherwise well with Csound, but it blocked somehow compiling new code ( compileOrc ) ● Working solution: icecast <- darkice <- jackd <- Csound ● Listen with any media player (for example): mplayer -nocache http://johannes.homeip.net:8000/pensiero (currently the stream is ogg encoded) ● TODO: minimize lag between sound production and listening ● Automatic connection of darkice and Cound jack output

Demo 1 send messages to test orchestra ● test_orc.csd – plays instr 100 (pluck sound) every 5 seconds ● HOST: 54.228.204.118 , port: 5005 ● Use CsoundQt or iPython Notebook, command line or anything that can send a string as TCP message ● Command line example: echo „giVal init 12“ | nc 54.228.204.118 5005 ● Prepared clients are up on: http://tarmo.uuu.ee/varia/failid/cs/pensiero-files/ ● Send your name first in the form ';name YOUR_NAME' for clearer output ● You can send new instruments, run instruments with event_i or scoreline_i opcode, (re)define tables etc – everything you can do within the orchestra header ● Please don't send scoreline_i „e“ ! (Check against that not implemented yet ...)

Demo 2 Forwarding live playing (test) ● Does not work properly yet! ● Tarmo plays on a midi keyboard at home, note on/off are sent as event_i / turnoff2 to pensiero- server ● Problem: right timings get easily disturbed – messages don't arrive exactly enough ● Some synchronisation system or other solution is necessay.

Demo 3: Streaming ● Tarmo plays „Va pensiero“ on MIDI keyboard at home, streams up on http://johannes.homeip.net:8000/pensiero ● pensiero-server runs in Tarmo's computer ● Send messages to HOST: johannes.homeip.net, PORT 5005

Demo 4 ● Keyboard is played and sound produced on spot ● (Messages to 54.228.204.118, port: 5005) ● Everybody can play along via Csound messages!

Possibilities for future ● Remote teamwork (for example developing new Csound instruments - needs extension for saving the code) ● Interactive, intercontinental performances ● Csound improvisations, jam sessions, live coding - another way for community events ● TODO: better streaming, improvements in server, tests, tests, tests

Thank you! Have a wonderful conference! Tarmo, Kita tarmo@otsakool.edu.ee kita@ield.kumamoto-u.ac.jp