Interactive constraints computer-aided composition ICMC 2017 Carlos Agon Philippe Esling Pierre Talbot (talbot@ircam.fr) Institute for Research and Coordination in Acoustics/Music (IRCAM) Université Pierre et Marie Curie (UPMC) 19th October 2017
Computer-aided composition Goals ◮ Delegating tedious computations to the machine. ◮ Parametrizing the patch with values to quickly try-and-test. ◮ . . . How does the composer interact with the machine? ◮ Mostly visual and dataflow programming languages: OpenMusic, PureData, Max,. . . ◮ Functional programming languages for the specifics: Lisp mostly. 2
Dataflow: a patch in OpenMusic 3
Constraints in computer-aided composition Constraint programming ◮ Declarative paradigm for solving combinatorial problems. ◮ We state the problem and let the system solve it for us. ◮ Example: pitches must form a decreasing sequence (from highest to lowest). Some examples of attempts to add constraints into CAC softwares: ◮ PWConstraints on top of PatchWork: constraints over the pitches, grouping the pitches together (modelling aspects). ◮ OMCloud on top of OpenMusic is based on a different constraint solving paradigm—local search—aiming at the ease of use. 4
Problem ◮ CAC softwares extended with constraints work in black box : one solution gets out of the box. ◮ But constraints are relations, not functions. ◮ Therefore, a constraint problem can have zero, one or many solutions. By functionalizing the constraint process, we miss a key point: Constraints are useful to describe a class of solutions but how to work with many solutions? 5
Proposal: Interactivity Experiment with an interactive constraint score editor. ◮ Bring the composer at the level of the solving process. ◮ He can consciously choose a solution. ◮ Development of an interactive search strategy to navigate in the solution space . 6
Proposal: Spacetime programming Interactivity and search strategies is a deeper problem: constraint solvers also work in a “black box” mode. We propose the process calculi spacetime programming. SP = constraint programming + synchronous paradigm. Spacetime programming ◮ Synchronous programming for interactive computing . ◮ A search strategy is viewed as a process: abstraction over the constraint solver. 7
Menu ◮ Introduction ◮ Interactivity in solvers ◮ Interactivity in CAC ◮ Conclusion 8
All-interval series: a MiniZinc model int : n = 12; array [1..n] of var 1..n: pitches ; array [1..n − 1] of var 1.. n − 1: intervals; constraint forall (i in 1.. n − 1) ( intervals [ i ] = abs(pitches[ i+1] − pitches[i ])); constraint alldifferent ( pitches ); constraint alldifferent ( intervals ); solve satisfy ; 12 & 4 12 # & 4 n # n # n n n n # n n # n n n n ♮ ♮ 9
All-interval series: a MiniZinc model int : n = 12; array [1..n] of var 1..n: pitches ; array [1..n − 1] of var 1.. n − 1: intervals; constraint forall (i in 1.. n − 1) ( intervals [ i ] = abs(pitches[ i+1] − pitches[i ])); constraint alldifferent ( pitches ); constraint alldifferent ( intervals ); solve satisfy ; 10
Synchronous paradigm ◮ Invented in the 80s to deal with reactive system subject to many (simultaneous) inputs. ◮ Continuous interaction with the environment. ◮ Mainly used in embedded systems. 11
Spacetime execution scheme ◮ The search tree is represented as a queue of nodes. ◮ We feed the program with one node of the tree per instant . ◮ The synchronous program fuels the queue with new nodes. global outputs inputs local synchronous program spacetime extension queue dequeue push 0..N 12
Spacetime programming Syntax � p, q, . . . � ::= communication fragment | spacetime Type x = e (variable declaration) | (ask) when cond then p end | x <- e (tell) | x . m ( . . . ) (method call) | synchronous fragment | (parallel composition) par p || q end | p ; q (sequential composition) | (suspension) suspend when cond in p end | loop p end (infinite loop) | pause (delay) | search tree fragment | space p end (branch creation) | (branch pruning) prune 13
Spacetime attribute Problem How to differentiate between variables in internal/global state and those onto the queue? We use a spacetime attribute to situate a variable in space and time. ◮ single_space : variable global to the search tree. ◮ single_time : variable local to one instant. ◮ world_line : backtrackable variable in the queue of nodes. 14
Menu ◮ Introduction ◮ Interactivity in solvers ◮ Interactivity in CAC ◮ Conclusion 15
Score editor: overview Constraint solving zone for the interactions with the composer. 16
A first interactive strategy The strategy usually implemented in CAC with constraints: stop at each solution. In practice: click on “space” to jump to the next solution. class EachSolution { world_line VStore domains = bot ; world_line CStore constraints = bot ; proc stop_at_solution = loop par || when domains |= constraints then stop end || pause end end } 12 & 4 12 # & 4 n # n n n # n n # n n # n n n ♮ n ♮ 17
Interaction with the composer The composer interacts with the search in-between instants. The spacetime attributes enable interactions with the search in two main ways: globally or only for the current search path. class PSolver { world_line CStore constraints = bot ; single_space CStore cpersistent = bot ; ... } 12 & 4 n # n # n # n n 18
Lazily navigating the solution space The next two scores represent a choice between ♯ D and ♯ G on the sixth note: 12 # n # n # n & 4 n n # n n # n n n # n n n 12 # n # n & 4 n n # n n # n n n # n # n n n SubSolver<RBinary, Model> left = new SubSolver(); SubSolver<Binary, Model> right = new SubSolver(); single_time L<Boolean> choice = bot ; choice <- top ; par || suspend when choice |= true then right .search() end || suspend when choice |= false then left .search() end end 19
Menu ◮ Introduction ◮ Interactivity in solvers ◮ Interactivity in CAC ◮ Conclusion 20
Constraints in music From a computer scientist perspective ◮ Probably not for generating music: machine learning methods do it better. ◮ Reasoning on a class of scores satisfying some properties. Example: we are not forced to write a particular pitch but a class of pitches satisfying some rules. ◮ Constraints do not force the composer to make any choice! 21
Conclusion ◮ Constraints are relational: interactive search helps to use them in this way. ◮ To program interactive search strategies, we use spacetime programming. Future work ◮ Current prototype with AIS only; enabling any MiniZinc model. ◮ This would allow composers to try the system and to develop more strategies. github.com/ptal/bonsai Stay tuned! github.com/ptal/repmus 22
Thank you for your attention. github.com/ptal/bonsai Stay tuned! github.com/ptal/repmus 23
Recommend
More recommend
