On April 1st of 2018, I spent a few hours using a browser-based software program that can create potentially endless compositions. The program uses machine learning and a database of classical scores to write its own musical rules deciding how and what notes should follow previous notes to complete/continue a musical phrase. During the process, I had a few questions: Is this music? Is this Sugar Skulls music? What are the implications if the listener answers yes to either of those questions?
Traditional computer programs are given a set of rules to execute a designated end. The ability of a program to execute its goal successfully is dependent upon the ability of the programmer to abstract the goal into a set of repeatable rules and to define the conditions in which those rules exist. The program should accomplish the goal explicitly and only that goal, programs bugs are unexpected behavior from actions or unaccounted conditions. Simple repeatable actions occurring within a limited set of conditions are ideal for traditional programming methods.
Western music theory is one attempt at formalizing musical creativity/expression and could be thought of as a series of rules derived from the analysis of compositions. Compositional rules like scales, modes, chord resolutions, rhythmic patterns, etc. describe a method of organization for an expected repeatable musical output. However, a large portion of music theory explores how and when to subvert its own rules. The creative act for the musician is not the execution of musical rules but the combination, mutation, organization, and disregard of musical rules according to stylistic taste.
The mathematical nature of musical harmony would seem to suggest the composition of music could be an ideal candidate for automation. However, the problem of how to automate the stylistic decisions a musician might make in regard to composition into a set of rules is exponential. This reducibility problem means automation in music has been applied to limited aspects of composition. Machine learning works with an opposite approach. Rather than a programmer define every action for every condition, a program is given a large number of examples of what its output should be and it devises its own rules to create the expected output. Simplistically, the larger and more varied the data set of examples is the more likely the desired output will be achieved.
The program I used was built using Google’s Magenta AI for Artists tools. The programmer’s data set was approximately 10,000 classical musical scores. The interface is a visual representation of a piano keyboard. The user selects a set of notes and the program composes music based on those notes. The output can be either sound or MIDI note information, so I sent the MIDI notes into the synthesizer that I typically compose with. Because Sugar Skulls compositions are modal, with each composition limited to a range of notes, I input a single chord for the program to use to generate the compositions of each track based on my own stylistic taste. The result is an album of the compositions generated by the AI program, played through a synthesizer patch.
I was curious how the result would mirror my own output as Sugar Skulls. If the result did not meet my expectations, how and what should be altered until the music felt like it met the abstract intuitive rules of my own stylistic taste. As mentioned, I chose the kinds of chords the compositions were derived from, as well as the sounds the compositions were performed with. The program was responsible for the content of the relationships between the notes; pitch, duration, rhythm, frequency, etc. I resisted the impulse to edit out perceived mistakes or arrange the output via sampling. The listener can make their own judgment as to whether the result is Sugar Skulls music or an interesting experiment.
Since I used the beta version of this program, many tools are now available that use similar technology to aid musicians in quickly generating many of the surface-level steps in music composition. It isn’t difficult to envision any number of dystopian scenarios where this kind of technology is exploitative. Given the model the program is built on, a logical leap would be to replace the data set with my own (or any musician’s scores) as the expected output. Another leap would include analyzing sound wave information to find what types of timbres or instrument sounds a musician chooses based on compositional conditions. Finding the threshold at which a musician’s style can be recreated is a loaded possibility.
Sampling in music has been a contentious topic since the introduction of the technology that makes it possible. One side considers sampling to be theft while the other considers it transformative recontextualization that is another tool of expression. There is no clear way to unpack who gets to decide who owns a work and what constitutes an original work. Machine learning presents the problem of a different kind of sampling. Rather than the appropriation and manipulation of sound artifacts, the stylistic approach of a musician can be appropriated to generate new content.
As the technology matures and improves, it will invariably alter the nature of musical creativity. The introduction of photography and the technology’s ability to render an image of an individual didn’t make portrait drawing and painting redundant. Photography changed and opened up the possibilities, meaning, and purpose of human rendered forms. In the case of music, AI driven compositional technology could have a democratizing effect on the creation of music. The creation of music is not just bound to the musician’s ability to perform an idea on an instrument or knowledge of music theory, but also the distillation of a musician’s stylistic influences. A tool that can manipulate content via styles like swatches on a palate already exists for image creation. However, just as deceased musicians can be made into holograms that tour previous material and generate revenue without the musician’s explicit consent, the stylistic essence of a musician can potentially be quantified and made to output new works without the musician present. The technology will present new methods to exploit musicians and create new modes of expression. Ultimately it will further blur the lines of listening to and creating music, which is a kind of subversion I fully support.

Enjoy!