Over the past years, Deep Learning has gained much popularity due to its high levels of accuracy when trained with large amounts of data. It has been crucial in solving challenging problems such as face recognition, speech recognition, and image classification. However, these are all problems with one right answer. We were curious about how Deep Learning would perform at a more creative task like music generation. It turns out, many researchers have worked on this problem in the past and it is still an active area of research. However, we did not want to take their word for it and decided to build a Deep Learning model from scratch to generate music.

We decided to build a Long-Short Term Memory network, a type of Recurrent Neural Network that is particularly well-suited for modeling sequences with long-term dependencies such as text, speech, and music. At a high level, we take midi files that contain folk melodies, process them to bring them into a format that makes sense to our network, let the network learn from the patterns exhibited in these melodies, and generate original folk melodies once the model is trained. We used two separate models to predict the notes and note lengths, but both models consisted of the same architecture.

Our model is deterministic, that is, starting from the same note always gives the same output. However, there are 41 possible starting notes, and we can transpose the result to any of the 12 musical keys. So in total, there are 41*12 = 492 possible outcomes, which is plenty to listen to.

We fed our network ~300 folk tunes from the Nottingham dataset and trained it for 50 epochs. We repeated this process for three iterations with a different drop out value at each iteration (0.15, 0.5, and 0.85). We evaluated our model using categorical cross entropy loss and categorical accuracy. At our best training iteration (drop out = 0.5), our model achieved a note accuracy of 45% on the training data and 35% on the testing data. Note length accuracy at this iteration was ~75% on both training and testing data.

Although these accuracy values gave us an idea of how well our model has performed, our main concern was whether the music generated by our model sounded similar to the music it was trained on. To get a better understanding of that, we compared our generated songs to two baseline approaches that we have created ourselves. We have created two other music generators that generate random numbers and map them to notes. We made one of these completely random, and we constrained the other one to stay in key. The tempo and range of notes were kept as invariants for fair evaluation. We created a survey in which we provided the participants with an example folk tune from the dataset, along with 3 more audio clips, one from each category : random notes, notes in key, and lstm generated melody. Then, we asked the user which of the three audio clips sounded most similar to the provided example. We are still collecting responses and will release the results when they are ready.