My small Python package from hell - graph2sound.

As an icebreaker for more actually meaningful posts, I thought I would elaborate a bit more on the Python module that I have written a while ago. You may find it on my gitlab where you can also find some different project that I have been working on. Add it to your Python project now with pip install graph2sound and adding import graph2sound as g2s to your code. These are mainly fun projects or tools; I have not committed to a big project yet.

Why?

Good question. I was in my course on control theory when I was looking at a plot of a nicely damped oscillator. Once upon an intrusive thought, I tried to imagine what the sound would be like if you modulated a sine wave by the relative height in the graph. Being not all too unfamiliar with simple signal processing, I tried my hand at writing a simple program which would make me a nice .WAV file for me to play. After writing a quick and dirty version, I decided to not bother any more, even though it sounded like crap.
It was not until I had picked up Python that I considered making a new version: this time, I would do it properly. Or so I thought. Turns out, signal processing is a lot harder than you might think. I am fairly certain the math checks out - the only situations where it doesn’t work properly are some edge cases which I have yet to implement properly.

How?

The way g2s works is by splitting up the tasks into four chunks: argument validation, frequency normalization, synthesizing the sound and playing the sound.

Argument validation

The bare minimum g2s needs is a one-dimensional array of numbers of at least two datapoints. Since sound has two dimensions, we need a datatype with that is at least two dimensional. In the function argkwargcleanup, the arguments are first checked if the dimensions are correct, then if there is a time argument supplied, if it is just as long as the signal and if the signal is multidimensional. The possibilities are y, y+t, y..y+t and y..y+t..t. The validation is not very robust; if you supply misshapen or ragged multi-dimensional data, the frequency table generator will most likely freak out.
The keyword arguments are also validated and sanitized, ready to be used as the options for the filters. I was later made aware of the argparsemodule, which would have been mighty useful and have saved me a lot of boilerplate. Alas, lessons for next time.

Frequency normalization

This is a fairly trivial section. The function freqlist returns a list (length of the number of segments) of frequencies, where all frequencies are chosen based on the relative value of the y signal. By using the y/t data, we first normalize the time to \([0,1]\). Next, we take the signal and normalize it to \([f_{min}, f_{max}]\) where \(f_{min}\) is the minimum frequency used (C3, 130.8 Hz) and \(f_{max}\) the maximum frequency (C7, 2093.0 Hz). This, however, first needs an exponential correction, since I want the frequency distribution like a piano roll, where a set distance is a set interval. We interpolate over both time and signal to create a contiguous array of frequencies so that the frequency changes are not as abrupt. Testing yielded that you need about 100 distinct frequencies per second to create a sound transcription that changes smoothly. Finally, we need to correct for an angular to ordinal frequency error in the playback.

Synthesizing the sound

Very hard. This is where you really need to take care of the math. In essence, the process is as follows:

• make a list of all datapoints that are actually returned by this function;
• calculate the points you will get per distinct frequency segment (truepoints);
• calculate the sum of the above points, where this value is equal or greater than the above (points);
• calculate the time per segment (tps);
• calculate the time per samplepoint (tpp);
• (apply filters as needed);
• go through the list of segment and for every segment with index i:
  • place at i in the list points the small wave segment waveform([0,time_per_segment-time_per_point]*frequency_of_segment + phase);
  • update phase by adding the phase traversed based on the time and frequency;
• cut off the points to truepoints. Since the wave is discrete, you have to account for the phase difference of one samplepoint each segment introduces by being one sample off. By correcting for this error and carefully interpolating the resulting signal, you can create a smooth but non-differentiable curve at each segment boundary.
  This is also the origin of the MATLAB filter; my original script did not implement this feature because I couldn’t figure out how to implement the phase shift of every segment.

Playing the sound

Take samplerate, take generated wave and throw it into a small module with .WAV compatibility. Done.

Final notes

I am weirdly proud of this little project. I thought up something weird, and somehow, I was able to put something on screen within a few hours that actually worked. Since you can now import the module as a pip download, I think it was a fun and borderline useful project. It works very well - to the point that the spectrogram actually matches the original graph. Thank you for putting up with my little story, see you around.