Audio Analysis

File Structure

Audio analysis in Nanolux is separated into two sets of files:

• core_analysis.cpp, core_analysis.h
• ext_analysis.cpp, ext_analysis.h

These sets are frequently referred to as "core" and "extended" analysis. Core analysis performs the heart of the audio processing, including audio sampling, the FFT calculation, and volume and frequency delta calculations. Extended analysis performs more specific functions that are typically one-offs and used only in one or two patterns.

Accessing Audio Features

Access of audio features, like frequency and volume, should be done through their respective global variables.

For most audio processing functions, Nanolux has a related function, often called something like "update_[FEATURE]." These functions are responsible for updating global variables, which then provide access points for audio features.

The benefit of this is that developers can spend less time worrying about pointers and properly freeing them, and instead just grab the exact values they need from an external variable.

If developing a new audio analysis function in ext_analysis.cpp, ensure the update function is ran inside of audio_analysis() in main.ino.

The following audio features can be accessed this way:

• volume (double)
• peak (double) (peak audio frequency)
• maxDelt (double) (frequency with largest delta between iterations)
• formants[3] (double)
• fbs (double) (five band split)

The only 2 features accessed differently are five-band-split and vowel detection. Five-band-split is partially dependent on the number of LED pixels that are on the segment of the LED strip, so any pattern that utilizes it must call update_five_band_split(int len) before accessing the associated global object. Vowel detection doesn't utilize pointers or variables (it returns an enum), so simply calling the vowel_detection() function will return the detected vowel.

If you want to develop your own audio processing function, the following objects and constants may be useful:

• vReal[]: After processing, contains the magnitude of each frequency "bucket." Each bucket contains a range of frequencies that correspond to an integer index.
• SAMPLES: An integer(int) of samples captured by the microphone which corresponds to the amount of data points captured every iteration by the Nanolux hardware. Must be some power of 2 between 128-1024.
• MAX_FREQUENCY: The maximum frequency detectable. Corresponds to the bucket at vReal[SAMPLES-1].
• MIN_FREQUENCY: The minimum frequency detectable. Corresponds to the bucket at vReal[0].

Additionally, the current pin used for analog input is defined as ANALOG_PIN, located in nanolux_util.h, along with some of the objects listed above.

Copying to a Global that is an Array

Some audio features, such as formants, require multiple values to be stored in an array. If your main analysis function returns a dynamically-allocated pointer:

// Create an array to store the formants
double* temp_formants = new double[3];

// Store the formants
temp_formants[0] = F0;
temp_formants[1] = F1;
temp_formants[2] = F2;

// Output the array containing each formant
return temp_formants;

You can utilize the temp_to_array function, which automatically performs the copying and deleting of the pointer.

/// @brief Used for moving a temporary pointer into
/// another pointer.
///
/// @param temp The temporary pointer to copy from.
/// @param arr  The global pointer to copy to.
/// @param len  The number of elements to copy.
///
/// Automatically deletes temp after copy is finished.
///
/// Intended to be used to copy temporary audio analysis
/// pointer data into global arrays.
void temp_to_array(double * temp, double * arr, int len){
  memcpy(arr, temp, sizeof(double) * len);
  delete[] temp;
}

/// @brief Moves data from the formant calculation
/// function to the global array.
void update_formants() {
  temp_to_array(density_formant(), formants, 3);
}