# Transfer Learning — A Case Study

I have been fascinated by transfer learning, a technique in machine learning where we leverage pre-trained models that can be retrained for a different class of input. It is a very popular way to harness the power of deep learning without having to go through the full process of training the network from scratch. Furthermore, it is not always possible to train a network from scratch due to lack of labeled data and/or infrastructure required to do so.

In this post I’ll go over a case study in which I evaluated transfer learning for a problem where I would normally use different techniques. In particular, I wanted to see if transfer learning could be used to classify frequency content of time domain signals. As humans, we have no problem looking at the screen of spectrum analyzers and inferring basic patterns of frequency content in the input signal. So, at least in theory, it should be possible to throw spectrums (as images) to a deep learning model and have it learn different classes in the input data.

## Problem Definition

Imagine there are devices which output time domain signals as shown below. These signals are plotted with X-axis representing time and Y-axis representing amplitude. The signals shown in red v/s those shown in blue differ ever so slightly from each other as a result of device manufacturing variability. Signals in red have slightly different frequency content than those in blue and we need an automatic way to classify that. So let’s do that using deep learning!

## Signal Generation

The signals shown were artificially generated for this case study. There are three frequencies involved and there is random noise added everywhere in both the frequency, phase and amplitude. For those of you who like to see code, take a look at code snippets (Go code) below to get idea of how these signals were constructed.

**for **i := **range **theta1 {

signal[i] = 10*math.Sin(theta1[i]+phase1) +

5*math.Sin(theta2[i]+phase2) +

math.Sin(theta3[i]+phase3) +

3*rand.Float64()

}

where `theta1`

, `theta2`

, `phase1`

, etc. are defined as:

`theta1, err := op.Linspace(0, float64(numCycles+rand.Intn(5))*math.`*Pi*, n)

**if **err != nil {

**return **nil, nil, err

}

and `phase1 := rand.Float64 *2 * math.Pi`

.

`op.Linspace`

is a higher level function to generate linearly spaced data points.

## Frequency Domain

Now let’s compare signals in frequency domain. Performing Fourier Transform or `FFT`

on the input signal produces following signature in the frequency domain. As you can see the red v/s blue signals can now be clearly separated with blue signals exhibiting slightly higher frequency content in each of the three frequency bands.

## FFT Computation in Go

I computed `FFT`

using an operator built using TensorFlow. For those interested in building software systems with TensorFlow as ML engine and Go (Golang) as the programming paradigm, it is possible to build wrappers on top of TensorFlow API. For building FFT operator, we can start with Python code to build the TensorFlow graph:

`import tensorflow as tf`

my_fft_input = tf.placeholder(tf.complex128, name="myFFTInput")

tf.spectral.fft(my_fft_input, name="fft")

Then export the graph as a protobuf file:

`graph = tf.Session().graph_def`

tf.io.write_graph(graph, "./model", "graph.pb", as_text=False)

Such graph can be imported in Go and executed by feeding data to it at runtime. I won’t go into details of interfacing TensorFlow with Go in this post as I covered some of these details in another post. I’ll just mention that eventually we have an interface in Go with following signature allowing us to perform `FFT`

on a slice of `float64`

data.

**type **Operator **interface **{

// FFT obtains discrete fast fourier transform over input

FFT(input []float64) ([]float64, error)

}

## Input For Transfer Learning

Now that we know we can tell signals apart in frequency domain, let’s prepare these spectral plots as input images for transfer learning. Our input will look as follows. We will have several images per class.

It is important to note that image classification does not know about the concept of frequency content of a signal, so in order to make this exercise meaningful, the input data needs to be controlled in the form of plot preparation. In particular, transfer learning will likely “learn” the location of spectral peaks in the plot, so it is necessary to prepare all plots with exactly the same scale. We are essentially mapping frequency scale onto image width.

Furthermore, it is necessary to not color these images differently because we don’t want the network to learn the differences in these signals based on the color. So what we feed to the training should would look as follows. Top three images belong to one of the classes and bottom three belong to the other.

## Training

TensorFlow community has done a great job enabling transfer learning for custom training. Please see link below to understand how to train on your data.

Training is as simple as running this command:`python retrain.py — image_dir=<folder_with_images>`

The model trains fairly quickly and shows following progress on TensorBoard. It seems it worked! The output is another graph that can be used to make inferences on new data of the same type.

## Summary

The objective was to evaluate transfer learning method for the special case of classifying signals based on their frequency content. We prepared our input in the form of `jpg`

images showing spectrum for each of our time domain signals. Furthermore, the code to generate signal and perform `FFT`

was written in Go using TensorFlow as a backend ML engine.

The model used was `Inception V3`

trained on `Imagenet`

data, which we retrained to identify classes in our dataset using transfer learning. The result was another graph that can now be used to make inferences.

## Coastal Bluffs of California

California coastline is beautiful and shows lots of geological features such as these bluffs abruptly rising on the coastline. I captured the featured image just south of Half Moon Bay in the San Francisco Bay Area. The clouds were pretty awesome and I made a long exposure shot to show motion of the waves.