Classifiy the characteristics of numerical values with Keras/Tensorflow

Some problems are really complex (and maybe impossible!) to solve only with classical imperative programming. This short and simple project will explain, how easy and effortless these problems could be solved with machine learning methods, in particular with deep learning and a neural network.

Let’s assume, you get a series of numerical values and these values are representing a curve (or the fragment) one of these three basic geometric shape types:

Samples with visualized sample points (10 each sample)

You may think: “Oh that’s pretty easy: The upper left one is not a circle, it’s a parabola!”

But in most cases, you won’t get a beautiful plot, you will rather get a bunch of x/y values as a vector or a tensor which have to be processed very fast.

[-8.75110346699612,-0.9645611495889332,-11.2332236496635,-0.42621292318119675,-17.053561559907408,4.09121179786454,-26.212117197727828,12.587713013548267,-38.70889056312478,25.063290723870004,-54.54388165609825,41.517944928829735,-73.71709047664822,61.95167562842744,-96.22851702477477,86.36448282266319,-122.07816130047782,114.75636651153694,-151.26602330375735,147.12732669504862]

This is a parabola, too! ;)

Ok, let’s take a look at the “classical approach”. Identifying a straight line is not difficult: Calculate the slope at two different sample points. If the values are equal, it’s a line. But how to distinguish a parabola from a circle? Or a parabola from another unknown exponential function? I don’t know…

With the help of a simple neural network (NN), this challenge becomes way easier! The NN don’t need any plot image to “learn” about the characteristics of a mathematical function. It just needs training data. LOTS of training data!

(All source code used in this article is available in my github repo: https://github.com/jschw/Keras-Timeseries-Classification)

Creating a dataset for training and testing

Let’s start with creating some random sample values. For example, the function to create a parabola looks like this:

Basically, the following mathematical equations are written down as functions with optional parameters which can be loaded with random values.

Parabola:

y = a*x^2 + b*x + c

Circle:

x = r * cos(theta) + Center_x,
y = r * sin(theta) + Center_y

Line:

y = a*x + b

To get more variants, all curves can be rotated by a given angle parameter.

A few parabola samples can be easily created:

Three random parabola samples

After creating a lot of samples (1000 each class), the data is being packed and stored as a CSV file. Ten samples of each class are stored in a separate file for testing the NN after training with some new (unknown) data. Each line contains a classlabel. This is important for the training, because it’s supervised learning.

Preview of the csv file content

Data preprocessing and training the NN

The next step is, to read in this CSV file and put all the data to a dataframe:

Preprocess the data:

• Split the data into values (X) and labels (Y)
• Encode the label strings (one-hot-encoding is used)

Now we can build the model and train it.

The NN architecure visualized with Netron

With the Netron viewer (https://github.com/lutzroeder/netron), we can take a look into the neural network model architecture.

As you can see, the structure isn’t very complex, only five fully connected Dense layers alternating with a Dropout layer are used. The capacity of the NN is enough for this task (14.371 trainable parameters).

The input layer expects the shape (1,20) what’s equal to 10 sample points (each point includes 2 values, x and y).

Output is a vector with three values, each for one classes confidence value.

After the model was trained for 40 epochs, it reaches an impressive accuracy of nearly 98%! That’s a very good accuracy, but keep in mind, real-world values (e.g. measured by a sensor) won’t be that clean and accuracy will drop.

You probably want to test your trained model with some samples of the test dataset or other values. No problem, just load and encode the test dataset as shown above. Then choose a random or specific sample:

x_data and y_data are the separated x and y values of the curve in order to plot the curve.

The values in predict_arr can be fed directly into the model:

If everything works flawlessly, you will get an output like this:

Example output for a circle sample

The NN has predicted correctly a circle with a confidence of 99.65 % in 0.035 ms. Processing times may vary on different systems (depends on CPU power or if a GPU is installed).

Conclusion

This simple implementation of a neural network points out how new technologies like machine learning (and in particular deep learning) can extend common methods of data processing.

You may think, to classify a parabola or circle isn’t transferable on real-world problems. But YES, it is, of course! E.g. for recognising a specific movement or to identify a sensor value pattern which can indicate an imminent failure.

Usage of this deep learning model in an IoT device alongside an industrial PLC environment is shown in one of my other articles: Jetson Nano Communication with PLC