Predicting continuous target variables with Images — Demand Forecasting

‘Have you ever build a model that predicts continuous values with images as input? This article just gives a brief about how to train your model with X as images and Y as continuous numbers’

There are very few cases where you would want to predict a value in a continuous range of numbers based on an Image. Most of the Image Deepnet problems tend to be built for classifying the Images but there also stands a case for Regression based on Images. This article just gives a basic idea for beginners on how to train a model with Images and continuous variables as the target.

This remains as simple as you think it is. We can use the same ImageDataGenerator from Keras as how you do it for a normal classification problem based on Images. The trick i.e the logic is to change the last layer’s Activation function and the Loss function as per Regression modeling techniques.

Our output variable can be of any numeric range and not between 0 to 1 or -1 to 1. So the last activation layer you must choose ‘linear’ unlike classification problems (Sigmoid, softmax, etc..). The inner activation layers depend on your task and the size of the neural network. Based on my experience I would say we could use Tanh or ReLU for smaller neural networks and ReLU for bigger neural networks. It is always safe to use Relu if you don’t know what to choose. You can choose leaky−ReLU if you are facing a dying ReLU problem.

For Loss functions, you can use the most common ones for Regression Problems like Mean Square Error (MSE) or Mean Absolute Error(MAE). I would always calculate MSE as its understandable for developers and would calculate the Mean Absolute Percentage Error (MAPE) so that it is well understood by the business person or the end-customer.

Let us jump into the code and see for ourselves. So here is my code I am trying to predict the Rate of Sale (ROS) for the products based on its images. The ROS is calculated as per our business formula. ROS is one of the key factors to predict the demand for a product. The rate at which the product is being sold would help us find what would be the demand for the product for an upcoming week or a season. We would take images of the products and their corresponding ROS and will try to predict the ROS based on the image features that the neural network can learn. It can be pattern, texture, color, or any other complexity.

So here I am using transfer learning from VGG16 as my dataset has sufficiently fewer data. I am freezing the layers so that it becomes nontrainable and we can make use of the learned parameters directly.

Transfer Modelling

The subset of hyperparameters used are as follows,

Parameters

I have taken the transfer model up to the ‘block5_conv3’ layer. You can break up to any layer of your choice. You could select the layer by displaying the model’s summary and choosing the layer that you want to stop with. The optimizer we will be using is RMSprop. Note that I have used ‘Linear’ in my last layer which would get me continuous value range output.

Freezing the layers

Now if you check the summary of the model it would look something like this.

Model Summary

Let us now code up the Train, Validation, and the Test generators. You can feel free to adjust the hyperparameters according to your results. Data augmentation surely does impact your training and would help you avoid overfitting and efficiently minimize loss.

So things that you will notice in the below code are as follows:

1. We would use the flow_from_dataframe for this case.
2. The image paths of the training images should be computed as a column and saved.
3. You should give x_col as the column name which has image paths and y_col to the column that you wanted to predict. In my case it is ROS.
4. In the test_generator you should give y_col = None so that we could predict that values after the model is generated.

Generators

The output of the above code

Now you could fit the model and wait for the model to train. It would usually take time as we are dealing with Image Deepnets. I am using a P2.xLarge GPU Instance from AWS/ The performance is better than training from CPU and I would get the epochs done faster. The code has run for just 10 epochs and let us see the results we have obtained.

Epochs

The model looks decent and it has trained well for a 10 epoch fit. You should always try out with different hyperparameters to get the best values.

Loss plot

So this model gave me the following scores which are not bad. It could get better with further testing and training. But yes, the whole idea is to just bring into notice that this is as simple as predicting a classification problem. There are many efficient methods to predict continuous values from Images or a hybrid model of Text and Images but we shall start with something as basic as this.

MSE : 0.1882

RMSE : 0.4339

MAPE : 34.47%

I hope you are there closer to your goal than you were yesterday. I would appreciate some claps for better reach and please feel free to comment on the methods that you follow and have come across. Thanks!

‘Make sure you are better than you were yesterday!’

GitLink : https://github.com/Vickyilango/Predicting-continuous-variables-with-Images

Please feel free to connect on any further discussions:
LinkedIn : https://www.linkedin.com/in/vigneshwarilango/
Gmail: mr.vigneshwarilango@gmail.com

Regards,
Vigneshwar Ilango