AI For Chest X-Rays Made Easy!

Neural Networks are all the rage in Machine Learning right now, with many different ways to create them. When starting out, it can be difficult to create a working neural net, so this tutorial is going to teach you how to get one working in ninety seconds (yes, actually ninety seconds!)

Getting Started

We’re going to be using a system called Teachable Machine, which is a website setup by Google to create Neural Networks quickly. It uses a method called Transfer Learning to retrain an existing neural network to suit the dataset that you’re looking to train it on.

Transfer Learning is an incredibly powerful technique because the majority of the network has already been taught to recognise key features that we have in the world e.g. lines, curves, eyes, arms etc. which makes it easy to add some specialist training on top. It’s basically the equivalent of teaching a 10 year old how to say ‘staphylococcus’ (as with Transfer Learning) vs trying to teach a baby how to say ‘staphylococcus’ (as with training a neural network from scratch). Both work (eventually…), but transfer learning is quicker, and in some cases produces better results!

Dataset

We’re going to redo an old tutorial and create a Chest X-Ray (CXR) classifier from the dataset available here: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia

Go ahead and download the dataset, and unzip. There should be 3 folders inside, test, train, and val. This is the best practice layout of a dataset for machine learning. When we train the network, we use the train folder, with the images inside there. Teachable Machine does it slightly differently, but when making a neural network from scratch, we then use the test folder to give it some images and evaluate its performance to see how well it does.

Pre-Processing

The images in the dataset are CXRs which have all been pre-processed to be the same size. Pre-processing is an important concept in machine learning in general, as the data that we have can often come in different shapes and sizes. Pre-processing means that we ensure the network receives a standardised set of set (e.g. if we have data about the dimensions of a cell, we get ALL the dimensions we need e.g. height, width, and depth, and we don’t use data with missing points). This is important to ensure the network has the best possible performance. It’s also a fine balance as processing too much can cause us to lose features which we may think are not important, but the network does, e.g. orientating all images to be upright may lead to a network which only works with images in the upright orientation rather than in all orientations.

Teachable Machine

Now that we’ve got the images downloaded, we can fire up the Teachable Machine. Go to https://teachablemachine.withgoogle.com and click “Get Started”, then click on Image Project

You’ll be presented with this dashboard – and this is where we’ll train the neural network.

On the left, we can input the images and the category that they belong to – in our case, the two categories are normal and pneumonia. Let’s change the labels to be normal and pneumonia (click the pencil icon to edit), and then upload the images from the train folder in the dataset.

Once done, it should look something like this:

We then click Train Model and watch the magic unfold. Because this magic is slightly complicated and involves a lot of calculations by the computer, it can take a few minutes to run. NOTE – do not switch tabs or the model won’t train!

If you want to modify the hyper-parameters of the model, you can click Advanced and change them here – we’d advise avoiding this for the first time so you can see the baseline performance first, and then play around to see which gives the best performance!

In the meantime, let’s learn a bit more about Transfer Learning and how it works.

Transfer Learning

Transfer Learning consists of using a pre-trained model which has been optimised to work on problem X, and refining the last part of the model such that it is able to work on problem Y (with Y being an arbitrarily large number of problems).

The key benefits of Transfer Learning are twofold:

1. Reduced time to train a model
2. Better accuracy per time trained (and in many cases, better accuracy overall)

The benefits are due to the network understanding the basics of “life”, e.g. what a line is, what a curve is, what an eye looks like. All things that a young child would learn over the period of a few years. Transfer learning then takes this knowledge, and specialises it to focus on more task-specific problems. In our case, this is “what does an abnormal chest x-ray look like”. Once a network has learnt to see basic shapes and understand what they represent then they are able to generalise and solve larger issues.

This is the reason why transfer learning has seen a rise in popularity in the last few years to become a useful method of creating networks that train quickly, and which perform well on small datasets (as the initial dataset to create the general network and instil the essence of basic things is quite large!).

Testing the Network

Now it’s finished training, let’s test it out! Set input to be on, and change the type to be file. Then upload any random image from the test folder that we downloaded (choose whether it’s normal or pneumonia and see what the network predicts!). Don’t be disheartened if the network doesn’t predict right the first time, test it out on a couple and see how well it does.

Model Metrics

We can see how well the model performed using the metrics that Teachable Machine provides. To find them, click on Advanced, and then Under the hood. Click calculate accuracy per class and calculate confusion matrix, and the network will analyse it’s performance on the dataset. On our run, we got an accuracy of 0.96 (aka 96% accurate) for normal CXRs, and 0.97 for pneumonia CXRs.

The confusion matrix is a method by which we can see the false positive and false negative rate for our network. We had 8 false positives (1.02% of normal CXRs reported as pneumonia), and 16 false negatives (2.04% of pneumonia CXRs reported as normal). Whilst this isn’t huge, it does lead to a lot of false results when scaled to thousands of images. So whilst this network is useful as a screening tool (and fun project!), it’s not 100% reliable for diagnostics.

Summary

Teachable Machine is a fantastically easy way to get started with neural networks, quickly! It takes a lot of the complexity out of the equation and allows you to prototype with a network easily, and with admirable results. There’s a lot you can do with Teachable Machine, and we’re going to cover it with more articles in the future.