The debate of preferring deep learning over machine learning is going on for a while among data scientists, especially after Google made the tensor-flow as an open source in 2016. Deep learning algorithms are out performing the machine learning approaches in some imaging applications like face recognition and object detection. Deep Learning (DL) algorithms are like black boxes, whereas the Machine Learning (ML) algorithms are like art. One needs to be innovative to handcraft the features from the images to build the ML models.

Deep learning algorithms need more data(images) to learn the patterns from it, but in many domains, it is not easy to acquire more data. Whenever there is not enough data, we can use a machine learning approach. In ML approach, a programmer needs to perform feature extraction. For example, say there are tumor images, in ML approach the job of the programmer is to extract the numeric features like, area, diameter, volume, smoothness, edge patterns etc., from the tumor images. To extract those features, one should have both programming and domain knowledge. In Deep Learning approach we will just feed more and more tumor images to the model and the features will be learned by the model on its own. The role of the programmer is minimum in the case of Deep Learning as the feature extraction process is not explicit.In this article we are going to take you through a step by step approach on implementing image processing case study through ML approach. The example that we are going to discuss in this article is about predicting the malignancy of the nodules(tumor) present inside the lung region.

The diagnosis of lung cancer at an early stage is critical and uncertain. The lung is a sponge kind of tissue anatomy. The process of taking a tissue from the lung is known as a biopsy, to examine its cancerous nature is a painful process and taking a tissue accurately from a small abnormal tissue cluster inside a lung is a challenging task. Physicians will not recommend  biopsy without reliable evidence for lung cancer. Physicians analyse the CT scan of a patient and if they suspect any symptoms of lung cancer, direct the patient to undergo one more CT scan after a time span of 3,6,9,12 or 18 months based on patient smoking habits and environmental conditions. After analyzing CT images at different intervals and based on the level of disease progression, a biopsy is performed.

The qualitative analysis performed by the physicians may vary from one expert to another due to human factors and the huge number of images that they need to analyse, which is time consuming and requires a trained eye. Hence, it has become essential to develop an intelligent computerized model to analyse the CT scans for identifying the malignancy in present. The objective of this case study is to segment the region of interest from the image and extract the features to build a machine learning model to detect the malignant nature of lung from CT scan images.

The database provided for this work consists of data from 50 patient CT scans. Each CT scan has 60 to 250 cross sectional CT images in it. The small white tissue clusters inside the lung parenchyma region is known as ‘nodule’. These nodules may be the potential indicator for lung cancer (but most of the nodules are generally benign !!!).MACHINE LEARNING APPROACH

We know that the machine learning algorithm needs the tabular numeric data as an input. We need to extract the features from all the suspected nodules from the lung parenchyma region. Our region of interest to diagnose the cancer is only the nodules present inside the lung, not the whole lung region. Therefore, the first step we need to perform is to segment this region of interest from the CT scan. The following section of the script will load the image and perform segmentation. There are many segmentation techniques reported in literature. In this work we used the OSTU threshold-based segmentation technique. 

import cv2

import matplotlib.pyplot as plt

import numpy as np

import time

img=cv2.imread(‘lung_img.jpg’,0)

cv2.imshow(“Lung Image”,img)

th,ostu_img = cv2.threshold(img,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)

In the above figure, along with nodules other tissue regions are also segmented as white regions. Morphological processing will be performed to remove those surrounded white regions. Different morphological techniques like erosion, dilation, inversion, opening, closing, filling etc., can be performed to get the region of interest alone from the segmented images. Choice of appropriate morphological technique is based on the output of the initial segmentation process. In this example case, we have used the clear-border morphology to remove all the unwanted white regions and keep only the nodules.

from skimage.segmentation import clear_border

img2=clear_border(ostu_img)

cv2.imshow(“clear_border”,img2

Also, we can perform different morphological operations to get the lung masks and the nodules as shown below:

img3=255-ostu_img

cv2.imshow(“Inverted_image”,img3)

 

img4=clear_border(img3)

cv2.imshow(“clear_border1”,img4)

 

se_fill=cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(21,21))

img4_fill = cv2.morphologyEx(img4, cv2.MORPH_CLOSE, se_fill)

cv2.imshow(“filled_image”,img4_fill)

 

se_open=cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(21,21))

img4_open = cv2.morphologyEx(img4_fill, cv2.MORPH_OPEN, se_open)

cv2.imshow(“Open_image”,img4_open)

 

paren=img & img4_open

cv2.imshow(“Parenchyma”,paren)

 

thos,nod_th = cv2.threshold(paren,100,255,cv2.THRESH_BINARY)

cv2.imshow(“Nodules”,nod_th)

 

se_open=cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))

nodules = cv2.morphologyEx(nod_th, cv2.MORPH_OPEN, se_open)

cv2.imshow(“Final_Nodules”,nodules)

After segmenting all the nodules from the lung CT images, different features need to be computed for all the nodules. Generally, features will be computed on three platforms: 1. Shape 2. Texture and 3. Colour. In this work we computed the shape based features.

nlabels, labels, stats, centroids = cv2.connectedComponentsWithStats(nodules)

cv2.imshow(“Final_Nodules”,nodules)

sizes = stats[1:, -1];

min_size = 15

nodules1 = np.zeros((labels.shape),dtype=’uint8′)

for i in range(0, nlabels-1):

    if sizes[i] >= min_size:

        nodules1[labels == i + 1] = 255 

cv2.imshow(“Final_Nodules1”,nodules1)

nlabels1, labels1, stats1, centroids1 = cv2.connectedComponentsWithStats(nodules1)

#pos1=np.where(stats[3]>10)

#nod1=np.zeros([512,512],dtype=’uint8′)

all_nod=[]

feat=np.zeros([nlabels1-1,9],dtype=’float’)

for i in range(1,nlabels1):

    nod1=np.zeros([512,512],dtype=’uint8′)

    pos=np.where(labels1==i)

    nod1[pos]=nodules1[pos]

    cv2.imshow(“nod1”,nod1)

    all_nod.append(nod1)

    

    x,y,w,h = cv2.boundingRect(nod1)

    feat[i-1,0] = float(w)/h #Aspect Ratio

    cc, hierarchy = cv2.findContours(nod1.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)

    area = cv2.contourArea(cc[0])

    feat[i-1,1]=area

    rect_area = w*h

    feat[i-1,2]= float(area)/rect_area #Extent

    hull = cv2.convexHull(cc[0])

    hull_area = cv2.contourArea(hull)

    feat[i-1,3]=hull_area #hull area

    feat[i-1,4] = float(area)/hull_area #solidity

    feat[i-1,5] = np.sqrt(4*area/np.pi) #equi_diameter

    [(x,y),(MA,ma),angle] = cv2.fitEllipse(cc[0])

    feat[i-1,6] =angle

feat[:,7]=centroids1[1:,0]

feat[:,8]=centroids1[1:,1]

Nine shape features for 18 segmented nodules from one CT scan are extracted after executing the above script.

Nodules Aspect Ratio Area Extent Hull

area

Solidity Equi-diameter Angle Centroid_X Centroid_Y
Nodule 1 1 30.5 0.376543 35.5 0.859155 6.23168 150.249 279.659 189.854
Nodule 2 0.348315 788.5 0.285792 1845 0.427371 31.6852 159.649 335.058 248.16
Nodule 3 0.5 11 0.34375 13.5 0.814815 3.74241 13.3688 358.421 212.526
Nodule 4 0.9375 1041.5 0.271224 2109.5 0.493719 36.4154 141.711 190.555 264.887
Nodule 5 0.7 42 0.6 43.5 0.965517 7.31273 13.31 203.741 239.519
Nodule 6 0.833333 11.5 0.383333 12 0.958333 3.82652 139.736 216.5 244.611
Nodule 7 0.833333 71.5 0.595833 74 0.966216 9.54131 156.121 323.244 247.081
Nodule 8 0.625 326.5 0.3265 614 0.531759 20.389 159.57 229.711 286.905
Nodule 9 1.5 29.5 0.546296 31 0.951613 6.12867 81.095 166.575 277.8
Nodule 10 1.28571 22 0.349206 29 0.758621 5.29257 119.901 255.094 278.969
Nodule 11 0.5 52 0.530612 55.5 0.936937 8.13686 18.0453 175.104 296.403
Nodule 12 1 11.5 0.319444 11.5 1 3.82652 45 333.722 292.722
Nodule 13 0.666667 65.5 0.303241 99 0.661616 9.13221 162.512 365.172 302.483
Nodule 14 0.702703 484.5 0.503638 576.5 0.840416 24.8372 9.18256 159.629 318.956
Nodule 15 1 21 0.259259 24 0.875 5.17088 136.903 384.129 317.387
Nodule 16 0.714286 18 0.514286 18.5 0.972973 4.78731 20.5219 320.923 357.962
Nodule 17 0.833333 14 0.466667 14.5 0.965517 4.22201 28.568 336.905 371.476
Nodule 18 1.75 11 0.392857 12 0.916667 3.74241 110.04 359 370.5

 

Out of these 18 nodules, nodule 14 has been detected as cancerous by the medical expert. Hence, the corresponding nodule has been labelled as ‘1’ while the other nodules have been labelled as ‘0’. 

Similarly, the nodule segmentation and feature extraction process can be carried out on different patient’s scan images and by concatenating all this information we can form a structured supervised data-set, on which any machine learning classification algorithms can be applied to build a predictive model. 

The skill of deriving the structured data from the un-structured data is the vital step to build the predictive model. We hope this article gives you the hand-on experience of converting the images in to structured data.

SHARE