![\"\"](\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Newspaper-Cutting.jpg\")
<\/a>The Western Times cutting from 1902 Source: www.irishmirror.ie<\/p><\/div>\n
A lot of human knowledge and amusing facts, like this newspaper cutting, still\u00a0exist in non-digital formats. The job of converting such documents to digital text format, like MS word doc, is performed by optical character recognition (OCR) tools. Notably, such machine-encoded-text is much easier to classify, search, modify, store, and analyze. An OCR conversion of the same cutting is displayed here. Most OCR tools do a good job but they still make mistakes as we saw for our newspaper cutting. Here, nobody cares about ‘lenee-book’ (as this OCR tool deciphered ‘Face-book’). This tool,\u00a0notably, got 49 out of 52 words correctly. This was not an easy\u00a0task even for our eye on this more than a century-old cutting. In terms of individual characters, we have got 98.3% accuracy here with just 5 mistakes out of 288 characters.<\/p>\n OCR is also a source of excitement for data scientists since, using OCRs you could use non-digital data, such as application forms, electricity bills, receipts, etc.,\u00a0for your analysis.<\/p>\n Almost all OCR software tools in the market are\u00a0powered by deep learning models. In this article, we will learn how deep learning converts a newspaper cutting to a word document. This is similar to how your eyes see a book and convert the\u00a0ink marks into the familiar characters in your brain. Yes, we are entering the realm of artificial intelligence.<\/p>\n In this and the subsequent article, you will build models, and learn how to improve the accuracy of an OCR using machine learning (ML) and deep learning models. You will build four models with these accuracies on the test data.<\/p>\n Keep an eye for what is going behind the scenes for the error to reduce over 85% from logistic regression to CNN.\u00a0But before we start, let’s meet Popeye and see how he is similar to machine learning and deep learning models.<\/p>\n Popeye and Deep Learning Models – by Roopam<\/p><\/div>\n Popeye the sailor’s love-interest Olive Oyl is kidnapped by his arch-rival Bluto. He is no match for Bluto’s strength but Popeye has to save Olive Oyl. Popeye gobbles down a can of spinach\u00a0and gains superhuman-strength to beat Bluto the brute. Our hero is triumphant again! Popeye gets all his strength from spinach and nothing else. If he eats potatoes or tomatoes or anything else he will continue to be inferior to Bluto. Machine learning or deep learning models are no different.<\/p>\n Deep learning models are powered by the way they gobble data. The data have to be in a specific format (like spinach) for ML and DL models to\u00a0learn from them. Let’s try to understand\u00a0this using this picture. We see the number ‘2’ in this 2-dimensional image or 2D pixelated data. ML and DL models can’t digest 2D or higher dimension data. They get their strength, like just spinach, from the 1D data (a vector) displayed in the image.<\/p>\n This is a black-and-white image hence has just 2-dimensions (2D). A color image with red-green-blue (RGB) components is a 3D data. A color video is 4-dimensional with an additional time dimension. This higher dimension data is transformed to\u00a01D for ML algorithms to digest them. OK, so now that you are aware of this important requirement, you are ready to start your…<\/p>\n The process is exactly the same for hand-written text as well. Someone has done this hard job of converting images of many hand-written digits, as shown below, to digital format (0,1,..,9).\u00a0 Here, you are seeing 50 different images from the same large image dataset of 70,000 images. These images and corresponding digits are part of a popular and freely available dataset called MNIST.<\/p>\n These are all 28×28 pixel images.\u00a0Now, the machine learning algorithm knows that the first five images are of the digit ‘0’ and the next five are ‘1’s and so on. We will make the algorithm learn or train using any 60,000 of these 70,000 images. The remaining 10,000 images will then be the test set or unknown images for the algorithm. If the algorithm does a good job on the unknown test set then we have our first OCR application for hand-written numbers or digits.<\/p>\n An image is nothing but a matrix of numbers as displayed here. Here, 255 displays the most intense shade (black here) and 0 displays absence of that shade (i.e white in this case). Similarly, greys will be some numbers between these extremes.<\/p>\n Now, we can easily transform this matrix to flatten it to a single row with X1 to X15 as predictor variables and Y as the target. Y is tagged by humans after looking at the image. We will have 60,000 such rows with 28×28 = 784 predictor variables for our data. Now let’s use this data to build our first logistic regression model.<\/p>\n We have discussed logistic regression<\/a><\/strong> in great detail in YOU CANalytics earlier. Check out those articles. In this case, since the target variable has 10 categories (digits: 0,1,..,9), we will build 10 different logistic regression models to predict the probability of an image being any of these 10 digits. A single logistic regression model predicts the probability of one digit against the remaining nine digits.<\/p>\n Here, you can see the output of the logistic regression model for one of the test images.<\/p>\n Logistic regression predicts 97.8% probability of this image being ‘0’. The models did a good job for this image.\u00a0 However, the logistic regression models made mistakes for close to 800 test-images out of 10,000 with 92% accuracy. Here you see one such image with the wrong output. (image # 38 in the test set.)<\/p>\n Here, logistic regression is classifying this image as either 2, 3, or 8 with the same probability (33%). We can all easily see that this is ‘2’. You may also want to see other places where the model has made mistakes. This will be an interesting and enlightening exercise for you. Use this Python script to identify places where the model and humans don’t agree.<\/p>\n The next model we will build using random forest classifiers will correct some of these mistakes.<\/p>\n Random forest is a class of machine learning models that builds several decision trees from the same data. Decision trees, for a random forest, are built on randomly selected data from the training data. Notably, this makes the random forest equivalent to a democratic political system where every decision tree has a vote and voice. Read these articles to learn more about decision trees:<\/p>\n Random forest is also a relatively transparent model which explicitly highlights the important features or input variables. This enables us to ask questions such as, where do your eyes focus when they look at the MNIST images to decipher the digits hidden in them? Incidentally, the random forest model gives a good approximation for the same in this feature importance heat map. As expected, your eyes largely ignore the white spaces in the images and so does the random forest model (shown as cold ‘dark blue’ in the heat map). The areas or pixels that are lit-up i.e yellow, green and light blue, are the areas of important features or places where your eyes focus.\u00a0Random forest model improved the test set accuracy to close to 97%. This model has also correctly classified the image of ‘2’ mistaken by the logistic regression models.<\/p>\n Around 97% seemed like the limit for the test set accuracy for this MNIST dataset before deep learning and neural networks came to the\u00a0rescue. Notably, an elaborated and well-tuned deep learning model can improve the accuracy to ~99.5%. This is a whopping\u00a083% reduction in the error over the other popular machine learning models. Let’s enter the territories of deep learning models with…<\/p>\n Recall, we solved this neural network in the last part to understand the\u00a0math of deep learning<\/a>.\u00a0<\/strong>In this network, we had nine parameters to solve i.e. six weights (W1<\/sub>, W2<\/sub>,.., W6<\/sub>) and three bias terms (b1<\/sub>, b2<\/sub>, and b3<\/sub>). The network we will use for the images of hand-written digits is displayed below. This much more complicated network will require us to solve for 669,706 parameters. This network, first of all, has 784 input variables from 28\u00d728 pixel images. This network has 2 hidden layers with 512 nodes per hidden layers. Finally, there are 10 possible outputs (0,1,..,9). These outputs are then converted to probabilities using the softmax function in the last layer.<\/p>\n Let’s do a quick calculation to understand where these\u00a0669,706 parameters are coming from. There are 784 input nodes connected to 512 nodes in the first hidden layer. Additionally, the first bias term is connected to the same 512 nodes in the first hidden layer. This makes the total connections (or parameters to calculate) between the input layer and first hidden layers equal to:<\/p>\n Similarly, the first and the second hidden layers are connected by<\/p>\n Finally, 10 output nodes are connected to 512 nodes the in the second hidden layer and one bias node.<\/p>\n You will get the same information from the summary function of the network in Python \/ Keras, as displayed here.<\/p>\n These 669706 parameters must have given you a good idea why deep learning models are called ‘black box’ models. With this complicated network, it is absolutely impossible to understand the individual contribution of the 784 input variables to classify the images into digits.<\/p>\n Luckily for us, Tensorflow can easily calculate these weights using the same math we learned in the last part. This deep learning model gives the accuracy of ~98% on the test dataset. This confusion matrix shows that most of the times the deep learning model is right i.e. at the diagonal values. Let’s see where this model has made mistakes by considering more than 10 mistakes of similar kinds. The model has, on occasions, confused 9 with 3, 4 and 7. Also, it identifies 8 as 3 or 7. Moreover, 5 is also confused with 3 on a few instances.<\/p>\n Popeye gets his energy from spinach but spinach can be of different qualities as well. A good quality spinach can give higher strength or at least prevent food poisoning or a bad stomach. Similarly, there are ways to improve the quality of input data for deep learning models as well. We will learn the same in the next part when we will explore convolutional neural networks to improve the accuracy further.<\/p>\n","protected":false},"excerpt":{"rendered":" Facebook was a major sensation and a source of great amusement in a British\u00a0country house in the early 20th century. It was such a big hit that it got a special mention in a newspaper published in the year 1902. Facebook, then, of course, had a completely different meaning than the online social media we<\/p>\n<\/a><\/p>\nDeep Learning Models and OCR<\/span><\/h2>\n
\n
<\/a>Data for Machine Learning &\u00a0<\/span>Deep Learning Models<\/span><\/h2>\n
<\/a><\/p>\nMachine Learning and Deep Learning Models<\/span><\/h2>\n
<\/a>You can find the Python notebook with the entire code used in this article here:\u00a0YOU CANalytics – Machine Learning & Deep Learning Models<\/a>.<\/strong>\u00a0The first thing to build an OCR is to get image data from an enormous number of newspaper articles (like we saw earlier). The next task is for humans to read these articles and convert them to a digital format on a Notepad application. Machines will then learn using both the original articles and the digital format. When machines have seen enough such data and learned from them, they will be able to read an unknown document and convert it to the digital format.<\/p>\n<\/a><\/p>\nData Preparation for Machine Learning & Deep Learning Models<\/span><\/h2>\n
<\/a><\/p>\nModel 1: Logistic Regression<\/span><\/h2>\n
<\/a><\/p>\n<\/a><\/p>\n \r\nimport numpy as np\r\nimport matplotlib.pyplot as plt\r\nnp.where((Number_test!=Number_pred_LR))\r\n# Change the number in paranthesis [38] to other numbers with mismatches\r\nplt.imshow(Image_test[38].reshape(28,28), cmap = matplotlib.cm.binary)\r\nclassifier.predict_proba(Image_test[38].reshape(1,-1))<\/pre>\n
Model 2 – Random Forest Model<\/span><\/h2>\n
<\/p>\n\n
Model 3 – Deep Learning Neural Networks<\/span><\/h2>\n
<\/a><\/p>\n<\/a><\/p>\n784 \u00d7 512 + 512 = 401,408 + 512 = 401,920<\/pre>\n
512 \u00d7 512 + 512 =\u00a0262,656<\/pre>\n
512 \u00d7 10 + 10 = 5130<\/pre>\n
nn_mnist.summary() <\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
_________________________________________________________________\r\nLayer (type) Output Shape Param # \r\n=================================================================\r\ndense_1 (Dense) (None, 512) 401920 \r\n_________________________________________________________________\r\ndense_2 (Dense) (None, 512) 262656 \r\n_________________________________________________________________\r\ndense_3 (Dense) (None, 10) 5130 \r\n=================================================================\r\nTotal params: 669,706\r\nTrainable params: 669,706\r\nNon-trainable params: 0\r\n_________________________________________________________________\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
Deep Learning Model & Results<\/span><\/h2>\n
<\/a><\/p>\nSign-off Note<\/span><\/h4>\n