{"id":11583,"date":"2019-03-31T10:14:03","date_gmt":"2019-03-31T04:44:03","guid":{"rendered":"http:\/\/ucanalytics.com\/blogs\/?p=11583"},"modified":"2019-03-31T10:55:07","modified_gmt":"2019-03-31T05:25:07","slug":"convolutional-neural-networks-cnn-simplified-part-4","status":"publish","type":"post","link":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/","title":{"rendered":"Convolutional Neural Networks (CNN) Simplified (Part 4)"},"content":{"rendered":"<div style=\"width: 785px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg\"><img data-attachment-id=\"11593\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-networks-cnn-filters-polaroid-glasses-by-roopam\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&amp;ssl=1\" data-orig-size=\"775,581\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Networks (CNN) Filters &#038; Polaroid Glasses &#8211; by Roopam\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;Convolutional Neural Networks (CNN) Filters &#038;Polaroid Glasses &#8211; by Roopam&lt;\/p&gt;\n\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=300%2C225&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=640%2C480&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-11593\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?resize=640%2C480\" alt=\"\" width=\"640\" height=\"480\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?w=775&amp;ssl=1 775w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?resize=250%2C187&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?resize=300%2C225&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?resize=768%2C576&amp;ssl=1 768w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><p class=\"wp-caption-text\">Convolutional Neural Networks (CNN) Filters &amp; Polarized Glasses &#8211; by Roopam<\/p><\/div>\n<p>Welcome back to the <strong><a href=\"http:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/\" target=\"_blank\" rel=\"noopener noreferrer\">deep learning example to build an OCR application<\/a><\/strong>. The idea of this simple application is to identify numbers in an image of written text. In the last part, we used three different models and got the following accuracy for identification of the test images:<\/p>\n<ul>\n<li class=\"first-child\"><strong>Model 1<\/strong> &#8211; Logistic regression: 92% accuracy<\/li>\n<li><strong>Model 2<\/strong> &#8211; Random forest: 96.8% accuracy<\/li>\n<li><strong>Model 3<\/strong> &#8211; Deep learning network (fully connected): 97.9% accuracy<\/li>\n<\/ul>\n<p>In this part, we will raise the accuracy further to 99.1% by using convolution neural networks (CNN). But before that, let&#8217;s visit a beach and sunbathe a bit.<\/p>\n<h2><span style=\"color: #3366ff;\"><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg\"><img data-attachment-id=\"11781\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/beach-and-polarized-glasses-convolutional-neural-networks\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?fit=514%2C481&amp;ssl=1\" data-orig-size=\"514,481\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Beach and Polarized Glasses &#8211; Convolutional Neural Networks\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?fit=300%2C281&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?fit=514%2C481&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"wp-image-11781 alignright\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?resize=211%2C197\" alt=\"\" width=\"211\" height=\"197\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?w=514&amp;ssl=1 514w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?resize=250%2C234&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Beach-and-Polarized-Glasses-Convolutional-Neural-Networks.jpg?resize=300%2C281&amp;ssl=1 300w\" sizes=\"(max-width: 211px) 100vw, 211px\" data-recalc-dims=\"1\" \/><\/a>Convolution Neural Networks and Polarized Glasses<\/span><\/h2>\n<p>Beaches are great places to vacation but don&#8217;t forget your polarized glasses when visiting a beach. Polarized glasses cut down the bright sunlight to half still allowing you to see the beautiful views properly. Essentially, you are seeing the same thing, possibly better, with half the information. The science behind polarized glasses is simple where the sun emits unpolarized light i.e. electromagnetic waves in all directions. The polarized lens, as in this diagram, blocks or filters out all the components of the sunlight except the vertical component.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg\"><img data-attachment-id=\"11791\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/polarized-lens-convolutional-neural-networks\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?fit=1320%2C304&amp;ssl=1\" data-orig-size=\"1320,304\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Polarized Lens &#038; Convolutional Neural Networks\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?fit=300%2C69&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?fit=640%2C148&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11791\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?resize=640%2C147\" alt=\"\" width=\"640\" height=\"147\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?w=1320&amp;ssl=1 1320w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?resize=250%2C58&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?resize=300%2C69&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?resize=768%2C177&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Polarized-Lens-Convolutional-Neural-Networks.jpg?resize=1024%2C236&amp;ssl=1 1024w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>In theory, a polarized lens should block exactly 50% of the visible light however, in reality, they usually block anything between 42-47% visible light. Moreover, physical limitations allow just some specific shades or filters for polarized glasses.<\/p>\n<p>Convolutional neural networks work similar to polarized lenses where they reduce the information of an image without losing the meaning. Moreover, convolutional neural networks don&#8217;t have physical limitations like polarized lenses hence you can create virtually infinite types of filters.<\/p>\n<p>This polarized lens kind of operation is performed in two steps in convolutional neural networks. These steps are called convolutional and max-pooling. These two steps are applied multiple times to reduce the information load of an image without losing its meaning. Let&#8217;s see how these two steps work.<\/p>\n<h2><span style=\"color: #3366ff;\">Convolution filters and Layers<\/span><\/h2>\n<p>In the previous article on <strong><a href=\"http:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/\" target=\"_blank\" rel=\"noopener noreferrer\">deep learning for OCR<\/a><\/strong>, we learned that an image is nothing but a matrix of numbers. For Instance, this logo of YOU CANalytics has three matrices with 120 rows and 607 columns each. The three matrices are for the red, blue, and green channels (RGB). Essentially, this image is nothing but an arrangement of 218520 numbers (120 x 607 x\u00a0 3).<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg\"><img data-attachment-id=\"11800\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/you-canalytics-logo-deep-learning-convolutional-neural-networks\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?fit=900%2C271&amp;ssl=1\" data-orig-size=\"900,271\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"YOU CANalytics logo &#8211; Deep Learning Convolutional Neural Networks\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?fit=300%2C90&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?fit=640%2C193&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter  wp-image-11800\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?resize=432%2C130\" alt=\"\" width=\"432\" height=\"130\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?w=900&amp;ssl=1 900w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?resize=250%2C75&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?resize=300%2C90&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Deep-Learning-Convolutional-Neural-Networks.jpg?resize=768%2C231&amp;ssl=1 768w\" sizes=\"(max-width: 432px) 100vw, 432px\" data-recalc-dims=\"1\" \/><\/a> Now, the polarized lens of convolutional neural networks i.e. convolutional filters is a matrix. Here, these 3 x 3 matrices correspond to images filters shown below.<a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg\"><img data-attachment-id=\"11801\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-filters\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?fit=1776%2C624&amp;ssl=1\" data-orig-size=\"1776,624\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network Filters\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?fit=300%2C105&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?fit=640%2C225&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11801\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?resize=640%2C225\" alt=\"\" width=\"640\" height=\"225\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?w=1776&amp;ssl=1 1776w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?resize=250%2C88&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?resize=300%2C105&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?resize=768%2C270&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?resize=1024%2C360&amp;ssl=1 1024w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Filters.jpg?w=1280 1280w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a> These 3 x 3 filters, when they traverse through the original image, produce effects such as detection of edges, sharpening or blurring the image. Here, you can see an example of a 3 x 3 filter traversing through a 5 x 5 image matrix to produce a filtered image. Note, the convolutional neural network learns each element of the filter matrix on its own, unlike these predefined filter matrices.<\/p>\n<div id=\"attachment_11805\" style=\"width: 536px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/CNN-Deep-Learning.gif\"><img aria-describedby=\"caption-attachment-11805\" data-attachment-id=\"11805\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/cnn-deep-learning\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/CNN-Deep-Learning.gif?fit=526%2C384&amp;ssl=1\" data-orig-size=\"526,384\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"CNN &#8211; Deep Learning\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;Source: researchgate.net&lt;\/p&gt;\n\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/CNN-Deep-Learning.gif?fit=300%2C219&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/CNN-Deep-Learning.gif?fit=526%2C384&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-11805\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/CNN-Deep-Learning.gif?resize=526%2C384\" alt=\"\" width=\"526\" height=\"384\" data-recalc-dims=\"1\" \/><\/a><p id=\"caption-attachment-11805\" class=\"wp-caption-text\">Source: researchgate.net<\/p><\/div>\n<p>A convolved feature usually has fewer dimensions than the original image i.e. a 5 x 5 image became 3 x 3 here. In practice, images are padded up a bit to produce convolved features with the same dimensions as the original image i.e. a 5 x 5 image will produce a 5 x 5 convolved feature. The padding is essentially adding zeros across the 5 x 5 image to make it a 7 x 7 image and then the convolved image will have 5 x 5 dimensions.<\/p>\n<p>Now, let&#8217;s run the edge detection filter across the YOU CANalytics logo and see what happens. You could find the Python code used for this section here: <strong><a href=\"http:\/\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU%20CANalytics%20logo%20Convolution%20and%20Maxpool.html\" rel=\"noopener noreferrer\">Convolution and MaxPooling using OpenCV (Jupyter Notebook).<\/a><\/strong><\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg\"><img data-attachment-id=\"11814\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/you-canalytics-logo-convolution\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?fit=1988%2C297&amp;ssl=1\" data-orig-size=\"1988,297\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"YOU CANalytics logo &#8211; Convolution\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?fit=300%2C45&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?fit=640%2C96&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter  wp-image-11814\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?resize=621%2C93\" alt=\"\" width=\"621\" height=\"93\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?w=1988&amp;ssl=1 1988w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?resize=250%2C37&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?resize=300%2C45&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?resize=768%2C115&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?resize=1024%2C153&amp;ssl=1 1024w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?w=1280 1280w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution.jpg?w=1920 1920w\" sizes=\"(max-width: 621px) 100vw, 621px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>The edges are enhanced in the output while the rest of the image has muted down i.e. it became black. The output matrix is still the same size (120 x 600 x 3)<\/p>\n<p>Remember, our goal for this entire operation is to reduce the information load without losing the meaning in the image. After applying the convolutional layer with padding, we got a matrix with the same dimension as the original image hence we have not reduced the information. The max-pooling layer in the next segment will do the job of reducing the information.<\/p>\n<h2><span style=\"color: #3366ff;\">Max-Pool Layer<\/span><\/h2>\n<p>This video gives a glimpse into the intuition behind the max-pooling operation. Please note, this is not exactly how max-pooling work.<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe loading=\"lazy\" class=\"youtube-player\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/f1fXCRtSUWU?version=3&#038;rel=1&#038;showsearch=0&#038;showinfo=1&#038;iv_load_policy=1&#038;fs=1&#038;hl=en-US&#038;autohide=2&#038;wmode=transparent\" allowfullscreen=\"true\" style=\"border:0;\" sandbox=\"allow-scripts allow-same-origin allow-popups allow-presentation\"><\/iframe><\/span><\/p>\n<p>Here, a smaller dimension image preserves the information within the larger dimension image. Max-pooling is a very simple matrix operation where you take the maximum value of the smaller blocks (say 2 x 2) of a large convolved feature matrix.<\/p>\n<div id=\"attachment_11810\" style=\"width: 760px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png\"><img aria-describedby=\"caption-attachment-11810\" data-attachment-id=\"11810\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/maxpool\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?fit=750%2C313&amp;ssl=1\" data-orig-size=\"750,313\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Maxpool\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?fit=300%2C125&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?fit=640%2C267&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"wp-image-11810 size-full\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?resize=640%2C267\" alt=\"\" width=\"640\" height=\"267\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?w=750&amp;ssl=1 750w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?resize=250%2C104&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Maxpool.png?resize=300%2C125&amp;ssl=1 300w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><p id=\"caption-attachment-11810\" class=\"wp-caption-text\">Source: Wikipedia.com<\/p><\/div>\n<p>Max-pool, as you must have noticed with the 2 x 2 block, reduces the size of the to the square root of the original image. Now, let&#8217;s notice what happens to the YOU CANalytics logo when you apply the max-pool operator.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg\"><img data-attachment-id=\"11826\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/you-canalytics-logo-max-pool-operation\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?fit=1975%2C313&amp;ssl=1\" data-orig-size=\"1975,313\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"YOU CANalytics Logo &#8211; Max Pool Operation\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?fit=300%2C48&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?fit=640%2C101&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11826\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?resize=640%2C101\" alt=\"\" width=\"640\" height=\"101\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?w=1975&amp;ssl=1 1975w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?resize=250%2C40&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?resize=300%2C48&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?resize=768%2C122&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?resize=1024%2C162&amp;ssl=1 1024w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?w=1280 1280w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-Logo-Max-Pool-Operation.jpg?w=1920 1920w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>The original 120 x 607 x 3 image is reduced to one-fourth its size i.e. 60 x 303 x 3. Max-pool operator, however, in the convolutional neural network is almost always applied after the convolutional layer. Hence, the process of reduction of the original image for a CNN like operation will look something like this.<img data-attachment-id=\"11813\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/you-canalytics-logo-convolution-and-max-pool\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?fit=1217%2C772&amp;ssl=1\" data-orig-size=\"1217,772\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"YOU CANalytics logo &#8211; Convolution and Max Pool\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?fit=300%2C190&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?fit=640%2C406&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11813\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?resize=640%2C406\" alt=\"\" width=\"640\" height=\"406\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?w=1217&amp;ssl=1 1217w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?resize=250%2C159&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?resize=300%2C190&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?resize=768%2C487&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU-CANalytics-logo-Convolution-and-Max-Pool.jpg?resize=1024%2C650&amp;ssl=1 1024w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/> Notably, the text and graph within the image are highlighted after these two operations. Now with this knowledge, you are ready to develop your convolutional neural network to improve the accuracy of the OCR engine to over 99%.<\/p>\n<h2><span style=\"color: #3366ff;\">Convolutional Neural Network (CNN) for OCR<\/span><\/h2>\n<p>The CCN architecture we will use to improve the accuracy of our OCR application has two max-pooling layers sandwiched between three convolutional layers. The last convolutional layer is flattened out, like the last part of this series, to feed into the fully connected network. This is the Python code used to train the CNN: <strong><a href=\"http:\/\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/YOU%20CANalytics%20-%20Deep%20Learning%20Convolutional%20Neural%20Networks%20for%20OCR.html\" rel=\"noopener noreferrer\">Convolution Neural Network &#8211; Python Code (Jupyter Notebook)<\/a><\/strong>. This is the network diagram with the number of parameters (weights) learned in each layer.<\/p>\n<pre>Layer (type)                 Output Shape              Param #   \n=================================================================\nconv2d_1 (Conv2D)            (None, 26, 26, 64)        640       \n_________________________________________________________________\nmax_pooling2d_1 (MaxPooling2 (None, 13, 13, 64)        0         \n_________________________________________________________________\nconv2d_2 (Conv2D)            (None, 11, 11, 64)        36928     \n_________________________________________________________________\nmax_pooling2d_2 (MaxPooling2 (None, 5, 5, 64)          0         \n_________________________________________________________________\nconv2d_3 (Conv2D)            (None, 3, 3, 32)          18464     \n_________________________________________________________________\nflatten_1 (Flatten)          (None, 288)               0         \n_________________________________________________________________\ndense_1 (Dense)              (None, 512)               147968    \n_________________________________________________________________\ndropout_1 (Dropout)          (None, 512)               0         \n_________________________________________________________________\ndense_2 (Dense)              (None, 64)                32832     \n_________________________________________________________________\ndense_3 (Dense)              (None, 10)                650       \n=================================================================\nTotal params: 237,482\nTrainable params: 237,482\nNon-trainable params: 0<\/pre>\n<p>A single input image of &#8216;0&#8217; when passed through this trained-network created the following impressions across the layers. The final output from the softmax layer, towards the extreme right, is the probability of digits between 0-9. Here, the trained network has identified the image as &#8216;0&#8217; with almost 100% accuracy.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg\"><img data-attachment-id=\"11831\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-ocr-application-architecture\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?fit=1980%2C627&amp;ssl=1\" data-orig-size=\"1980,627\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network &#8211; OCR application architecture\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?fit=300%2C95&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?fit=640%2C203&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11831\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?resize=640%2C203\" alt=\"\" width=\"640\" height=\"203\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?w=1980&amp;ssl=1 1980w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?resize=250%2C79&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?resize=300%2C95&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?resize=768%2C243&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?resize=1024%2C324&amp;ssl=1 1024w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?w=1280 1280w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-OCR-application-architecture.jpg?w=1920 1920w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>Let&#8217;s see how the original image gets transformed into the output probability. In the first convolutional layer, the original 28 x 28 x 1 image is converted to 64 different images with 26 x 26 x 1 dimensions. Here, 64\u00a0 convolution filters of size 3 x 3 are used. The first convolutional layer has 640 parameters to calculate since it will estimate nine values of 3 X 3 filter matrix and one bias term for 64 different outputs in the next layer i.e.<\/p>\n<pre> <img decoding=\"async\" src=\"https:\/\/s0.wp.com\/latex.php?latex=%283+%5Ctimes+3+%2B+1%29+%5Ctimes+64+%3D+10+%5Ctimes+64+%3D+640+&#038;bg=ffffff&#038;fg=000&#038;s=2&#038;c=20201002\" alt=\"(3 &#92;times 3 + 1) &#92;times 64 = 10 &#92;times 64 = 640 \" class=\"latex\" \/><\/pre>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg\"><img data-attachment-id=\"11849\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-input-image\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?fit=288%2C280&amp;ssl=1\" data-orig-size=\"288,280\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network &#8211; Input Image\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?fit=288%2C280&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?fit=288%2C280&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"alignright  wp-image-11849\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?resize=147%2C143\" alt=\"\" width=\"147\" height=\"143\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?w=288&amp;ssl=1 288w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Input-Image.jpg?resize=250%2C243&amp;ssl=1 250w\" sizes=\"(max-width: 147px) 100vw, 147px\" data-recalc-dims=\"1\" \/><\/a>These 64 filter matrices, learned by CNN after training the network, when multiplied with the input image of zero (28 x 28 x 1) produces these 64 outputs. All the 64 outputs are shown in one single image below. Notice, how the original zero in the input image is preserved in the outputs for the first convolutional layer.\u00a0 When I say preserved, it means that human eyes can still see the zeros in these outputs. Soon, you will notice that when we apply more convolutional filters our eyes will not be able to identify the input zero, however, the network will still preserve the digit in the original image to make the right estimation.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg\"><img data-attachment-id=\"11847\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-conolutional-layer-1-output\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?fit=1305%2C411&amp;ssl=1\" data-orig-size=\"1305,411\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network &#8211; Conolutional Layer 1 Output\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?fit=300%2C94&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?fit=640%2C202&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11847\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?resize=640%2C202\" alt=\"\" width=\"640\" height=\"202\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?w=1305&amp;ssl=1 1305w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?resize=250%2C79&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?resize=300%2C94&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?resize=768%2C242&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Conolutional-Layer-1-Output.jpg?resize=1024%2C323&amp;ssl=1 1024w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>The next max-pool layers have no parameter to learn since they are simple max operations across the input matrices. The next convolution layer, layer-2, will produce 64 images from the 64 input images of the first max-pool layer. The total learnable parameters between these layers are calculated as:<\/p>\n<pre> <img decoding=\"async\" src=\"https:\/\/s0.wp.com\/latex.php?latex=%283+%5Ctimes+3+%5Ctimes+64+%5Ctimes+64%29+%2B+64+%3D+36928+&#038;bg=ffffff&#038;fg=000&#038;s=2&#038;c=20201002\" alt=\"(3 &#92;times 3 &#92;times 64 &#92;times 64) + 64 = 36928 \" class=\"latex\" \/><\/pre>\n<p>After the second convolutional operation, the zero in the image has started fading but there is still a glimpse of the zero in the original image.<a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg\"><img data-attachment-id=\"11850\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-convolutional-layer-2-output\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?fit=1305%2C407&amp;ssl=1\" data-orig-size=\"1305,407\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network &#8211; Convolutional Layer 2 Output\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?fit=300%2C94&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?fit=640%2C199&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-11850 aligncenter\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?resize=640%2C200\" alt=\"\" width=\"640\" height=\"200\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?w=1305&amp;ssl=1 1305w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?resize=250%2C78&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?resize=300%2C94&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?resize=768%2C240&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-2-Output.jpg?resize=1024%2C319&amp;ssl=1 1024w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a><\/p>\n<p>The third and final convolutional operation estimates 18646 parameters. Remember, the last convolutional layer produces 32 output images of much lower dimensions of 3 x 3 x 32.<\/p>\n<pre><img decoding=\"async\" src=\"https:\/\/s0.wp.com\/latex.php?latex=%283+%5Ctimes+3+%5Ctimes+64+%5Ctimes+32%29+%2B+32+%3D+18464+&#038;bg=ffffff&#038;fg=000&#038;s=2&#038;c=20201002\" alt=\"(3 &#92;times 3 &#92;times 64 &#92;times 32) + 32 = 18464 \" class=\"latex\" \/><\/pre>\n<p>The output for the third convolutional layer is displayed here.<a href=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg\"><img data-attachment-id=\"11851\" data-permalink=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/convolutional-neural-network-convolutional-layer-3-output\/\" data-orig-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?fit=1305%2C248&amp;ssl=1\" data-orig-size=\"1305,248\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Convolutional Neural Network &#8211; Convolutional Layer 3 Output\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?fit=300%2C57&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?fit=640%2C122&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-11851\" src=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?resize=640%2C122\" alt=\"\" width=\"640\" height=\"122\" srcset=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?w=1305&amp;ssl=1 1305w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?resize=250%2C48&amp;ssl=1 250w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?resize=300%2C57&amp;ssl=1 300w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?resize=768%2C146&amp;ssl=1 768w, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2019\/03\/Convolutional-Neural-Network-Convolutional-Layer-3-Output.jpg?resize=1024%2C195&amp;ssl=1 1024w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/a> Our human eyes can&#8217;t see the embedded digit in these 32 output images from the third convolutional layer. However, these images have compressed the information from the original image and preserved the embedded zero in the original image. Or more precisely, this pattern generated for the digit 0 is significantly different from the patterns formed by digits 1 to 9 on a 3 x 3 x 32 canvas.<\/p>\n<p>This output becomes the input for the fully connected network similar to the one we used in the previous part. Recall, last time when we used the original images as input without the convolutional and max-pooling operations we got the test set accuracy as 97.9%. This time with these additional operations the accuracy has shot up to 99.1%. Here, CNN has reduced the error by close to 60% from the best model we had earlier.<\/p>\n<h4><span style=\"color: #3366ff;\">Sign-off Note<\/span><\/h4>\n<p>If you have not noticed, the drawing at the top of this article captures Audrey Hepburn in a famous scene from the movie &#8216;Breakfast at Tiffany&#8217;s&#8217;. In this scene, before Hepburn instructs a cab driver, &#8216;Grand Central Station. And step on it, darling.&#8217;, she briefly slid her glasses close to her lips and put it back on. In that brief moment, she must have experienced the power of convolutional neural networks on an unconscious level while seeing the New York city with and without lenses.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Welcome back to the deep learning example to build an OCR application. The idea of this simple application is to identify numbers in an image of written text. In the last part, we used three different models and got the following accuracy for identification of the test images: Model 1 &#8211; Logistic regression: 92% accuracy<\/p>\n<p><a class=\"excerpt-more blog-excerpt\" href=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":1,"featured_media":11593,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[86],"tags":[],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v17.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |\" \/>\n<meta property=\"og:description\" content=\"Welcome back to the deep learning example to build an OCR application. The idea of this simple application is to identify numbers in an image of written text. In the last part, we used three different models and got the following accuracy for identification of the test images: Model 1 &#8211; Logistic regression: 92% accuracyRead More...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/\" \/>\n<meta property=\"og:site_name\" content=\"YOU CANalytics |\" \/>\n<meta property=\"article:author\" content=\"roopam\" \/>\n<meta property=\"article:published_time\" content=\"2019-03-31T04:44:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-03-31T05:25:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&#038;ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"775\" \/>\n\t<meta property=\"og:image:height\" content=\"581\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Roopam Upadhyay\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\",\"name\":\"YOU CANalytics\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"sameAs\":[],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"width\":607,\"height\":120,\"caption\":\"YOU CANalytics\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"name\":\"YOU CANalytics |\",\"description\":\"Explore the Power of Data Science\",\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1\",\"width\":775,\"height\":581,\"caption\":\"Convolutional Neural Networks (CNN) Filters &Polaroid Glasses - by Roopam\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/\",\"name\":\"Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage\"},\"datePublished\":\"2019-03-31T04:44:03+00:00\",\"dateModified\":\"2019-03-31T05:25:07+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucanalytics.com\/blogs\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Convolutional Neural Networks (CNN) Simplified (Part 4)\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage\"},\"author\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\"},\"headline\":\"Convolutional Neural Networks (CNN) Simplified (Part 4)\",\"datePublished\":\"2019-03-31T04:44:03+00:00\",\"dateModified\":\"2019-03-31T05:25:07+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage\"},\"wordCount\":1537,\"commentCount\":4,\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1\",\"articleSection\":[\"Deep Learning Neural Networks\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#respond\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\",\"name\":\"Roopam Upadhyay\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"caption\":\"Roopam Upadhyay\"},\"description\":\"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay\",\"sameAs\":[\"roopam\"],\"url\":\"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/","og_locale":"en_US","og_type":"article","og_title":"Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |","og_description":"Welcome back to the deep learning example to build an OCR application. The idea of this simple application is to identify numbers in an image of written text. In the last part, we used three different models and got the following accuracy for identification of the test images: Model 1 &#8211; Logistic regression: 92% accuracyRead More...","og_url":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/","og_site_name":"YOU CANalytics |","article_author":"roopam","article_published_time":"2019-03-31T04:44:03+00:00","article_modified_time":"2019-03-31T05:25:07+00:00","og_image":[{"width":775,"height":581,"url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1","type":"image\/jpeg"}],"twitter_misc":{"Written by":"Roopam Upadhyay","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/ucanalytics.com\/blogs\/#organization","name":"YOU CANalytics","url":"https:\/\/ucanalytics.com\/blogs\/","sameAs":[],"logo":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#logo","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","width":607,"height":120,"caption":"YOU CANalytics"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/#logo"}},{"@type":"WebSite","@id":"https:\/\/ucanalytics.com\/blogs\/#website","url":"https:\/\/ucanalytics.com\/blogs\/","name":"YOU CANalytics |","description":"Explore the Power of Data Science","publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1","width":775,"height":581,"caption":"Convolutional Neural Networks (CNN) Filters &Polaroid Glasses - by Roopam"},{"@type":"WebPage","@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage","url":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/","name":"Convolutional Neural Networks (CNN) Simplified (Part 4) &ndash; YOU CANalytics |","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage"},"datePublished":"2019-03-31T04:44:03+00:00","dateModified":"2019-03-31T05:25:07+00:00","breadcrumb":{"@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucanalytics.com\/blogs\/"},{"@type":"ListItem","position":2,"name":"Convolutional Neural Networks (CNN) Simplified (Part 4)"}]},{"@type":"Article","@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#article","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage"},"author":{"@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6"},"headline":"Convolutional Neural Networks (CNN) Simplified (Part 4)","datePublished":"2019-03-31T04:44:03+00:00","dateModified":"2019-03-31T05:25:07+00:00","mainEntityOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#webpage"},"wordCount":1537,"commentCount":4,"publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1","articleSection":["Deep Learning Neural Networks"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ucanalytics.com\/blogs\/convolutional-neural-networks-cnn-simplified-part-4\/#respond"]}]},{"@type":"Person","@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6","name":"Roopam Upadhyay","image":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","caption":"Roopam Upadhyay"},"description":"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay","sameAs":["roopam"],"url":"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Convolutional-Neural-Networks-CNN-Filters-Polaroid-Glasses-by-Roopam.jpg?fit=775%2C581&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3L0jT-30P","jetpack-related-posts":[{"id":11156,"url":"https:\/\/ucanalytics.com\/blogs\/deep-learning-and-neural-networks-simplified-part-1\/","url_meta":{"origin":11583,"position":0},"title":"Deep Learning and Neural Networks \u2013 Simplified (Part 1)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"The entire field of artificial intelligence, in the last few years, is built upon deep learning or deep neural networks. Notably, Apple's Siri, Google-DeepMinds' AlphaGo, or the self-driving mechanism in Tesla cars are all based on deep learning. Here, my goal is to make deep learning neural networks\u00a0much more accessible\u2026","rel":"","context":"In &quot;Deep Learning Neural Networks&quot;","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-Front.jpg?fit=960%2C592&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-Front.jpg?fit=960%2C592&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-Front.jpg?fit=960%2C592&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-Front.jpg?fit=960%2C592&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":11578,"url":"https:\/\/ucanalytics.com\/blogs\/deep-learning-models-simplified-part-3\/","url_meta":{"origin":11583,"position":1},"title":"Deep Learning Models Simplified (Part 3)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Facebook was a major sensation and a source of great amusement in a British\u00a0country house in the early 20th century. It was such a big hit that it got a special mention in a newspaper published in the year 1902. Facebook, then, of course, had a completely different meaning than\u2026","rel":"","context":"In &quot;Deep Learning Neural Networks&quot;","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":6808,"url":"https:\/\/ucanalytics.com\/blogs\/learning-resources-artificial-intelligence-cognitive-computing-deep-learning-neural-networks\/","url_meta":{"origin":11583,"position":2},"title":"Learning Resources : Artificial Intelligence, Cognitive Computing, Deep Learning, &#038; Neural Networks","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"This article is an effort to make you into a \"semi-expert\" in artificial intelligence, cognitive computing, deep learning and neural networks from scratch. Here I will share a few cool learning resources for these topics. These resources include documentaries, TED talks, online lecture videos, and books. There are several videos\u2026","rel":"","context":"In &quot;Analytics Book Club&quot;","block_context":{"text":"Analytics Book Club","link":"https:\/\/ucanalytics.com\/blogs\/category\/analytics-book-club\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/Alice-in-Wonderland-Copy.jpg?fit=433%2C274&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":4054,"url":"https:\/\/ucanalytics.com\/blogs\/artificial-neural-networks-retail-case-study-example-part-8\/","url_meta":{"origin":11583,"position":3},"title":"Artificial Neural Networks, Made Easy &#8211; Retail Case Study Example (Part 8)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Welcome back to\u00a0our retail case study example\u00a0for campaign and marketing analytics. So far in this case study, we were working on a\u00a0classification problem to identify customers with a higher likelihood to purchase products from\u00a0the campaign catalogues. In the last article on\u00a0model selection, we noticed that artificial neural networks performed better\u2026","rel":"","context":"In &quot;Marketing Analytics&quot;","block_context":{"text":"Marketing Analytics","link":"https:\/\/ucanalytics.com\/blogs\/category\/marketing-analytics\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/Artificial-Neural-Networks.jpg?fit=346%2C336&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":11290,"url":"https:\/\/ucanalytics.com\/blogs\/math-of-deep-learning-neural-networks-simplified-part-2\/","url_meta":{"origin":11583,"position":4},"title":"Math of Deep Learning Neural Networks &#8211; Simplified (Part 2)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Welcome back to this series of articles on deep learning and neural networks. In the last part, you learned how training a\u00a0deep learning network is similar to a plumbing job. This time you will learn the math of deep learning. We will continue to use the plumbing analogy to simplify\u2026","rel":"","context":"In &quot;Deep Learning Neural Networks&quot;","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":3973,"url":"https:\/\/ucanalytics.com\/blogs\/model-selection-retail-case-study-example-part-7\/","url_meta":{"origin":11583,"position":5},"title":"Model Selection &#8211; Retail Case Study Example (Part 7)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Model Selection This is a continuation of our retail case study example for campaign and marketing analytics. In the previous two parts, we discussed a couple of decision tree algorithms (CART and C4.5)\u00a0for classification. Recall a previous case study example on\u00a0banking and risk management where we discussed logistic regression\u00a0which is\u2026","rel":"","context":"In &quot;Marketing Analytics&quot;","block_context":{"text":"Marketing Analytics","link":"https:\/\/ucanalytics.com\/blogs\/category\/marketing-analytics\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/11583"}],"collection":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/comments?post=11583"}],"version-history":[{"count":0,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/11583\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media\/11593"}],"wp:attachment":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media?parent=11583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/categories?post=11583"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/tags?post=11583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}