Seeing with Your Tongue!<\/span><\/h2>\n![\"Artificial](\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/Artificial-Neural-Networks.jpg?resize=295%2C286\")
<\/a>Artificial Neural Networks – by Roopam<\/p><\/div>\n
Eric Weihenmayer climbed Mount Everest in 2001. By doing this he became the first and till date the only blind person to achieve this feat. He pursues his passion in extreme rock climbing through a device called BrainPort, which helps him to see using his tongue! This device has a camera at one end connected to several hundred tiny electrodes that Eric places on his tongue to experience obstacles on his path. This experience for Eric is made possible through the incredible learning adaptiveness of the human brain. Initially, when Eric started using this device, he used to feel\u00a0just some tingly sensation on his tongue associated with some experience. Slowly his brain learned to correlate each experience with a distinct sensation and enabled him to experience seeing. This is a phenomenal story about our brain’s capability to adapt – the property that has inspired machine learning algorithm called artificial neural networks.<\/p>\n
Neural Networks’ Feed-Forward & Feedback Loops<\/span><\/h2>\nThe brain connects with other parts of the body through an intricate network of neurons called biological neural networks. The brain works with a powerful mechanism involving both feed-forward and feedback loops within these intricate neural networks. For example, the feed-forward mechanism involves inputs from the sensory organs, like eyes and ears, converted into outputs i.e, information and understanding. The feedback mechanism, on the other hand, makes the brain communicate with sensory organs and modify their inputs.<\/p>\n
To\u00a0learn about this better, let’s perform a couple of small experiments. \u00a0For the first experiment, close your eyes and say the following 3 words, at an interval of 10 seconds each, with the intention to visualize them.<\/p>\n
\n- Dragon<\/li>\n
- Village<\/li>\n
- Dragon\u00a0killer<\/li>\n<\/ul>\n
Most probably, you had\u00a0visualized an elaborated\u00a0\u00a0scene with a dragon attacking a village and getting killed by a dragon slayer. What you have just accomplished is a phenomenal capability of the brain to extract information about these words in a split second, and visualize a whole sequence of events without using your eyes. This is also the source of the elaborated imagination that human brains possess. In this case, one form of input (words) has generated another form of input (visualization) through a complicated process in our brain.<\/p>\n
This second experiment will help us understand the feed-forward and feedback loop in our brain. It’s quite likely that you have come across the following sentence on some social media site. Anyway, read the sentence in the following text box.<\/p>\n
\n\n\nAoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n <\/p>\n
Incredible, isn’t it? You \u00a0brain has gone through several cycles of feed-forward and feedback to read these jumbled letters in a matter of seconds. The brain, in this case, has superseded the incomplete and jumbled input i.e. information from eyes to meaningful output i.e. understanding of this sentence. Artificial neural networks try to mimic our phenomenal brain for prediction purposes through both feed-forward and feedback loop between input and output variables.<\/p>\n
Artificial Neural Networks – Retail Case Study Example<\/span><\/h2>\n![\"Artifical](\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/Artifical-Neural-Networks.jpg?resize=291%2C374\")
<\/a>Artificial Neural Networks<\/p><\/div>\n
Artificial neural networks are nowhere close to the intricacies that biological neural networks possess, but we must not forget the latter has gone through millions of years of evolution. On the other hand, artificial neural networks (from here on referred to as neural networks) have a history of close to half a century. In the 1990s, neural networks lost favour to other machine learning algorithms like support vector machines, etc. However, in the last decade or so, there is a renewed interest in neural networks\u00a0because of the rise of deep learning. Let us try to understand the design\u00a0of neural networks and their functionalities using our retail case study.<\/p>\n
As displayed in the adjacent figure, neural networks could be broadly divided into three layers – input, hidden, and output. The hidden layer is the additional feature\u00a0that separates neural networks from other predictive models. If we remove hidden layer from this architect it will become a simple regression (for estimation) or logistic regression (for classification), architect. The input layer in this architect is simply the input variables. For our retail case study as discussed in the last articles some of the input variables\u00a0are:<\/p>\n
\n\n\n\n\n- Recency: # recent visits to the company\u2019s website and purchases<\/li>\n
- Frequency: time lag between purchases in the last 6 months<\/li>\n
- Payment mode used: cash on delivery, credit card, internet banking etc.<\/li>\n
- Marketing data aggregator\u2019s:\u00a0life-stage segmentations (i.e. luxury buffs, up-scale ageing, first-time earners etc.)<\/li>\n
- Last year\u2019s expenditure trend: amount spent last year<\/li>\n
- Coupon usage pattern of customer<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
<\/p>\n
The output layer, for our classification problem to\u00a0identify customers who will respond to campaigns, is the binary variable that represents historic responders (0\/1).<\/p>\n
Mathematical Construct of Neural Networks<\/span><\/h2>\nThis section describes the mathematical construct of neural networks. If this seems a bit\u00a0complicated to you, for now, I suggest you\u00a0jump to the next section about the usage of neural networks.<\/p>\n
Let’s come back to the hidden layer, each hidden layer has several hidden nodes (orange circles in the above figure). Each hidden node takes a weighted sum of input from every input variable. The following expression represents the weighted sum of input variables that the hidden nodes take as input. These input variables can be compared with input signals our sensory organs send to our brain, for example, in a case of fire around you – you see fire, you hear fire burning, you smell smoke, and your skin feels hot (a complete sensory experience through several input nodes).<\/p>\n
![\"(\textup{Hidden](\"https:\/\/s0.wp.com\/latex.php?latex=%28%5Ctextup%7BHidden+Node%7D%29_%7Bi%7D%3DW_%7Bi%28Input%5Crightarrow+Hidden%29%7D%5E%7BT%7D%5Ctimes+%5Ctextup%7B%28Input+Variables%29%7D_%7Bi%7D%2BW_%7B0%7D+&bg=ffffff&fg=000&s=0&c=20201002\")
<\/pre>\nTo begin with, the above weights Wi(Input\u2192Hidden) <\/em><\/sub>\u00a0&\u00a0W<\/em>0<\/em>\u00a0<\/sub>are chosen at random, then they are modified iteratively to match the desired outputs (in output layer). Continuing with the above example of fire, if the sensory signals about the fire are too strong the tendency of the creature for self-preservation will take over.\u00a0However, sensory signals about the fire from cooking stove need to be accounted\u00a0as well for humans to cook. Hence, weights need to adjust for the fire usage and self-preservation.<\/p>\nIn the hidden layer, the above\u00a0linear weighted sum [(Hidden Node)i<\/sub><\/em>] is converter to non-linear form through a non-linear function. This conversion is usually performed using the sigmoid activation function, yes this is the same logit function of the logistic regression. The following expression represents this process<\/p>\n![\"{P(Hidden)}_{j}=\frac{e^{(\textup{Hidden](\"https:\/\/s0.wp.com\/latex.php?latex=%7BP%28Hidden%29%7D_%7Bj%7D%3D%5Cfrac%7Be%5E%7B%28%5Ctextup%7BHidden+Node%7D%29_%7Bi%7D%7D%7D%7B1%2Be%5E%7B%28%5Ctextup%7BHidden+Node%7D%29_%7Bi%7D%7D%7D++&bg=ffffff&fg=000&s=2&c=20201002\")
<\/pre>\nRemember 0\u00a0\u2264\u00a0P(Hidden)j\u00a0\u2264 1;\u00a0<\/em>this output [P(Hidden)j<\/sub><\/em>] for the different hidden nodes (j) becomes the input variables for the final output node. As described below<\/p>\n![\"(\textup{Output})=U_{j(Hidden\rightarrow](\"https:\/\/s0.wp.com\/latex.php?latex=%28%5Ctextup%7BOutput%7D%29%3DU_%7Bj%28Hidden%5Crightarrow+Output%29%7D%5E%7BT%7D%5Ctimes+P%28Hidden%29_%7Bj%7D%2BU_%7B0%7D+&bg=ffffff&fg=000&s=0&c=20201002\")
<\/pre>\nThis linear weighted output is again converted to non-linear form through sigmoid function. The following is the probability of conversion of a customer P(Customer Response) based on his\/her input variables.<\/p>\n
![\"{P(\textup{Customer](\"https:\/\/s0.wp.com\/latex.php?latex=%7BP%28%5Ctextup%7BCustomer+Response%7D%29%7D%3D%5Cfrac%7Be%5E%7B%28%5Ctextup%7BOutput%7D%29%7D%7D%7B1%2Be%5E%7B%28%5Ctextup%7BOutput%7D%29%7D%7D++&bg=ffffff&fg=000&s=2&c=20201002\")
<\/pre>\nNeural network algorithms (like back propagation) iteratively modify weights for both links (i.e. Input\u2192Hidden\u2192Output) to reduce the error of prediction. Remember the weights for our architect are\u00a0weights Wi(Input\u2192Hidden),\u00a0<\/em><\/sub>W<\/em>0<\/em>\u00a0<\/sub>weights Uj(Hidden\u2192Output) <\/em><\/sub>\u00a0&\u00a0U<\/em>0<\/em>\u00a0<\/sub><\/p>\nPros and Cons of Using Neural Networks<\/span><\/h2>\nLet’s quickly sum of some of the important pros and cons of using neural networks for model development<\/p>\n
Pros<\/p><\/blockquote>\n
\n- \n
\n![\"Data](\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/Data-Science-Tasks-by-Roopam.jpg?resize=300%2C176\")
Neural networks offer highly versatile methods to solve four of the six broad categories of data science tasks i.e. classification, estimation, forecasting, and clustering (self-organizing map). These six broad categories of data science tasks were discussed in the previous article<\/a>\u00a0as displayed in the adjacent diagram.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n <\/p>\n
\n- \n
\n- Neural networks\u00a0are fairly insensitive to noise in the input data (similar to our brain) because of the hidden layer that absorbs noisy information.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
<\/p>\n
\n- They are better equipped to handle the fuzzy \/ non-linear relationship between input variables, and the output variable.<\/li>\n<\/ol>\n
Cons<\/p><\/blockquote>\n
\n- \n
\n- Neural networks are often considered as black boxes (again similar to our brain) because they don’t explicitly highlight the relationship between input and output variables. This is highly unlike the decision trees<\/a> which offer highly intuitive solutions.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
<\/p>\n
<\/a>Artificial Neural Networks – by Roopam<\/p><\/div>\n
Eric Weihenmayer climbed Mount Everest in 2001. By doing this he became the first and till date the only blind person to achieve this feat. He pursues his passion in extreme rock climbing through a device called BrainPort, which helps him to see using his tongue! This device has a camera at one end connected to several hundred tiny electrodes that Eric places on his tongue to experience obstacles on his path. This experience for Eric is made possible through the incredible learning adaptiveness of the human brain. Initially, when Eric started using this device, he used to feel\u00a0just some tingly sensation on his tongue associated with some experience. Slowly his brain learned to correlate each experience with a distinct sensation and enabled him to experience seeing. This is a phenomenal story about our brain’s capability to adapt – the property that has inspired machine learning algorithm called artificial neural networks.<\/p>\n
Neural Networks’ Feed-Forward & Feedback Loops<\/span><\/h2>\nThe brain connects with other parts of the body through an intricate network of neurons called biological neural networks. The brain works with a powerful mechanism involving both feed-forward and feedback loops within these intricate neural networks. For example, the feed-forward mechanism involves inputs from the sensory organs, like eyes and ears, converted into outputs i.e, information and understanding. The feedback mechanism, on the other hand, makes the brain communicate with sensory organs and modify their inputs.<\/p>\n
To\u00a0learn about this better, let’s perform a couple of small experiments. \u00a0For the first experiment, close your eyes and say the following 3 words, at an interval of 10 seconds each, with the intention to visualize them.<\/p>\n
\n- Dragon<\/li>\n
- Village<\/li>\n
- Dragon\u00a0killer<\/li>\n<\/ul>\n
Most probably, you had\u00a0visualized an elaborated\u00a0\u00a0scene with a dragon attacking a village and getting killed by a dragon slayer. What you have just accomplished is a phenomenal capability of the brain to extract information about these words in a split second, and visualize a whole sequence of events without using your eyes. This is also the source of the elaborated imagination that human brains possess. In this case, one form of input (words) has generated another form of input (visualization) through a complicated process in our brain.<\/p>\n
This second experiment will help us understand the feed-forward and feedback loop in our brain. It’s quite likely that you have come across the following sentence on some social media site. Anyway, read the sentence in the following text box.<\/p>\n
\n\n\nAoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n <\/p>\n
Incredible, isn’t it? You \u00a0brain has gone through several cycles of feed-forward and feedback to read these jumbled letters in a matter of seconds. The brain, in this case, has superseded the incomplete and jumbled input i.e. information from eyes to meaningful output i.e. understanding of this sentence. Artificial neural networks try to mimic our phenomenal brain for prediction purposes through both feed-forward and feedback loop between input and output variables.<\/p>\n
Artificial Neural Networks – Retail Case Study Example<\/span><\/h2>\n<\/a>Artificial Neural Networks<\/p><\/div>\n
Artificial neural networks are nowhere close to the intricacies that biological neural networks possess, but we must not forget the latter has gone through millions of years of evolution. On the other hand, artificial neural networks (from here on referred to as neural networks) have a history of close to half a century. In the 1990s, neural networks lost favour to other machine learning algorithms like support vector machines, etc. However, in the last decade or so, there is a renewed interest in neural networks\u00a0because of the rise of deep learning. Let us try to understand the design\u00a0of neural networks and their functionalities using our retail case study.<\/p>\n
As displayed in the adjacent figure, neural networks could be broadly divided into three layers – input, hidden, and output. The hidden layer is the additional feature\u00a0that separates neural networks from other predictive models. If we remove hidden layer from this architect it will become a simple regression (for estimation) or logistic regression (for classification), architect. The input layer in this architect is simply the input variables. For our retail case study as discussed in the last articles some of the input variables\u00a0are:<\/p>\n
\n\n\n\n\n- Recency: # recent visits to the company\u2019s website and purchases<\/li>\n
- Frequency: time lag between purchases in the last 6 months<\/li>\n
- Payment mode used: cash on delivery, credit card, internet banking etc.<\/li>\n
- Marketing data aggregator\u2019s:\u00a0life-stage segmentations (i.e. luxury buffs, up-scale ageing, first-time earners etc.)<\/li>\n
- Last year\u2019s expenditure trend: amount spent last year<\/li>\n
- Coupon usage pattern of customer<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
<\/p>\n
The output layer, for our classification problem to\u00a0identify customers who will respond to campaigns, is the binary variable that represents historic responders (0\/1).<\/p>\n
Mathematical Construct of Neural Networks<\/span><\/h2>\nThis section describes the mathematical construct of neural networks. If this seems a bit\u00a0complicated to you, for now, I suggest you\u00a0jump to the next section about the usage of neural networks.<\/p>\n
Let’s come back to the hidden layer, each hidden layer has several hidden nodes (orange circles in the above figure). Each hidden node takes a weighted sum of input from every input variable. The following expression represents the weighted sum of input variables that the hidden nodes take as input. These input variables can be compared with input signals our sensory organs send to our brain, for example, in a case of fire around you – you see fire, you hear fire burning, you smell smoke, and your skin feels hot (a complete sensory experience through several input nodes).<\/p>\n
<\/pre>\nTo begin with, the above weights Wi(Input\u2192Hidden) <\/em><\/sub>\u00a0&\u00a0W<\/em>0<\/em>\u00a0<\/sub>are chosen at random, then they are modified iteratively to match the desired outputs (in output layer). Continuing with the above example of fire, if the sensory signals about the fire are too strong the tendency of the creature for self-preservation will take over.\u00a0However, sensory signals about the fire from cooking stove need to be accounted\u00a0as well for humans to cook. Hence, weights need to adjust for the fire usage and self-preservation.<\/p>\nIn the hidden layer, the above\u00a0linear weighted sum [(Hidden Node)i<\/sub><\/em>] is converter to non-linear form through a non-linear function. This conversion is usually performed using the sigmoid activation function, yes this is the same logit function of the logistic regression. The following expression represents this process<\/p>\n<\/pre>\nRemember 0\u00a0\u2264\u00a0P(Hidden)j\u00a0\u2264 1;\u00a0<\/em>this output [P(Hidden)j<\/sub><\/em>] for the different hidden nodes (j) becomes the input variables for the final output node. As described below<\/p>\n<\/pre>\nThis linear weighted output is again converted to non-linear form through sigmoid function. The following is the probability of conversion of a customer P(Customer Response) based on his\/her input variables.<\/p>\n
<\/pre>\nNeural network algorithms (like back propagation) iteratively modify weights for both links (i.e. Input\u2192Hidden\u2192Output) to reduce the error of prediction. Remember the weights for our architect are\u00a0weights Wi(Input\u2192Hidden),\u00a0<\/em><\/sub>W<\/em>0<\/em>\u00a0<\/sub>weights Uj(Hidden\u2192Output) <\/em><\/sub>\u00a0&\u00a0U<\/em>0<\/em>\u00a0<\/sub><\/p>\nPros and Cons of Using Neural Networks<\/span><\/h2>\nLet’s quickly sum of some of the important pros and cons of using neural networks for model development<\/p>\n
Pros<\/p><\/blockquote>\n
\n- \n
\n- Neural networks offer highly versatile methods to solve four of the six broad categories of data science tasks i.e. classification, estimation, forecasting, and clustering (self-organizing map). These six broad categories of data science tasks were discussed in the previous article<\/a>\u00a0as displayed in the adjacent diagram.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
<\/p>\n
\n- \n
\n- Neural networks\u00a0are fairly insensitive to noise in the input data (similar to our brain) because of the hidden layer that absorbs noisy information.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
<\/p>\n
\n- They are better equipped to handle the fuzzy \/ non-linear relationship between input variables, and the output variable.<\/li>\n<\/ol>\n
Cons<\/p><\/blockquote>\n
\n- \n
\n- Neural networks are often considered as black boxes (again similar to our brain) because they don’t explicitly highlight the relationship between input and output variables. This is highly unlike the decision trees<\/a> which offer highly intuitive solutions.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
<\/p>\n
Most probably, you had\u00a0visualized an elaborated\u00a0\u00a0scene with a dragon attacking a village and getting killed by a dragon slayer. What you have just accomplished is a phenomenal capability of the brain to extract information about these words in a split second, and visualize a whole sequence of events without using your eyes. This is also the source of the elaborated imagination that human brains possess. In this case, one form of input (words) has generated another form of input (visualization) through a complicated process in our brain.<\/p>\n
This second experiment will help us understand the feed-forward and feedback loop in our brain. It’s quite likely that you have come across the following sentence on some social media site. Anyway, read the sentence in the following text box.<\/p>\n
| Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n <\/p>\n Incredible, isn’t it? You \u00a0brain has gone through several cycles of feed-forward and feedback to read these jumbled letters in a matter of seconds. The brain, in this case, has superseded the incomplete and jumbled input i.e. information from eyes to meaningful output i.e. understanding of this sentence. Artificial neural networks try to mimic our phenomenal brain for prediction purposes through both feed-forward and feedback loop between input and output variables.<\/p>\n Artificial Neural Networks – Retail Case Study Example<\/span><\/h2>\n<\/a>Artificial Neural Networks<\/p><\/div>\n Artificial neural networks are nowhere close to the intricacies that biological neural networks possess, but we must not forget the latter has gone through millions of years of evolution. On the other hand, artificial neural networks (from here on referred to as neural networks) have a history of close to half a century. In the 1990s, neural networks lost favour to other machine learning algorithms like support vector machines, etc. However, in the last decade or so, there is a renewed interest in neural networks\u00a0because of the rise of deep learning. Let us try to understand the design\u00a0of neural networks and their functionalities using our retail case study.<\/p>\n As displayed in the adjacent figure, neural networks could be broadly divided into three layers – input, hidden, and output. The hidden layer is the additional feature\u00a0that separates neural networks from other predictive models. If we remove hidden layer from this architect it will become a simple regression (for estimation) or logistic regression (for classification), architect. The input layer in this architect is simply the input variables. For our retail case study as discussed in the last articles some of the input variables\u00a0are:<\/p>\n
|