{"id":10717,"date":"2018-04-28T14:00:19","date_gmt":"2018-04-28T08:30:19","guid":{"rendered":"http:\/\/ucanalytics.com\/blogs\/?p=10717"},"modified":"2018-08-21T11:11:12","modified_gmt":"2018-08-21T05:41:12","slug":"machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2","status":"publish","type":"post","link":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/","title":{"rendered":"Machine Learning : Regularization – Ridge, Lasso, & Elastic Net Simplified (Part 2)"},"content":{"rendered":"
\n
\"\"<\/a>

Machine Learning: Regularization (Ridge, Lasso, and Elastic Nets) – by Roopam<\/p><\/div>\n

In the previous article, we started with the theme that\u00a0overfitting is an inherent problem in machine learning associated with big data<\/a><\/strong>. Essentially, if you have many variables and their\u00a0polynomial\u00a0terms (X-variables) in a model you could fit any response data (y-variable) to perfection. This perfect fit for the observed data is overfitting since this model will generalize\u00a0poorly to the unobserved data.\u00a0 In this post, you will learn the ways of overcoming this problem with regularization techniques. But before that let’s visit a funfair!<\/p>\n

Regularization and Funfair<\/span><\/h2>\n

\"\"<\/a>I was 6 or 7 when one evening my grandmother took all her grandchildren to the local funfair. At the entry,\u00a0she announced that we could take as many rides as we liked and as many times as we wanted. Yay! shouted all us kids with joy. She continued – as long as the expense was kept under 2 rupees, and we would get back home by 7 PM. This second instruction could have been a big dampener. However, I remember having a great time at the fair managing grandma’s instructions.<\/p>\n

Many years later, when I was in Orlando, I visited Disney Land (or was it Universal Studios?) with my friends. This was a much grander funfair than the one my grandma took me to. I managed to get a 3-day pass at a discounted price.\u00a0 This time I could actually take as many rides and as many times I wanted. I was finally liberated from my grandma’s instructions. After I took a few head-spinning rides, I was so rattled that by evening I opted to rest\u00a0under a tree while my friends were finishing up. I didn’t go to the amusement park for the remaining two days. I guess constraints imposed by my grandma were not all that bad.<\/p>\n

You will soon realize that my grandma’s instructions and regularization in machine learning are quite similar.<\/p>\n

Regularization – Intuition (Grandma to the Rescue)<\/span><\/h2>\n

In the subsequent segments, you will notice a few intimidating equations. However,\u00a0 you must not get intimidated\u00a0by them since they are very similar to my grandma’s instructions. The first instruction was to try as many rides as you wish. This instruction had no constraints. In terms of the objective for machine learning it can be represented as:<\/p>\n

\"Loss\<\/pre>\n
Now, you must have seen this equation for the loss function in one of my earlier posts on gradient descent optimization for linear regression<\/a><\/strong>. This loss function\u00a0is essentially the mean square error (MSE) in the estimation of the y variable using the x variables. The objective is to minimize the MSE or the loss function. The\u00a0y-variable\u00a0is estimated by the linear combination of the X variables.<\/p>\n
\"\hat{y}_{i}=\theta_{0}+\theta_{1}x_{1}^{(i)}+\theta_{2}x_{2}^{(i)}+\dots+\theta_{M}x_{M}^{(i)}<\/pre>\n
Here, the equation can take any value for the \u03b8 parameters between -\u221e to \u221e. This complete freedom also leads to overfitting since there are no constraints on the \u03b8s. Now, similar to grandma’s second instruction, let’s add a few regulations on this complete freedom. The most commonly used regularizations are L1 and L2. Let’s start with L2 first.<\/p>\n
L2 Regularization: Ridge Regression<\/span><\/h2>\n
Here, I will take a few liberties to create linkages between my grandma’s second instruction and regularization. Let me relive her instructions one more time. First, let’s just consider the time constraint imposed by her i.e. to get back home by 7 PM (spend an hour at the fair). Let’s assume there were 10 rides and each ride takes 3 minutes. Then I could have taken all the 10 rides in the first 30 minutes. The remaining 30 minutes could then be spent on rides on which I had the most fun.<\/p>\n
This is similar to L2 regularization which is ridge regression in this case.\u00a0 This regularization term is added after MSE in the loss function i.e. \u03bb\u2211\u03b82<\/sup>. This is similar to saying spend 1 hour in the fair not infinite. Here \u03bb is a constant which is also caller hyper-parameter.<\/p>\n
\"LF:<\/pre>\n
This constraint is imposed on the \u03b8 parameters. Essentially, all the \u03b8s will have some small value similar to how I spent the first 30 minutes exploring all the rides. Then the \u03b8 parameters with the highest contribution to minimizing MSE will get higher values. Hence, L2 regularization assigns values to all the \u03b8 parameters or all the X variables feature in the final equation.<\/p>\n
L1 Regularization: Lasso\u00a0<\/span><\/h2>\n
The other constraint by my grandma was on the total expenditure. As a kid, growing by in India 35 years ago, time was always more available than money. Hence, I could not have taken all the rides. Here, I would observe and\u00a0evaluate the rides from a distance first. Then I would take the rides I believed I would enjoy. This is L1 regularization or Lasso. The additional term for this is again at the end.i.e. \u03bb\u2211|\u03b8|\u00a0<\/span><\/p>\n
\"LF:<\/pre>\n
Lasso or L1 regularization will ensure that only useful predictor variables or Xs get a weight or non-zero\u00a0\u03b8 parameters. Lasso essentially sets \u03b8s to zero for less useful x variables.<\/p>\n
Elastic Net Regularization: Ridge + Lasso<\/span><\/h2>\n
Elastic net is essentially imposing both L1 and L2 at the same time. This is similar to my grandmother’s instruction about managing both time and expense at the same time. Notice, both these terms are at the end of the loss function.<\/p>\n
\"LF:<\/pre>\n
This time we have two hyperparameters. The first hyperparameter is lambda (\u03bb) similar to the previous equations. The second hyperparameter is alpha (\u03b1) which takes the value between 0 and 1. When \u03b1 is 1, then the regularization term is purely lasso. For\u00a0\u03b1=0.5, the regularization has 50% regularization from lasso and 50% form Ridge. Now, let’s solve the problem of the\u00a0sine function we started in the previous part.<\/p>\n
Sine Curve – Regularization to Reduce Overfitting<\/span><\/h2>\n
We had the population for the sine curve displayed with the gray line in the plot between -3\u03c0 to 3\u03c0. This is an unusual situation where we know the population distribution with complete certainty\u00a0– unlike any other analysis in real life. The R-code to replicate the analysis is\u00a0Regularization – Lasso & Ridge (Elastic Net) and Cross-Validation<\/a><\/strong><\/p>\n
\"\"<\/a><\/p>\n
Now, we will draw the training and testing sample from the population. Notice, the sample data (dots) are not exactly on the population line. Since the act of measurement is not perfect and adds some random noise to the data. The training sample (blue dots) will be used to build the model. The trained models will be validated on the testing sample (red dots). The testing sample is the unobserved data for the model. Hence, the accuracy on the testing sample is a good proof that the model is generalizable.<\/p>\n
The First Model – Complete Freedom or No Regularization<\/span><\/h2>\n
Here, response variable, y, is the sine function i.e. sin(x). The X-variables, predictors, are the 15 polynomial terms of x.<\/p>\n
\"sin(x)=\theta<\/pre>\n
The first model is built without any regularization terms – grandma’s first instruction! This means the model has a complete freedom to choose the \u03b8-parameters without any constraints. This model’s estimated curve is shown with the orange line in this plot.<\/p>\n
\"\"<\/a>The model-fit looks quite good for both train and test data between -4 to +4 range. The model, however, goes nuts on either corner. This erratic behavior could be attributed to fewer blue dots (training data) in these positions. The model has, essentially, tried to perfectly fit every blue dot. This is the reason the MSE for the training sample is very low at 0.02. This perfect fit, however, is not generalizable. Hence, the MSE for the testing sample is close to 2 million (whoops). We need to make the model more generalizable by adding regularization terms.<\/p>\n
Second Model – Intuitive Regularization Term<\/span><\/h2>\n
Now, let’s add an intuitive regularization term to this model. We know, from the previous article, that for the sine curve the even powers of x (i.e. x2<\/sup>, x4<\/sup>, x6<\/sup>…) have no contribution. In other words,\u00a0\u03b82\u00a0<\/sub>=0, \u03b84<\/sub>=0, \u03b86 <\/sub>=0 and so on. This is a good case for L1 regularization. Remember, Lasso or L1 regularization reduces the \u03b8 parameters for useless terms to zero – unlike Ridge regression. This means we will set hyper-parameter alpha to one for this model (\u03b1=1).<\/p>\n
Moreover, the non-regularized model was performing quite OK for the x range of -4 to 4. Hence, we don’t want to add a very strong regularization term to our model. This means we will keep the value of the second hyper-parameter, lambda, small i.e. \u03bb = 0.0001.<\/span><\/p>\n
\"\"<\/a><\/p>\n
For this regularized model, the MSE for the training data has significantly gone up from 0.02 to 1377. However, the MSE of test data is down by a factor of 100. This model is doing much better on the unobserved test data.<\/p>\n
So the model’s accuracy depends on the hyper-parameters: alpha and lambda. But how do we know that we have used the best possible value for lambda and alpha in the loss function? A solution to the problem of identification of optimal values for the hyper-parameters is…<\/p>\n
Cross-Validation<\/span><\/h2>\n
Cross-validation is a way to test several combinations of hyper-parameters to identify their optimal values. This method works on the training sample. But wait a minute, we know if we train the model on the entire training data then overfitting is inevitable. There is a clever trick to guard against overfitting i.e. extract a validation sample from the training data. Let’s see how a 5-fold cross validation works.<\/p>\n
\"\"<\/p>\n
We have 30 data points in our training data. We will randomly divide these data into 5 groups (5-fold). Each group will have the same size i.e. 6 observations per fold. The models are then trained on 4 folds of data, and the remaining fold is used to test the model. This way you will have 5 models per combination of hyperparameters. The hyper-parameter combination that gives the best model statistics for the testing folds is considered as the optimal. The best model statistics is the lowest value of both mean and standard deviation for MSE (mean standard errors).<\/p>\n
Third Model – Explore\u00a0\u03bb Values with Cross-Validation<\/span><\/h2>\n
We will run a quick and dirty cross-validation model to conclude this article. In the last part of this series, we shall explore cross-validation and hyper-parameter tuning in greater details. In this model, we will keep the alpha fixed to keep things simple i.e.\u00a0\u03b1=1. We will try different values for lambda(\u03bb) to find the best value of\u00a0\u03bb. We will try one-thousand different values of\u00a0\u03bb between 10-6\u00a0<\/sup>to 1.\u00a0 It turned out that the MSE for\u00a0\u03bb=0.00448 is the lowest for all the 1000 tried values. The model for this value of lambda is:<\/p>\n
\"\"<\/a>For this third model, the MSE for both training and testing sample is significantly lower than the second model. Our guess for the value of \u03bb was not bad, but cross-validation has done a much better job of finding the best value. This model has essentially lost some level of predictive power in the middle of the plot to achieve a greater accuracy at the corners.<\/p>\n
Sign-off Note<\/span><\/h4>\n
As a kid I had no clue that at the fair I was optimizing while being regularized by my grandmother’s instructions. I guess we are training our machines the same way our previous generations trained us.<\/p>\n","protected":false},"excerpt":{"rendered":"
In the previous article, we started with the theme that\u00a0overfitting is an inherent problem in machine learning associated with big data. Essentially, if you have many variables and their\u00a0polynomial\u00a0terms (X-variables) in a model you could fit any response data (y-variable) to perfection. This perfect fit for the observed data is overfitting since this model will<\/p>\n
Read More…<\/a><\/p>\n","protected":false},"author":1,"featured_media":10718,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[84,85],"tags":[],"jetpack_publicize_connections":[],"yoast_head":"\nMachine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |\" \/>\n<meta property=\"og:description\" content=\"In the previous article, we started with the theme that\u00a0overfitting is an inherent problem in machine learning associated with big data. Essentially, if you have many variables and their\u00a0polynomial\u00a0terms (X-variables) in a model you could fit any response data (y-variable) to perfection. This perfect fit for the observed data is overfitting since this model willRead More...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/\" \/>\n<meta property=\"og:site_name\" content=\"YOU CANalytics |\" \/>\n<meta property=\"article:author\" content=\"roopam\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-28T08:30:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-08-21T05:41:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"736\" \/>\n\t<meta property=\"og:image:height\" content=\"531\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Roopam Upadhyay\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\",\"name\":\"YOU CANalytics\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"sameAs\":[],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"width\":607,\"height\":120,\"caption\":\"YOU CANalytics\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"name\":\"YOU CANalytics |\",\"description\":\"Explore the Power of Data Science\",\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1\",\"width\":736,\"height\":531},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/\",\"name\":\"Machine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage\"},\"datePublished\":\"2018-04-28T08:30:19+00:00\",\"dateModified\":\"2018-08-21T05:41:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucanalytics.com\/blogs\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning : Regularization – Ridge, Lasso, & Elastic Net Simplified (Part 2)\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage\"},\"author\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\"},\"headline\":\"Machine Learning : Regularization – Ridge, Lasso, & Elastic Net Simplified (Part 2)\",\"datePublished\":\"2018-04-28T08:30:19+00:00\",\"dateModified\":\"2018-08-21T05:41:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage\"},\"wordCount\":1843,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1\",\"articleSection\":[\"Machine Learning and Artificial Intelligence\",\"Regularization and Cross Validation\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#respond\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\",\"name\":\"Roopam Upadhyay\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"caption\":\"Roopam Upadhyay\"},\"description\":\"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay\",\"sameAs\":[\"roopam\"],\"url\":\"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |","og_description":"In the previous article, we started with the theme that\u00a0overfitting is an inherent problem in machine learning associated with big data. Essentially, if you have many variables and their\u00a0polynomial\u00a0terms (X-variables) in a model you could fit any response data (y-variable) to perfection. This perfect fit for the observed data is overfitting since this model willRead More...","og_url":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/","og_site_name":"YOU CANalytics |","article_author":"roopam","article_published_time":"2018-04-28T08:30:19+00:00","article_modified_time":"2018-08-21T05:41:12+00:00","og_image":[{"width":736,"height":531,"url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1","type":"image\/jpeg"}],"twitter_misc":{"Written by":"Roopam Upadhyay","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/ucanalytics.com\/blogs\/#organization","name":"YOU CANalytics","url":"https:\/\/ucanalytics.com\/blogs\/","sameAs":[],"logo":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#logo","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","width":607,"height":120,"caption":"YOU CANalytics"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/#logo"}},{"@type":"WebSite","@id":"https:\/\/ucanalytics.com\/blogs\/#website","url":"https:\/\/ucanalytics.com\/blogs\/","name":"YOU CANalytics |","description":"Explore the Power of Data Science","publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1","width":736,"height":531},{"@type":"WebPage","@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage","url":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/","name":"Machine Learning : Regularization - Ridge, Lasso, & Elastic Net Simplified (Part 2) – YOU CANalytics |","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage"},"datePublished":"2018-04-28T08:30:19+00:00","dateModified":"2018-08-21T05:41:12+00:00","breadcrumb":{"@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucanalytics.com\/blogs\/"},{"@type":"ListItem","position":2,"name":"Machine Learning : Regularization – Ridge, Lasso, & Elastic Net Simplified (Part 2)"}]},{"@type":"Article","@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#article","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage"},"author":{"@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6"},"headline":"Machine Learning : Regularization – Ridge, Lasso, & Elastic Net Simplified (Part 2)","datePublished":"2018-04-28T08:30:19+00:00","dateModified":"2018-08-21T05:41:12+00:00","mainEntityOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#webpage"},"wordCount":1843,"commentCount":2,"publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1","articleSection":["Machine Learning and Artificial Intelligence","Regularization and Cross Validation"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ucanalytics.com\/blogs\/machine-learning-regularization-ridge-lasso-elastic-net-simplified-part-2\/#respond"]}]},{"@type":"Person","@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6","name":"Roopam Upadhyay","image":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","caption":"Roopam Upadhyay"},"description":"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay","sameAs":["roopam"],"url":"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Regularization-Lasso-and-Ridge.jpg?fit=736%2C531&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3L0jT-2MR","jetpack-related-posts":[{"id":10702,"url":"https:\/\/ucanalytics.com\/blogs\/machine-learning-non-linear-regression-regularization-cross-validation-simplified-part-1\/","url_meta":{"origin":10717,"position":0},"title":"Machine Learning: Non-linear Regression, Regularization & Cross Validation Simplified  (Part 1)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"In this 3-part series of articles, you will gain an intuitive understanding of some fundamental concepts in machine learning such as: Building blocks of curves Non-linear regression Curve fitting and overfitting Regularization to prevent overfitting Hyper-parameters in machine learning Cross-validation to fine-tune models You will also get hands-on practice to\u2026","rel":"","context":"In "Machine Learning and Artificial Intelligence"","block_context":{"text":"Machine Learning and Artificial Intelligence","link":"https:\/\/ucanalytics.com\/blogs\/category\/machine-learning-and-artificial-intelligence\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Machine-Learning-Regularization-Simplified-Front-Page.jpg?fit=928%2C522&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Machine-Learning-Regularization-Simplified-Front-Page.jpg?fit=928%2C522&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Machine-Learning-Regularization-Simplified-Front-Page.jpg?fit=928%2C522&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/12\/Machine-Learning-Regularization-Simplified-Front-Page.jpg?fit=928%2C522&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":10774,"url":"https:\/\/ucanalytics.com\/blogs\/machine-learning-cross-validation-and-hyper-parameter-tuning-part-3\/","url_meta":{"origin":10717,"position":1},"title":"Machine Learning : Cross Validation and Hyper-Parameter Tuning (Part 3)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"In the last part of this series on fundamental machine learning, you learned about\u00a0regularization and cross-validation. Here, you will gain a sound understanding of model hyper-parameter tuning to develop robust\u00a0models. The machines do learn but they still need a good human tutor. In the last part, you were also introduced\u2026","rel":"","context":"In "Machine Learning and Artificial Intelligence"","block_context":{"text":"Machine Learning and Artificial Intelligence","link":"https:\/\/ucanalytics.com\/blogs\/category\/machine-learning-and-artificial-intelligence\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/01\/Cross-Validation-and-Hyperparameter-Tuning-Cover-3.jpg?fit=746%2C560&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/01\/Cross-Validation-and-Hyperparameter-Tuning-Cover-3.jpg?fit=746%2C560&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/01\/Cross-Validation-and-Hyperparameter-Tuning-Cover-3.jpg?fit=746%2C560&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/01\/Cross-Validation-and-Hyperparameter-Tuning-Cover-3.jpg?fit=746%2C560&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":8388,"url":"https:\/\/ucanalytics.com\/blogs\/regression-analysis-pricing-case-study-example-part-1\/","url_meta":{"origin":10717,"position":2},"title":"Regression Analysis – Pricing Case Study Example (Part 1)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"How to figure out if you are paying the right price for the property you are about to purchase? Welcome to a new data science case study example on YOU CANalytics to identify the right housing price. Pricing is a highly important and\u00a0specialized function for any business. A right price\u2026","rel":"","context":"In "Pricing Case Study Example"","block_context":{"text":"Pricing Case Study Example","link":"https:\/\/ucanalytics.com\/blogs\/category\/pricing-case-study-example\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/07\/Connect-the-Dots.jpg?fit=397%2C603&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":7923,"url":"https:\/\/ucanalytics.com\/blogs\/intuitive-machine-learning-gradient-descent-simplified\/","url_meta":{"origin":10717,"position":3},"title":"Intuitive Machine Learning : Gradient Descent Simplified","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"How do machines learn? They learn the same way as humans. Humans learn from experience and so do machines. For machines, the experience is in the form of data. Machines use powerful algorithms to make sense of the data. They identify underlining patterns within the data to learn things about\u2026","rel":"","context":"In "Gradient Descent"","block_context":{"text":"Gradient Descent","link":"https:\/\/ucanalytics.com\/blogs\/category\/gradient-descent\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":11578,"url":"https:\/\/ucanalytics.com\/blogs\/deep-learning-models-simplified-part-3\/","url_meta":{"origin":10717,"position":4},"title":"Deep Learning Models Simplified (Part 3)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Facebook was a major sensation and a source of great amusement in a British\u00a0country house in the early 20th century. It was such a big hit that it got a special mention in a newspaper published in the year 1902. Facebook, then, of course, had a completely different meaning than\u2026","rel":"","context":"In "Deep Learning Neural Networks"","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2374,"url":"https:\/\/ucanalytics.com\/blogs\/learn-r-12-books-and-online-resources\/","url_meta":{"origin":10717,"position":5},"title":"Learn R : 12 Free Books and Online Resources","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Please read the disclaimer about the Free PDF Books in this article at the bottom R, an open-source statistical and data mining programming language, is slowly but surely catching up in its race with commercial software like SAS & SPSS. I believe R will eventually replace SAS as the language\u2026","rel":"","context":"In "Analytics Book Club"","block_context":{"text":"Analytics Book Club","link":"https:\/\/ucanalytics.com\/blogs\/category\/analytics-book-club\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Swordsmith1.jpg?fit=768%2C1024&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Swordsmith1.jpg?fit=768%2C1024&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Swordsmith1.jpg?fit=768%2C1024&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Swordsmith1.jpg?fit=768%2C1024&ssl=1&resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/10717"}],"collection":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/comments?post=10717"}],"version-history":[{"count":0,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/10717\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media\/10718"}],"wp:attachment":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media?parent=10717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/categories?post=10717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/tags?post=10717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}