{"id":10469,"date":"2017-09-27T16:32:41","date_gmt":"2017-09-27T11:02:41","guid":{"rendered":"http:\/\/ucanalytics.com\/blogs\/?p=10469"},"modified":"2018-09-16T17:39:16","modified_gmt":"2018-09-16T12:09:16","slug":"gradient-descent-logistic-regression-simplified-step-step-visual-guide","status":"publish","type":"post","link":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/","title":{"rendered":"Gradient Descent for Logistic Regression Simplified – Step by Step Visual Guide"},"content":{"rendered":"
\n

\"\"<\/p>\n

If you want to gain a sound understanding of machine learning then you must know gradient descent optimization.\u00a0In this article, you will get a detailed and intuitive understanding of gradient descent to solve machine learning algorithms. The entire tutorial uses images and visuals to make things easy to grasp.<\/p>\n

Here, we will use an example from sales and marketing to identify customers who will purchase perfumes. Gradient descent, by the way, is a numerical method to solve such business problems using machine learning algorithms such as regression, neural networks, deep learning etc. Moreover, in this article, you will build an end-to-end logistic regression model using gradient descent. Do check out this earlier article on gradient descent for estimation through linear regression\u00a0<\/a><\/strong><\/p>\n

But before that let’s boldly go where no man has gone before and explore a few linkages between…<\/p>\n

Gradient Descent &\u00a0Star Trek<\/span><\/h2>\n

Star Trek is a science fiction TV series created by Gene Roddenberry. The original show was first aired in the mid-1960s. Since then Star Trek has become one of the largest\u00a0franchises for Hollywood. Till date, Star Trek has seven different television series, and thirteen motion pictures based on its different avatars.<\/p>\n

The original 1960s show is among the first few TV shows I remember from my childhood. Captain Kirk and Spock are cult\u00a0figures. The opening line of Star Trek still gives me goosebumps. It goes like this:<\/p>\n

Space, the final frontier. These are the voyages of the starship Enterprise. Its five-year mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone before.<\/p>\n

– Opening Dialogue, Star Trek<\/p>\n<\/blockquote>\n

Star Trek is about exploring the unknown. Essentially, trekking as a concept is about making a difficult journey to arrive at the destination. You will soon learn that gradient descent, a numeric approach to solve machine learning algorithm, is no different than trekking.<\/p>\n

Moreover, Star Trek has this fascinating device called Transporter – a machine that could teleport Captain Kirk and his crew members to the desired location in no time.\u00a0Keep the ideas of trekking and Transporter in your mind because soon you will play Captain Kirk to solve a gradient descent problem. But before that let’s define our business problem and solution objectives.<\/p>\n

Market Research Problem – Logistic Regression<\/span><\/h2>\n

You are a market researcher and helping the perfume industry to understand their customer\u00a0segments. You had asked your team to survey 200 buyers (1) and 200 non-buyers (0) of perfumes. This information is stored in the y variable of this marketing dataset<\/strong><\/a>. Moreover, you have asked these surveyees about their monthly expenditure on cosmetics (x1 <\/sub>– reported in 100) and their annual income (x2<\/sub>\u00a0– reported in 100000). In this plot, you plotted these 400 surveyees on the x1<\/sub> and x2<\/sub> axes. Moreover, you have marked the buyers and non-buyers in different colors. You can find the complete code used in this article at Gradient Descent – Logistic Regression (R Code)<\/a>.<\/strong><\/p>\n

\"\"<\/a><\/p>\n

Clearly, the buyers and non-buyers are clustered in separate areas of this plot. Now, you want to create a clear boundary or wall to separate the buyers from non-buyers. This is kind of similar to the wall Donald Trump wants to build between the USA and Mexico. You could still see some Mexicans on the American territory and\u00a0vice-a-versa.<\/p>\n

\"\"<\/a><\/p>\n

Another representation of this wall is the density plot as shown below. Notice, in this plot, we have taken both\u00a0x1<\/sub> and x2 <\/sub>collectively as x1<\/sub>\u00a0+ x2<\/sub>. The reason why we could do this because the data was prepared in such a way to simplify things.<\/p>\n

\"\"<\/a><\/p>\n

As you may have noticed, if you divide this graph in half at x1<\/sub>\u00a0+ x2<\/sub>= 140 then on the right-hand side you predominantly have the buyers. Also, the left-hand side has more non-buyers then buyers. However, there is still a bit of infringement of the buyers into the non-buyers’ territory and\u00a0vice-a-versa.<\/p>\n

A better representation would be to divide the territories fuzzily or probabilistically to incorporate both cultures and set of people. This is precisely the point all the political leaders like Donald Trump miss. This is possibly also the reason we have territory problems like Gaza Strip, Kashmir etc.<\/p>\n

Logistic Regression Equation and Probability<\/span><\/h2>\n

Mathematics, however, allows data to mingle and live in better harmony. To discover this, let’s plot y as a function of x1<\/sub> + x2<\/sub>\u00a0in a simple scatter plot. Don’t forget y=1 is for the buyers of perfumes and y=0 is for the non-buyers.<\/p>\n

\"\"<\/a>
\nThe reason we have plotted this bland looking scatter plot is that we want to fit a logit function P(y=1) =\"\frac{1}{(1+e^{-z})} to this dataset. Here, P(y=1) is the probability of being a buyer in the entire space of\u00a0x1<\/sub> + x2<\/sub>. Moreover, z is a linear combination of x1 and x2 represented as\u00a0\"z=\beta_{0}.<\/p>\n

In addition, for simplicity, we will assume \u03b21<\/sub> = \u03b22 <\/sub>. This is not a good assumption in a general sense but will work for our dataset which was designed to simplify things.<\/p>\n

\"\"<\/a><\/p>\n

Now, all we need to do is find the values of \u03b20<\/sub>\u00a0,\u00a0<\/sub>\u03b21<\/sub>\u00a0and \u03b22 <\/sub>that will minimize the prediction errors within this data. In other words, this means we want to find the values of\u00a0\u03b20<\/sub>\u00a0,\u00a0<\/sub>\u03b21<\/sub>\u00a0and \u03b22 <\/sub>so that most, if not all, buyers get high probabilities on this logit function P(y=1). Similarly, most non-buyers must get low probabilities on P(y=1).<\/p>\n

The quest for such\u00a0\u03b2 values is the job of gradient descent. Like other quests, as you will soon see, this requires a long journey and lots of walking.<\/p>\n

Gradient Descent Optimization and Trekking<\/span><\/h2>\n

Gradient descent works similar to a hiker walking down a hilly terrain. Essentially, this terrain is similar to error or loss functions for machine learning algorithms. The idea with the ML algorithms, as already discussed, is to get to the bottom-most or minimum error\u00a0point by changing ML coefficients \u00a0\u03b20<\/sub>\u00a0,\u00a0<\/sub>\u03b21<\/sub>\u00a0and \u03b22 <\/sub>. The hilly terrain is spread across the x, and y-axis on the physical world. Similarly, the error function is spread across coefficients of machine learning algorithm i.e.\u00a0\u03b20<\/sub>\u00a0,\u00a0<\/sub>\u03b21<\/sub>\u00a0and \u03b22 <\/sub>.<\/p>\n

\"\"<\/a><\/p>\n

The idea is to find the values of the coefficients such that the error becomes minimum. A better analogy of gradient descent algorithm is through Star Trek, Captain Kirk, and Transporter – the teleportation device. I promise you will get to say “Beam Me Up, Scotty<\/em>” the legendary line Captain Kirk use to instruct Scotty, the operator of Transporter, to teleport him around.<\/p>\n

Captain Kirk to Solve Gradient Descent<\/span><\/h2>\n

Assume you are Captain Kirk. A wicked alien has abducted several crew members of the Starship Enterprise including Spock. The alien has given you the last chance to save your crew if only you can solve a problem. The problem involves finding the minimum value of the variable y for all the possible values of x between -\u221e to \u221e.<\/p>\n

\"y<\/pre>\n
Now, you are Captain Kirk and not a mathematician so you will use your own method to find the minimum or lowest value of y by changing the values of x. You have found a landscape in the exact shape of the function with x and y-axes that spreads across the Universe. You will ask Scotty to teleport you to a random location on this landscape and then you will walk down the landscape to find the value of x that will generate the\u00a0lowest value of y. By the way, y here is similar to the error function and x is similar to the ML coefficients i.e\u00a0\u03b2 values.\"\"<\/a><\/p>\n
Now all of us humans, including Captain Kirk, can figure out which way is the downward slope just by walking. It takes a lot more effort to walk upwards than downwards. We could also figure out a flat plane because the effort is same no matter which direction you walk. In the landscape Captain Kirk is walking, there are just 5 flat points with A, B, and C as the 3 bottom points. However, the problem with this terrain is that there are three minimum values – here A and B are the local minima. C, on the other hand, is the global minimum or the lowest value of y at x=3. Different locations that Captain Kirk starts by being beamed at random will settle him at different minima since he will only walk down.<\/p>\n
We will come back to this problem of local minima, but before that let’s identify the mathematical equivalent of walking up or down which the actual gradient descent optimization will use.<\/p>\n
Gradient Descent and Differential Calculus<\/span><\/h2>\n
The mathematical equivalent of human ability to identify downward slope is differentiation of a function also called gradients. The differentiation of the landscape Captain Kirk was walking is:<\/p>\n
\"\frac{\partial<\/pre>\n
 <\/p>\n
Now, if you insert x=3.1 in this equation you will get the \"\frac{\partial as 4.83. The positive value indicates it’s an upslope ahead. This mean Captain Kirk, or the pointer for gradient descent algorithm, needs to walk backward or toward the lower values of x. Similarly, at x=2.9, \u00a0the \"\frac{\partial = -3.27. The -ve value means it is a downslope ahead, and Captain Kirk will walk forward or towards the higher values of x.<\/p>\n
At x=3, the global minima,\u00a0\"\frac{\partial =0. This means there is no slope or you have reached a flat plane. As it turns out, the above polynomial equation can be simplified to this. It was a trick question from the alien.<\/p>\n
\"\frac{\partial<\/pre>\n
We can find minimum values of this function without gradient descent by equating this equation to 0. \"\frac{\partial =0 for, x = -2, -1, 1, 2, and 3. So, if Captain Kirk knew some math he could have avoided all the walking.<\/p>\n
\"\"<\/a>Let’s use this knowledge to find the machine learning prediction solution to our market research data.<\/p>\n
Solution – Logistics Regression<\/span><\/h2>\n
In the market research data, you are trying to fit the logit function to find the probability of perfume buyers P(y=1). We don’t want to write P(y=1) many times hence we will define a simpler notation : P(y=1)= \"\ltimes.<\/p>\n
\"P(y=1)=\frac{1}{1+e^{-z}}\u00a0=\ltimes<\/pre>\n
We also \u00a0know that z in the above equation is a linear function of x values with\u00a0\u00a0\u03b2 coefficients i.e.<\/p>\n
\"z=\beta_{0}<\/pre>\n
Now, to solve this logit function we need to minimize the error or loss function with respect to the \u03b2 coefficients. The loss function ( \"\ell ) for a logistic regression problem is:<\/p>\n
\"\ell<\/pre>\n
This loss function ( \"\ell ) is cleverly designed to make the problem a convex optimization problem which is a fancy way to say that the function is like a bowl. Like a bowl, this function has just one base or global minimum value and there are no local minima. This plot shows the loss function for our dataset – notice how it is like a bowl. This convex function has solved a big problem that Captain Kirk faced of having several local minima.
\n\"\"<\/a><\/p>\n
Remember, for simplicity, we are assuming \u03b21<\/sub> = \u03b22<\/sub>\u00a0in the above plot so that we could use x1<\/sub> and x2<\/sub> collectively as\u00a0x1<\/sub> + x2<\/sub>. The data was also prepared to keep this assumption but despite this, the real\u00a0\u03b21<\/sub>\u00a0and \u03b22<\/sub>\u00a0will have different values if you will solve for independent x1<\/sub> and x2<\/sub>.<\/p>\n
Getting to the bottom<\/span><\/h2>\n
Now, you want to solve logit equation by minimizing the loss function by changing\u00a0\u03b21<\/sub>\u00a0and \u03b20<\/sub>. To identify the gradient or slope of the function at each point we need to identify the\u00a0derivatives of the loss function with respect to\u00a0\u03b21<\/sub>\u00a0and \u03b20<\/sub>. Since we assumed \u03b21<\/sub> = \u03b22 <\/sub>we are essentially working with one variable (x1<\/sub> + x2<\/sub>) instead of two (x1<\/sub>\u00a0& x2<\/sub>).<\/p>\n
It turned out that derivatives are simply (I have shown the derivation of this at the bottom of this article after the Sign-Off Note – and trust me these results are not that difficult to derive so do check them out) :<\/p>\n
\"\frac{\partial<\/pre>\n
and,<\/p>\n
\"\frac{\partial<\/pre>\n
By using these values you can now send Captain Kirk to find the bottom of this loss function bowl. Also, while running the gradient descent code it is prudent to normalize the x values in the data. This will reduce Captain Kirk’s walking time or make the algorithm run faster. It turned out that the loss function bowl has the bottom at\u00a0\u03b20<\/sub> =\u00a0-0.0315 and\u00a0\u03b21<\/sub> = 2.073 for normalized x1+x2 values.<\/p>\n
\"\"<\/a><\/p>\n
 <\/p>\n
The normalized x1<\/sub>+x2<\/sub> value\u00a0\u03b2 values can be easily modified to actual values of x1<\/sub> and x2<\/sub> and in that case non-normalized \u03b20<\/sub> = -15.4438 and\u00a0\u03b21<\/sub> = 0.1095. I will leave it up to you to figure out the logic of denormalizing the \u03b2 values.<\/p>\n
If you will run the gradient descent without assuming\u00a0\u03b21<\/sub> = \u03b22 <\/sub>then\u00a0\u03b20<\/sub> =-15.4233, \u00a0\u03b21<\/sub> = 0.1090, and \u03b22<\/sub> =\u00a00.1097. I suggest you try all these solutions using this code:\u00a0Gradient Descent – Logistic Regression (R Code)<\/a>.<\/strong><\/p>\n
Sign off Note<\/span><\/h4>\n
The original script of the Star Trek series was written without the mention of Transporter or the teleporting device. It was later realized that the lower budget of the show would not allow filming of the starship landing on unknown planets. They then devised a smaller vessel ship that was also beyond the budget of the show. Finally, a low-cost teleportation device (Transporter) was conceived. Thank heavens for the low-budget creativity because of which we have Transporter – a fascinating device!<\/p>\n
Derivation of Gradients for Gradient Descent Function<\/span><\/h4>\n
You just need to know the following four basic derivatives to derive the lowest value of the loss function i.e.<\/p>\n
1: Derivative of x with respect to itself is<\/p>\n
\"\frac{\partial}{\partial<\/pre>\n
2:\u00a0Derivative of x2<\/sup> with respect to x is<\/p>\n
\"\frac{\partial}{\partial<\/pre>\n
3: Derivative of ex<\/sup> with respect to x is<\/p>\n
\"\frac{\partial}{\partial<\/pre>\n
4: And, finally derivative of ln(x) is<\/p>\n
\"\frac{\partial}{\partial<\/pre>\n
Loss function is defined as<\/p>\n
\"\ell<\/pre>\n
The logit function which feeds into the loss function is<\/p>\n
\"\ltimes=P(y=1)=\frac{1}{1+e^{-z}}\u00a0<\/pre>\n
And finally, z is a linear function of x values with\u00a0\u00a0\u03b2 coefficients which feeds in \u00a0the logit function<\/p>\n
\"z=\beta_{0}<\/pre>\n
Now, we want to find the derivative or slope of loss function with respect to\u00a0\u03b2 coefficients i.e.<\/p>\n
\"\frac{\partial<\/pre>\n
This equation can be expanded to individual components of loss function ( \"\ell ), logit, and z<\/p>\n
\"\frac{\partial<\/pre>\n
Let’s calculate the individual components of this formula. The first component is<\/p>\n
\"\frac{\partial<\/pre>\n
The second component:<\/p>\n
\"\frac{\partial<\/pre>\n
Since,<\/p>\n
\"\frac{\partial<\/pre>\n
Finally,\u00a0the third component<\/p>\n
\"\frac{\partial<\/pre>\n
Hence, the product of three components will provide us with the derivative of the loss function with respect to the beta coefficients<\/p>\n
\"\frac{\partial<\/pre>\n
Similarly,<\/p>\n
\"\frac{\partial<\/pre>\n
and,<\/p>\n
\"\frac{\partial<\/pre>\n
That’s it.<\/p>\n","protected":false},"excerpt":{"rendered":"
If you want to gain a sound understanding of machine learning then you must know gradient descent optimization.\u00a0In this article, you will get a detailed and intuitive understanding of gradient descent to solve machine learning algorithms. The entire tutorial uses images and visuals to make things easy to grasp. Here, we will use an example<\/p>\n
Read More…<\/a><\/p>\n","protected":false},"author":1,"featured_media":10480,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[87,84],"tags":[],"jetpack_publicize_connections":[],"yoast_head":"\nGradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Gradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |\" \/>\n<meta property=\"og:description\" content=\"If you want to gain a sound understanding of machine learning then you must know gradient descent optimization.\u00a0In this article, you will get a detailed and intuitive understanding of gradient descent to solve machine learning algorithms. The entire tutorial uses images and visuals to make things easy to grasp. Here, we will use an exampleRead More...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/\" \/>\n<meta property=\"og:site_name\" content=\"YOU CANalytics |\" \/>\n<meta property=\"article:author\" content=\"roopam\" \/>\n<meta property=\"article:published_time\" content=\"2017-09-27T11:02:41+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-09-16T12:09:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"997\" \/>\n\t<meta property=\"og:image:height\" content=\"587\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Roopam Upadhyay\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\",\"name\":\"YOU CANalytics\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"sameAs\":[],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120\",\"width\":607,\"height\":120,\"caption\":\"YOU CANalytics\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/\",\"name\":\"YOU CANalytics |\",\"description\":\"Explore the Power of Data Science\",\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1\",\"width\":997,\"height\":587},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage\",\"url\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/\",\"name\":\"Gradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage\"},\"datePublished\":\"2017-09-27T11:02:41+00:00\",\"dateModified\":\"2018-09-16T12:09:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucanalytics.com\/blogs\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Gradient Descent for Logistic Regression Simplified – Step by Step Visual Guide\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage\"},\"author\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\"},\"headline\":\"Gradient Descent for Logistic Regression Simplified – Step by Step Visual Guide\",\"datePublished\":\"2017-09-27T11:02:41+00:00\",\"dateModified\":\"2018-09-16T12:09:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage\"},\"wordCount\":2393,\"commentCount\":26,\"publisher\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#organization\"},\"image\":{\"@id\":\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1\",\"articleSection\":[\"Gradient Descent\",\"Machine Learning and Artificial Intelligence\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#respond\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6\",\"name\":\"Roopam Upadhyay\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/ucanalytics.com\/blogs\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g\",\"caption\":\"Roopam Upadhyay\"},\"description\":\"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay\",\"sameAs\":[\"roopam\"],\"url\":\"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Gradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/","og_locale":"en_US","og_type":"article","og_title":"Gradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |","og_description":"If you want to gain a sound understanding of machine learning then you must know gradient descent optimization.\u00a0In this article, you will get a detailed and intuitive understanding of gradient descent to solve machine learning algorithms. The entire tutorial uses images and visuals to make things easy to grasp. Here, we will use an exampleRead More...","og_url":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/","og_site_name":"YOU CANalytics |","article_author":"roopam","article_published_time":"2017-09-27T11:02:41+00:00","article_modified_time":"2018-09-16T12:09:16+00:00","og_image":[{"width":997,"height":587,"url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1","type":"image\/jpeg"}],"twitter_misc":{"Written by":"Roopam Upadhyay","Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/ucanalytics.com\/blogs\/#organization","name":"YOU CANalytics","url":"https:\/\/ucanalytics.com\/blogs\/","sameAs":[],"logo":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#logo","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2015\/11\/YOU-CANalytics-Logo.jpg?fit=607%2C120","width":607,"height":120,"caption":"YOU CANalytics"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/#logo"}},{"@type":"WebSite","@id":"https:\/\/ucanalytics.com\/blogs\/#website","url":"https:\/\/ucanalytics.com\/blogs\/","name":"YOU CANalytics |","description":"Explore the Power of Data Science","publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucanalytics.com\/blogs\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1","contentUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1","width":997,"height":587},{"@type":"WebPage","@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage","url":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/","name":"Gradient Descent for Logistic Regression Simplified - Step by Step Visual Guide – YOU CANalytics |","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage"},"datePublished":"2017-09-27T11:02:41+00:00","dateModified":"2018-09-16T12:09:16+00:00","breadcrumb":{"@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucanalytics.com\/blogs\/"},{"@type":"ListItem","position":2,"name":"Gradient Descent for Logistic Regression Simplified – Step by Step Visual Guide"}]},{"@type":"Article","@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#article","isPartOf":{"@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage"},"author":{"@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6"},"headline":"Gradient Descent for Logistic Regression Simplified – Step by Step Visual Guide","datePublished":"2017-09-27T11:02:41+00:00","dateModified":"2018-09-16T12:09:16+00:00","mainEntityOfPage":{"@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#webpage"},"wordCount":2393,"commentCount":26,"publisher":{"@id":"https:\/\/ucanalytics.com\/blogs\/#organization"},"image":{"@id":"https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1","articleSection":["Gradient Descent","Machine Learning and Artificial Intelligence"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ucanalytics.com\/blogs\/gradient-descent-logistic-regression-simplified-step-step-visual-guide\/#respond"]}]},{"@type":"Person","@id":"https:\/\/ucanalytics.com\/blogs\/#\/schema\/person\/55961a1cea272ecdf290cb387be069b6","name":"Roopam Upadhyay","image":{"@type":"ImageObject","@id":"https:\/\/ucanalytics.com\/blogs\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/dd1aa0b0e813f7639800bcfad6a554f1?s=96&d=mm&r=g","caption":"Roopam Upadhyay"},"description":"This blog contains my personal views and thoughts on predictive Analytics and big data. - Roopam Upadhyay","sameAs":["roopam"],"url":"https:\/\/ucanalytics.com\/blogs\/author\/roopam\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2017\/09\/Star-Trek-GD-main.jpg?fit=997%2C587&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3L0jT-2IR","jetpack-related-posts":[{"id":7923,"url":"https:\/\/ucanalytics.com\/blogs\/intuitive-machine-learning-gradient-descent-simplified\/","url_meta":{"origin":10469,"position":0},"title":"Intuitive Machine Learning : Gradient Descent Simplified","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"How do machines learn? They learn the same way as humans. Humans learn from experience and so do machines. For machines, the experience is in the form of data. Machines use powerful algorithms to make sense of the data. They identify underlining patterns within the data to learn things about\u2026","rel":"","context":"In "Gradient Descent"","block_context":{"text":"Gradient Descent","link":"https:\/\/ucanalytics.com\/blogs\/category\/gradient-descent\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/03\/Rplot01.png?fit=1000%2C600&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":11290,"url":"https:\/\/ucanalytics.com\/blogs\/math-of-deep-learning-neural-networks-simplified-part-2\/","url_meta":{"origin":10469,"position":1},"title":"Math of Deep Learning Neural Networks – Simplified (Part 2)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Welcome back to this series of articles on deep learning and neural networks. In the last part, you learned how training a\u00a0deep learning network is similar to a plumbing job. This time you will learn the math of deep learning. We will continue to use the plumbing analogy to simplify\u2026","rel":"","context":"In "Deep Learning Neural Networks"","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/08\/Deep-Learning-Neural-Networks-and-Plumbing-Job.jpg?fit=796%2C597&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":11578,"url":"https:\/\/ucanalytics.com\/blogs\/deep-learning-models-simplified-part-3\/","url_meta":{"origin":10469,"position":2},"title":"Deep Learning Models Simplified (Part 3)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Facebook was a major sensation and a source of great amusement in a British\u00a0country house in the early 20th century. It was such a big hit that it got a special mention in a newspaper published in the year 1902. Facebook, then, of course, had a completely different meaning than\u2026","rel":"","context":"In "Deep Learning Neural Networks"","block_context":{"text":"Deep Learning Neural Networks","link":"https:\/\/ucanalytics.com\/blogs\/category\/deep-learning-neural-networks\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2018\/10\/Popeye-Deep-Learning.jpg?fit=960%2C686&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":3973,"url":"https:\/\/ucanalytics.com\/blogs\/model-selection-retail-case-study-example-part-7\/","url_meta":{"origin":10469,"position":3},"title":"Model Selection – Retail Case Study Example (Part 7)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"Model Selection This is a continuation of our retail case study example for campaign and marketing analytics. In the previous two parts, we discussed a couple of decision tree algorithms (CART and C4.5)\u00a0for classification. Recall a previous case study example on\u00a0banking and risk management where we discussed logistic regression\u00a0which is\u2026","rel":"","context":"In "Marketing Analytics"","block_context":{"text":"Marketing Analytics","link":"https:\/\/ucanalytics.com\/blogs\/category\/marketing-analytics\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2014\/09\/photo.jpg?fit=1200%2C1029&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":756,"url":"https:\/\/ucanalytics.com\/blogs\/case-study-example-banking-logistic-regression-3\/","url_meta":{"origin":10469,"position":4},"title":"Logistic Regression – Banking Case Study Example (Part 3)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"The Beautiful Formula Mathematicians often conduct competitions for the most beautiful formulae of all. The first position, almost every time, goes to the formula discovered by Leonhard Euler. Displayed below is the formula. $latex e^{i\\pi }+1=0 &s=2$ This formula is phenomenal because it is a combination of the five most\u2026","rel":"","context":"In "Banking Risk Case Study Example"","block_context":{"text":"Banking Risk Case Study Example","link":"https:\/\/ucanalytics.com\/blogs\/category\/risk-analytics\/banking-risk-case-study-example\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/10\/The-Beautiful-Equation-Copy.jpg?fit=1200%2C879&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/10\/The-Beautiful-Equation-Copy.jpg?fit=1200%2C879&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/10\/The-Beautiful-Equation-Copy.jpg?fit=1200%2C879&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/10\/The-Beautiful-Equation-Copy.jpg?fit=1200%2C879&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/10\/The-Beautiful-Equation-Copy.jpg?fit=1200%2C879&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":110,"url":"https:\/\/ucanalytics.com\/blogs\/credit-scorecards-logistic-regression-part-5\/","url_meta":{"origin":10469,"position":5},"title":"Credit Scorecards – Logistic Regression (part 5 of 7)","author":"Roopam Upadhyay","date":false,"format":false,"excerpt":"A Primer on Logistic Regression - Are you Happy? A few years ago, my wife and I took a couple of weeks\u2019 vacation to England and Scotland. Just before boarding the British Airway\u2019s plane, an air-hostess informed us that we were upgraded to business class. Jolly good! What a wonderful\u2026","rel":"","context":"In "Credit Risk Analytics Series"","block_context":{"text":"Credit Risk Analytics Series","link":"https:\/\/ucanalytics.com\/blogs\/category\/risk-analytics\/credit-risk-analytics-series\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Logit-Happy.jpg?fit=614%2C386&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Logit-Happy.jpg?fit=614%2C386&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2013\/07\/5-Logit-Happy.jpg?fit=614%2C386&ssl=1&resize=525%2C300 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/10469"}],"collection":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/comments?post=10469"}],"version-history":[{"count":0,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/posts\/10469\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media\/10480"}],"wp:attachment":[{"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/media?parent=10469"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/categories?post=10469"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucanalytics.com\/blogs\/wp-json\/wp\/v2\/tags?post=10469"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}