![\"sumo-and-regression-model\"](\"https:\/\/i0.wp.com\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/09\/Sumo-and-Regression-Model.jpg\")
<\/a>Regression Models and Information Asymmetry – by Roopam<\/p><\/div>\n
This is a continuation of our case study example to estimate property pricing. In this part, you will learn nuances of regression modeling by building three different regression models and compare their results.\u00a0We will also use results of the principal component analysis<\/a><\/strong>, discussed in the last part, to develop a regression model. You can find all the parts of this case study at the following links: regression analysis case study example<\/a>.<\/strong><\/p>\n However, before we start building regression models let me highlight the importance of information in pricing and also explain how data science & regression creates a level playing field by eliminating information asymmetry.<\/p>\n According to Freakonomics, real-estate agents are like Ku Klux Klan (KKK), a notorious secret society responsible for lynching African\u00a0Americans.\u00a0The entire existence of KKK and real-estate agents depends on the asymmetry\u00a0of information. The more information they have than others, the stronger they become. KKK was eventually destroyed by efforts to make their secret information public. Once the mask was removed from KKK’s face and information asymmetry was gone, the secret society was blown away like a puff of dust.\u00a0Data science has a massive role to play for the democratization of information. Imagine the world where everyone has access to all the data\/information, and everyone knows how to extract knowledge from that data. In this scenario, expertise will rely\u00a0upon sophisticated human skills like creativity and innovation rather than secrecy and deceit.<\/p>\n As we move towards our regression case study, we can take a big lesson from this Freakonomics’ analysis on\u00a0real estate agents. While in our case study example we are using just a handful of predictor variables to estimate housing price, there are so many interesting phenomena outside our dataset that determine housing price like if the owner of the house is also a real estate agent. As a data scientist, it’s our job to unearth these interesting phenomena and build robust models.<\/p>\n In this case study example, you are building regression models to help an investment firm make money through property price arbitrage. You are under a lot of pressure from your client to deliver the price estimation model soon. You have prepared your data by adjusting it for outliers and missing values. To begin with, you will build a complete model with all the predictor variables. You can find the entire R code used in this article at this link:\u00a0regression-models-r-code<\/a>.<\/strong><\/p>\n The first step in model building is to fetch data in R and identify numeric and categorical predictor variables. Moreover, we will also tag house price as the target or response variable. This is exactly what the next few lines of code is doing.<\/p>\n Now, we will tag the variables based on their properties.<\/p>\n The next step is to divide your sample into training and test set. We will build all the 3 models on the training set and evaluate the\u00a0performance of the model on the test set. These datasets are formed by random selection of 70% of data as the training set and the remaining 30% dataset is the testing set.<\/p>\n Now, we will build our first regression model with all the available variables in our dataset.<\/p>\n These are the results of the first regression model.<\/p>\nInformation Asymmetry & Regression Models<\/span><\/h2>\n
<\/a>You want to sell your house that you had purchased 8 years ago. You get mixed messages from different sources about the state of the real-estate market. Some say the housing market is booming and others believe it’s a bust. You are thoroughly confused and in good faith approached a real-estate agent to help you crack a good deal. The agent is an expert. He gets a 1.5% commission on the selling price hence will grab you the best deal. Freakonomics, a book by Steven Levitt & Stephen Dubner, argues otherwise. Levitt, as a part of his research, analyzed sales patterns\u00a0of houses owned by real-estate agents versus their customers. He observed that houses owned\u00a0by agents were selling at a 3% higher price than their customers. So how do these agents get better deals for their own properties? Incidentally, they keeping their own properties listed on the market for roughly 10 more days than their clients’ properties. A 300,000 dollars house will fetch $10,000 more in these additional 10 days. However, if the property belongs to the client, the agent will get just $150 for 10 extra days of effort. He would rather close the deal early and move on to the next deal.<\/p>\nCase Study Example – Regression Model<\/span><\/h2>\n
Step 1: fetch data for regression modeling & tag the variables<\/span><\/h4>\n
\r\nClean_Data = read.csv('http:\/\/ucanalytics.com\/blogs\/wp-content\/uploads\/2016\/09\/Regression-Clean-Data.csv')<\/pre>\n\r\nnumeric=c('Dist_Taxi','Dist_Market','Dist_Hospital','Carpet','Builtup','Rainfall')\r\ncategoric = c('Parking', 'City_Category')\r\nTarget = c('House_Price')\r\n<\/pre>\nStep 2: prepare train and test data for regression modeling<\/span><\/span><\/strong><\/span><\/h4>\n
\r\nset.seed(42)\r\ntrain = sample(nrow(Clean_Data), 0.7*nrow(Clean_Data))\r\ntest = setdiff(seq_len(nrow(Clean_Data)), train)<\/pre>\n
Step 3: build 1st regression model with all the available variables<\/strong> <\/span><\/span><\/h4>\n
\r\nOrg_Reg=lm(House_Price~.,data=Clean_Data[train,c(Target,numeric,categoric)])\r\nsummary(Org_Reg)\r\n<\/pre>\n