{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Identifying bying clients\n",
    "\n",
    "In this notebook we will create a decision tree based model to identify clients who pay (buyers) and understand what makes a user a client (most relevant features).\n",
    "\n",
    "We divide our notebook into four stages: data preparation, feature engineering, model building, feature importance analysis"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import psycopg2\n",
    "from IPython.display import display\n",
    "import yaml\n"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1. Data preparation\n",
    "\n",
    "In this step we load from database (PostgreSQL in this example) data and keep it locally as a CSV file. '\n",
    "The main reason for that is to be able to reproduce results quickly."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "# Create a connection to the database\n",
    "\n",
    "# Load a config file with credentials\n",
    "conf = yaml.load(\n",
    "    open(\"credentials.yml\"), Loader=yaml.FullLoader)['pg']\n",
    "# Create a connection\n",
    "conn = psycopg2.connect(\n",
    "    host=conf['host'],\n",
    "    port=conf['port'],\n",
    "    database=conf['database'],\n",
    "    user=conf['user'],\n",
    "    password=conf['password']\n",
    ")"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Here we prepare two functions to obtain data from the databases (or .csv files if they were pre-downloaded)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [],
   "source": [
    "def load_events(source='csv') -> pd.DataFrame:\n",
    "    \"\"\" Obtains session events from database or csv file\"\"\"\n",
    "    if source == 'db':\n",
    "        q = f'select * from connector_events where sessionid IN {sessions}'\n",
    "        all_events = pd.read_sql(q, conn)\n",
    "        all_events.to_csv('all_events_1454.csv', index=False)\n",
    "    elif source == 'csv':\n",
    "        all_events = pd.read_csv('all_events_1454_sep.csv', sep='|')\n",
    "    else:\n",
    "        raise ValueError(\"source parameter should be either 'csv' or 'db'\")\n",
    "    return all_events\n",
    "\n",
    "def load_sessions(source='csv') -> pd.DataFrame:\n",
    "    \"\"\" Obtains sessions information from database or csv file\"\"\"\n",
    "    if source == 'db':\n",
    "        q = f\"select * from connector_user_sessions where sessionid in {sessions}\"\n",
    "        all_sessions = pd.read_sql(q, conn)\n",
    "        # Saving as a CSV file is optional\n",
    "        all_sessions.to_csv(\"all_sessions.csv\", sep='|', index=False)\n",
    "    elif source == 'csv':\n",
    "        all_sessions = pd.read_csv(\"all_sessions.csv\", sep='|')\n",
    "    else:\n",
    "        raise ValueError(\"source parameter should be either 'csv' or 'db'\")\n",
    "    return all_sessions\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "In the connector's events table we found a client who had a button with a label \"PAY\" by performing a simple query\n",
    "\n",
    "`q = \"select session_id from connector_events where mouseclick_label = 'PAY\";`\n",
    "\n",
    "We went on to find all session ids of this client from our internal tables and saved it in `all_sessions_1454.csv`.\n",
    "This step is unnecessary for clients and only explained for general clarity.\n",
    "**The most important takeaway here, is that we have prepared a list of sessions for which we know\n",
    "whether a click on \"PAY\" button has been made or not.**"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [],
   "source": [
    "sessions_info = pd.read_csv(\"all_sessions_1454.csv\")\n",
    "sessions = tuple(sessions_info['session_id'])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "First off, let's see that the dataset is highly imbalanced, because the number of buyers is much less than the number of ordinary visitors.\n",
    "In fact, the percentage of buying clients (0.04%) is so small, it's not event seen on the pie chart.\n",
    "Hence we're going to use special techniques for imbalanced datasets."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "[encoded data, 12600 characters]\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "buyers_count = sessions_info[sessions_info.paid == 1].shape[0]\n",
    "regular_count = sessions_info.shape[0] - buyers_count\n",
    "ax = plt.subplot(111)\n",
    "wedges, texts, _ = ax.pie(x=(buyers_count, regular_count),\n",
    "        shadow=False,\n",
    "        labels=['Paying visitors', 'Regular visitors'],\n",
    "        autopct='%1.2f%%',\n",
    "        explode=(0, 0.8))\n",
    "\n",
    "for w in wedges:\n",
    "    w.set_linewidth(1)\n",
    "    w.set_edgecolor('white')\n",
    "\n",
    "plt.show()\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\david\\appdata\\local\\programs\\python\\python38\\lib\\site-packages\\IPython\\core\\interactiveshell.py:3361: DtypeWarning: Columns (9,11,12,13,26) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  if (await self.run_code(code, result,  async_=asy)):\n"
     ]
    }
   ],
   "source": [
    "all_events = load_events()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's take a look at the events dataset by printing one session"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "data": {
      "text/plain": "          sessionid  connectioninformation_downlink  \\\n0  4207534060820504                             NaN   \n2  4207534060820504                             NaN   \n3  4207534060820504                             NaN   \n4  4207534060820504                             NaN   \n5  4207534060820504                             NaN   \n6  4207534060820504                             NaN   \n7  4207534060820504                             NaN   \n\n  connectioninformation_type consolelog_level consolelog_value  \\\n0                        NaN              NaN              NaN   \n2                        NaN              NaN              NaN   \n3                        NaN              NaN              NaN   \n4                        NaN              NaN              NaN   \n5                        NaN              NaN              NaN   \n6                        NaN              NaN              NaN   \n7                        NaN              NaN              NaN   \n\n   customevent_messageid  customevent_name  customevent_payload  \\\n0                    NaN               NaN                  NaN   \n2                    NaN               NaN                  NaN   \n3                    NaN               NaN                  NaN   \n4                    NaN               NaN                  NaN   \n5                    NaN               NaN                  NaN   \n6                    NaN               NaN                  NaN   \n7                    NaN               NaN                  NaN   \n\n   customevent_timestamp errorevent_message  ...  issueevent_messageid  \\\n0                    NaN                NaN  ...          5.680858e+09   \n2                    NaN                NaN  ...          5.680913e+09   \n3                    NaN                NaN  ...                   NaN   \n4                    NaN                NaN  ...          5.680859e+09   \n5                    NaN                NaN  ...                   NaN   \n6                    NaN                NaN  ...                   NaN   \n7                    NaN                NaN  ...          5.680898e+09   \n\n  issueevent_timestamp issueevent_type  \\\n0         1.614202e+12      click_rage   \n2         1.614202e+12      click_rage   \n3                  NaN             NaN   \n4         1.614202e+12      click_rage   \n5                  NaN             NaN   \n6                  NaN             NaN   \n7         1.614202e+12             cpu   \n\n                            issueevent_contextstring  issueevent_context  \\\n0  SIGN OUT Triston Armstrong DEVELOPER Join GitS...                 NaN   \n2                                        SAVE & NEXT                 NaN   \n3                                                NaN                 NaN   \n4  SIGN OUT Triston Armstrong DEVELOPER Join GitS...                 NaN   \n5                                                NaN                 NaN   \n6                                                NaN                 NaN   \n7                          https://app.gitstart.com/                 NaN   \n\n             issueevent_payload  customissue_name  customissue_payload  \\\n0                           NaN               NaN                  NaN   \n2                           NaN               NaN                  NaN   \n3                           NaN               NaN                  NaN   \n4                           NaN               NaN                  NaN   \n5                           NaN               NaN                  NaN   \n6                           NaN               NaN                  NaN   \n7  {\"Duration\":10581,\"Rate\":94}               NaN                  NaN   \n\n     received_at  batch_order_number  \n0  1616761976450                2962  \n2  1616761976855                3003  \n3  1616761976460                2965  \n4  1616761976460                2966  \n5  1616761976464                2968  \n6  1616761976539                2972  \n7  1616761976661                2988  \n\n[7 rows x 49 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sessionid</th>\n      <th>connectioninformation_downlink</th>\n      <th>connectioninformation_type</th>\n      <th>consolelog_level</th>\n      <th>consolelog_value</th>\n      <th>customevent_messageid</th>\n      <th>customevent_name</th>\n      <th>customevent_payload</th>\n      <th>customevent_timestamp</th>\n      <th>errorevent_message</th>\n      <th>...</th>\n      <th>issueevent_messageid</th>\n      <th>issueevent_timestamp</th>\n      <th>issueevent_type</th>\n      <th>issueevent_contextstring</th>\n      <th>issueevent_context</th>\n      <th>issueevent_payload</th>\n      <th>customissue_name</th>\n      <th>customissue_payload</th>\n      <th>received_at</th>\n      <th>batch_order_number</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>5.680858e+09</td>\n      <td>1.614202e+12</td>\n      <td>click_rage</td>\n      <td>SIGN OUT Triston Armstrong DEVELOPER Join GitS...</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976450</td>\n      <td>2962</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>5.680913e+09</td>\n      <td>1.614202e+12</td>\n      <td>click_rage</td>\n      <td>SAVE &amp; NEXT</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976855</td>\n      <td>3003</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976460</td>\n      <td>2965</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>5.680859e+09</td>\n      <td>1.614202e+12</td>\n      <td>click_rage</td>\n      <td>SIGN OUT Triston Armstrong DEVELOPER Join GitS...</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976460</td>\n      <td>2966</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976464</td>\n      <td>2968</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976539</td>\n      <td>2972</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>4207534060820504</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>...</td>\n      <td>5.680898e+09</td>\n      <td>1.614202e+12</td>\n      <td>cpu</td>\n      <td>https://app.gitstart.com/</td>\n      <td>NaN</td>\n      <td>{\"Duration\":10581,\"Rate\":94}</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>1616761976661</td>\n      <td>2988</td>\n    </tr>\n  </tbody>\n</table>\n<p>7 rows × 49 columns</p>\n</div>"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_events[all_events.sessionid == all_events.iloc[0].sessionid].head(10)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "We mostly see NaNs and that's OK.\n",
    "One row in session events files contains user's actions grouped by timestamp.\n",
    "For example, if there was a mouse click event at a time $t$, only the columns corresponding to that click\n",
    "(such as mouseclick_label, mouseclick_hesitationtime etc) will be filled and the rest will be NaNs.\n",
    "The columns sessionid, received_at, batch_order_number will always be filled as the contain information about the\n",
    "session unique identifier, the time at which the event was received by connectors worker and the order number\n",
    "to ensure the chronological ordering is preserved when inserting in database in batches."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "['sessionid',\n 'connectioninformation_downlink',\n 'connectioninformation_type',\n 'consolelog_level',\n 'consolelog_value',\n 'customevent_messageid',\n 'customevent_name',\n 'customevent_payload',\n 'customevent_timestamp',\n 'errorevent_message',\n 'errorevent_messageid',\n 'errorevent_name',\n 'errorevent_payload',\n 'errorevent_source',\n 'errorevent_timestamp',\n 'jsexception_message',\n 'jsexception_name',\n 'jsexception_payload',\n 'metadata_key',\n 'metadata_value',\n 'mouseclick_id',\n 'mouseclick_hesitationtime',\n 'mouseclick_label',\n 'pageevent_firstcontentfulpaint',\n 'pageevent_firstpaint',\n 'pageevent_messageid',\n 'pageevent_referrer',\n 'pageevent_speedindex',\n 'pageevent_timestamp',\n 'pageevent_url',\n 'pagerendertiming_timetointeractive',\n 'pagerendertiming_visuallycomplete',\n 'rawcustomevent_name',\n 'rawcustomevent_payload',\n 'setviewportsize_height',\n 'setviewportsize_width',\n 'timestamp_timestamp',\n 'user_anonymous_id',\n 'user_id',\n 'issueevent_messageid',\n 'issueevent_timestamp',\n 'issueevent_type',\n 'issueevent_contextstring',\n 'issueevent_context',\n 'issueevent_payload',\n 'customissue_name',\n 'customissue_payload',\n 'received_at',\n 'batch_order_number']"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a glance at all available features\n",
    "list(all_events.columns)\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 2. Feature engineering\n",
    "\n",
    "How are we going to predict buying users?\n",
    "What kind of features identify them?\n",
    "\n",
    "- Did the user visit the website earlier?\n",
    "- Was the website not illustrating images?\n",
    "- Did users experience many issues on the website?\n",
    "- What pages did he visit?\n",
    "\n",
    "All of those questions sound important.\n",
    "In these section we will extract the answers to this questions from the datasets of events and sessions for each user.\n",
    "We'll create a vector of numerical features and assign them to each user who visited the site.\n",
    "Our goal is to see if the feature sets or buyers and  is separable by some nonlinear function with a good precision. We'll be looking for this function using decision trees model.\n",
    "Of course one can experiment further by applying any other algorithm."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Unfortunately, none of the paying clients received ids.\n",
    "Hence we'll only be looking into the session features."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [],
   "source": [
    "# Create a DataFrame for session features\n",
    "labels = sessions_info['paid']\n",
    "session_features = sessions_info.drop(['paid'], axis=1)\n",
    "session_features.rename({'session_id': 'sessionid'}, axis=1, inplace=True)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's get to some meaningful features. For example, let's create a function\n",
    "that will tell if a specific event happened during the session."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [],
   "source": [
    "def add_feature_about_event_presence(session_features, column, feature_name, dtype='int'):\n",
    "    temp_sessions = all_events[['sessionid', column]].dropna()\n",
    "    temp_sessions = temp_sessions.drop_duplicates(subset='sessionid', keep='last')\n",
    "    if temp_sessions.shape[0] == 0:\n",
    "        return session_features\n",
    "    session_features = session_features.merge(temp_sessions, how='left', on='sessionid')\n",
    "\n",
    "    if dtype == 'int':\n",
    "        session_features.loc[session_features[column] > 0, feature_name] = 1\n",
    "    elif dtype == 'str':\n",
    "        session_features.loc[session_features[column] != '', feature_name] = 1\n",
    "    session_features[feature_name] = session_features[feature_name].fillna(0)\n",
    "    session_features = session_features.drop([column], axis=1)\n",
    "    return session_features"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "outputs": [],
   "source": [
    "for column, feature_name in [('errorevent_messageid', 'error_event'),\n",
    "                             ('customevent_messageid', 'custom_event'),\n",
    "                             ('jsexception_message', 'js_exception'),\n",
    "                             ('customissue_name', 'custom_issue')\n",
    "                             ]:\n",
    "    session_features = add_feature_about_event_presence(session_features, column, feature_name)\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "At some point it's interesting to take a look\n",
    "at the maximum values of some parameters during the session.\n",
    "These features can be added with the function below:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [],
   "source": [
    "def add_max_val(session_features, column):\n",
    "    feature_df = all_events[['sessionid', column]].dropna()\n",
    "    feature_df_agg = feature_df.groupby('sessionid').agg('max').reset_index()\n",
    "    if feature_df_agg.shape[0] > 0:\n",
    "        session_features = session_features.merge(feature_df_agg, how='left', on='sessionid')\n",
    "        session_features[col] = session_features[col].fillna(0)\n",
    "    return session_features"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "outputs": [],
   "source": [
    "max_feature_columns = ['pageevent_firstcontentfulpaint',\n",
    "                       'pageevent_firstpaint',\n",
    "                       'pageevent_speedindex',\n",
    "                       'pagerendertiming_timetointeractive',\n",
    "                       'pagerendertiming_visuallycomplete',\n",
    "                       'rawcustomevent_name',\n",
    "                       'rawcustomevent_payload',\n",
    "                       'setviewportsize_height',\n",
    "                       'setviewportsize_width']\n",
    "\n",
    "for col in max_feature_columns:\n",
    "    session_features = add_max_val(session_features, col)\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "We also should handle categorical variables, the ones that can take on one of a limited,\n",
    "and usually fixed, number of possible values (such us user's browser)."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [],
   "source": [
    "def add_categorial_feature(session_features, column, feature_name):\n",
    "    categories = list(session_features[column].unique())\n",
    "    session_features[feature_name] = pd.Categorical(session_features[column], categories=categories).codes"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\david\\appdata\\local\\programs\\python\\python38\\lib\\site-packages\\IPython\\core\\interactiveshell.py:3361: DtypeWarning: Columns (1,2,3,4,5,8,9,10,11) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  if (await self.run_code(code, result,  async_=asy)):\n"
     ]
    }
   ],
   "source": [
    "# Get additional information from sessions table\n",
    "sessions_table = load_sessions()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "54057\n"
     ]
    }
   ],
   "source": [
    "sessions_table = sessions_table.drop_duplicates(subset=['sessionid'], keep='last')\n",
    "print(sessions_table.shape[0])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "outputs": [],
   "source": [
    "for iss in ['click_rage', 'missing_resource', 'dead_click', 'js_exception', 'bad_request',  'cpu', 'memory']:\n",
    "    session_features[iss] = session_features['issue_types'].apply(lambda x: 1 if iss in x else 0)\n",
    "session_features = session_features.drop(['issue_types'], axis=1)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "outputs": [
    {
     "data": {
      "text/plain": "          sessionid  events_count  pages_count  error_event  \\\n0  4323603410944837             2            1          0.0   \n1  4323604627948361            11            1          0.0   \n2  4323603204776517             1            1          0.0   \n3  4323548146402182            37            7          0.0   \n4  4323554393301661            10            3          0.0   \n5  4323594072990251            17            5          0.0   \n6  4323655897228014             1            1          0.0   \n7  4323630712718240             7            1          0.0   \n8  4323592422793765            11            1          0.0   \n9  4323618781102971             7            2          0.0   \n\n   pageevent_firstcontentfulpaint  pageevent_firstpaint  pageevent_speedindex  \\\n0                             0.0                   0.0                   0.0   \n1                             0.0                   0.0                   0.0   \n2                             0.0                   0.0                   0.0   \n3                          3940.0                3850.0                3825.0   \n4                             0.0                   0.0                   0.0   \n5                             0.0                   0.0                 166.0   \n6                             0.0                   0.0                   0.0   \n7                             0.0                   0.0                   0.0   \n8                          2312.0                2251.0                2312.0   \n9                             0.0                   0.0                   0.0   \n\n   pagerendertiming_timetointeractive  pagerendertiming_visuallycomplete  \\\n0                                 0.0                                0.0   \n1                                 0.0                                0.0   \n2                                 0.0                                0.0   \n3                                 0.0                                0.0   \n4                                 0.0                                0.0   \n5                                 0.0                                0.0   \n6                                 0.0                                0.0   \n7                                 0.0                                0.0   \n8                                 0.0                                0.0   \n9                                 0.0                                0.0   \n\n   setviewportsize_height  setviewportsize_width  click_rage  \\\n0                     0.0                    0.0           0   \n1                     0.0                    0.0           1   \n2                     0.0                    0.0           0   \n3                     0.0                    0.0           1   \n4                     0.0                    0.0           0   \n5                     0.0                    0.0           0   \n6                     0.0                    0.0           0   \n7                     0.0                    0.0           0   \n8                     0.0                    0.0           0   \n9                     0.0                    0.0           1   \n\n   missing_resource  dead_click  js_exception  bad_request  cpu  memory  \n0                 0           0             0            0    0       0  \n1                 0           0             0            0    0       0  \n2                 0           0             0            0    1       0  \n3                 1           0             0            0    1       0  \n4                 0           0             0            0    0       0  \n5                 0           0             0            0    1       0  \n6                 0           0             0            0    0       0  \n7                 0           0             0            0    0       0  \n8                 0           1             0            0    0       0  \n9                 0           0             0            0    0       0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sessionid</th>\n      <th>events_count</th>\n      <th>pages_count</th>\n      <th>error_event</th>\n      <th>pageevent_firstcontentfulpaint</th>\n      <th>pageevent_firstpaint</th>\n      <th>pageevent_speedindex</th>\n      <th>pagerendertiming_timetointeractive</th>\n      <th>pagerendertiming_visuallycomplete</th>\n      <th>setviewportsize_height</th>\n      <th>setviewportsize_width</th>\n      <th>click_rage</th>\n      <th>missing_resource</th>\n      <th>dead_click</th>\n      <th>js_exception</th>\n      <th>bad_request</th>\n      <th>cpu</th>\n      <th>memory</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>4323603410944837</td>\n      <td>2</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>4323604627948361</td>\n      <td>11</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4323603204776517</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4323548146402182</td>\n      <td>37</td>\n      <td>7</td>\n      <td>0.0</td>\n      <td>3940.0</td>\n      <td>3850.0</td>\n      <td>3825.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4323554393301661</td>\n      <td>10</td>\n      <td>3</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>4323594072990251</td>\n      <td>17</td>\n      <td>5</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>166.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>4323655897228014</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>4323630712718240</td>\n      <td>7</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>4323592422793765</td>\n      <td>11</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>2312.0</td>\n      <td>2251.0</td>\n      <td>2312.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>4323618781102971</td>\n      <td>7</td>\n      <td>2</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session_features.head(10)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "outputs": [
    {
     "data": {
      "text/plain": "(57794, 18)"
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session_features.shape"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "outputs": [],
   "source": [
    "def add_one_hot_encoded_feature(origin_df, session_features, column):\n",
    "    df = origin_df[['sessionid', column]]\n",
    "    dummies = pd.get_dummies(df[column], prefix=column, dummy_na=True)\n",
    "    df = pd.concat([df, dummies], axis=1)\n",
    "    session_features = session_features.merge(df, how='left', on='sessionid')\n",
    "    session_features = session_features.drop([column], axis=1)\n",
    "    return session_features"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "outputs": [],
   "source": [
    "for col in ['user_browser', 'user_country', 'user_device', 'connection_type']:\n",
    "    session_features = add_one_hot_encoded_feature(sessions_table, session_features, col)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "outputs": [
    {
     "data": {
      "text/plain": "(57794, 34)"
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session_features.shape"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "outputs": [],
   "source": [
    "columns_to_merge = ['connection_effective_bandwidth', 'session_start_timestamp',\n",
    "                    'session_duration', 'user_device_heap_size',\n",
    "                    'user_device_memory_size', 'avg_cpu', 'avg_fps',  'max_cpu',\n",
    "                    'max_fps', 'max_total_js_heap_size', 'max_used_js_heap_size',\n",
    "                    'js_exceptions_count', 'long_tasks_total_duration', 'long_tasks_max_duration',\n",
    "                    'long_tasks_count', 'inputs_count', 'clicks_count', 'sessionid'\n",
    "                    ]\n",
    "session_features = session_features.merge(sessions_table[columns_to_merge],\n",
    "                                          how='left',\n",
    "                                          on='sessionid')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "outputs": [
    {
     "data": {
      "text/plain": "          sessionid  events_count  pages_count  error_event  \\\n0  4323603410944837             2            1          0.0   \n1  4323604627948361            11            1          0.0   \n2  4323603204776517             1            1          0.0   \n3  4323548146402182            37            7          0.0   \n4  4323554393301661            10            3          0.0   \n\n   pageevent_firstcontentfulpaint  pageevent_firstpaint  pageevent_speedindex  \\\n0                             0.0                   0.0                   0.0   \n1                             0.0                   0.0                   0.0   \n2                             0.0                   0.0                   0.0   \n3                          3940.0                3850.0                3825.0   \n4                             0.0                   0.0                   0.0   \n\n   pagerendertiming_timetointeractive  pagerendertiming_visuallycomplete  \\\n0                                 0.0                                0.0   \n1                                 0.0                                0.0   \n2                                 0.0                                0.0   \n3                                 0.0                                0.0   \n4                                 0.0                                0.0   \n\n   setviewportsize_height  ...  max_cpu  max_fps  max_total_js_heap_size  \\\n0                     0.0  ...     89.0    120.0              51399294.0   \n1                     0.0  ...      0.0      0.0                     0.0   \n2                     0.0  ...     90.0    190.0              49323074.0   \n3                     0.0  ...     68.0     60.0             140152925.0   \n4                     0.0  ...     38.0     61.0              86162824.0   \n\n   max_used_js_heap_size  js_exceptions_count  long_tasks_total_duration  \\\n0             46629158.0                  0.0                     9161.0   \n1                    0.0                  0.0                        0.0   \n2             47140794.0                  0.0                      407.0   \n3            121761837.0                  0.0                        0.0   \n4             82145777.0                  0.0                        0.0   \n\n   long_tasks_max_duration  long_tasks_count  inputs_count  clicks_count  \n0                     68.0               7.0           0.0           0.0  \n1                      0.0               0.0           1.0           0.0  \n2                     73.0               3.0           1.0           0.0  \n3                      0.0               0.0           0.0           0.0  \n4                      0.0               0.0           0.0           0.0  \n\n[5 rows x 51 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sessionid</th>\n      <th>events_count</th>\n      <th>pages_count</th>\n      <th>error_event</th>\n      <th>pageevent_firstcontentfulpaint</th>\n      <th>pageevent_firstpaint</th>\n      <th>pageevent_speedindex</th>\n      <th>pagerendertiming_timetointeractive</th>\n      <th>pagerendertiming_visuallycomplete</th>\n      <th>setviewportsize_height</th>\n      <th>...</th>\n      <th>max_cpu</th>\n      <th>max_fps</th>\n      <th>max_total_js_heap_size</th>\n      <th>max_used_js_heap_size</th>\n      <th>js_exceptions_count</th>\n      <th>long_tasks_total_duration</th>\n      <th>long_tasks_max_duration</th>\n      <th>long_tasks_count</th>\n      <th>inputs_count</th>\n      <th>clicks_count</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>4323603410944837</td>\n      <td>2</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>89.0</td>\n      <td>120.0</td>\n      <td>51399294.0</td>\n      <td>46629158.0</td>\n      <td>0.0</td>\n      <td>9161.0</td>\n      <td>68.0</td>\n      <td>7.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>4323604627948361</td>\n      <td>11</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4323603204776517</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>90.0</td>\n      <td>190.0</td>\n      <td>49323074.0</td>\n      <td>47140794.0</td>\n      <td>0.0</td>\n      <td>407.0</td>\n      <td>73.0</td>\n      <td>3.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4323548146402182</td>\n      <td>37</td>\n      <td>7</td>\n      <td>0.0</td>\n      <td>3940.0</td>\n      <td>3850.0</td>\n      <td>3825.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>68.0</td>\n      <td>60.0</td>\n      <td>140152925.0</td>\n      <td>121761837.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4323554393301661</td>\n      <td>10</td>\n      <td>3</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>38.0</td>\n      <td>61.0</td>\n      <td>86162824.0</td>\n      <td>82145777.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 51 columns</p>\n</div>"
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session_features = session_features.dropna(how='all', axis=0)\n",
    "session_features = session_features.fillna(0)\n",
    "session_features.head()\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "outputs": [
    {
     "data": {
      "text/plain": "(57794, 51)"
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session_features.shape"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "outputs": [],
   "source": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3. Build model"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Decision tree model is chosen because it is known to work great with heterogenous datasets and correlated features"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "import sklearn\n",
    "from sklearn.model_selection import train_test_split"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = train_test_split(session_features.drop(['sessionid'], axis=1),\n",
    "                                                    labels,\n",
    "                                                    test_size=0.15,\n",
    "                                                    random_state=42)\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "outputs": [
    {
     "data": {
      "text/plain": "(6, 17)"
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Ensure that test set has paying clients\n",
    "sum(y_test), sum(y_train)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "outputs": [
    {
     "data": {
      "text/plain": "0.0003460630241836984"
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Ratio of paying sessions will be denoted by EPSILON\n",
    "EPSILON = y_train[y_train == 1].shape[0]/y_train.shape[0]\n",
    "EPSILON"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\david\\appdata\\local\\programs\\python\\python38\\lib\\site-packages\\xgboost\\sklearn.py:888: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[17:33:08] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
     ]
    },
    {
     "data": {
      "text/plain": "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n              colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,\n              importance_type='gain', interaction_constraints='',\n              learning_rate=0.300000012, max_delta_step=0, max_depth=6,\n              min_child_weight=1, missing=nan, monotone_constraints='()',\n              n_estimators=100, n_jobs=8, num_parallel_tree=1, random_state=0,\n              reg_alpha=0, reg_lambda=1, scale_pos_weight=1, subsample=1,\n              tree_method='exact', validate_parameters=1, verbosity=None)"
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgc0 = xgb.XGBClassifier()\n",
    "xgc0.fit(x_train, y_train)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[17:33:09] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
     ]
    },
    {
     "data": {
      "text/plain": "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n              colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,\n              importance_type='gain', interaction_constraints='',\n              learning_rate=0.300000012, max_delta_step=0, max_depth=6,\n              min_child_weight=1, missing=nan, monotone_constraints='()',\n              n_estimators=100, n_jobs=8, num_parallel_tree=1, random_state=0,\n              reg_alpha=0, reg_lambda=1, scale_pos_weight=2889.6470588235293,\n              subsample=1, tree_method='exact', validate_parameters=1,\n              verbosity=None)"
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgc = xgb.XGBClassifier(scale_pos_weight=1/EPSILON)\n",
    "xgc.fit(x_train, y_train)\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4. Evaluate and choose the best model\n",
    "\n",
    "In this section we will built two models\n",
    "and see at their performances using different metrics"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from xgboost import plot_importance\n",
    "from sklearn.metrics import plot_roc_curve, recall_score, precision_score, accuracy_score, confusion_matrix\n",
    "from sklearn.metrics import plot_confusion_matrix, precision_recall_curve, plot_precision_recall_curve\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "outputs": [],
   "source": [
    "def report(models, x_test, y_test, y_pred=None, model_names=None):\n",
    "\n",
    "    fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 14))\n",
    "\n",
    "    for model, name in zip(models, model_names):\n",
    "\n",
    "        y_out = model.predict(x_test)\n",
    "\n",
    "        precision = precision_score(y_test, y_out)\n",
    "        recall = recall_score(y_test, y_out)\n",
    "        accuracy = accuracy_score(y_test, y_out)\n",
    "\n",
    "        print(f'Model: {name}')\n",
    "        print(f'Precision: {precision}')\n",
    "        print(f'Recall: {recall}')\n",
    "        print(f'Accuracy: {accuracy}')\n",
    "        print('-------------------------')\n",
    "        print()\n",
    "\n",
    "        roc_auc = plot_roc_curve(model, x_test, y_test, ax=ax1)\n",
    "        precision_recall = plot_precision_recall_curve(model, x_test, y_test, ax=ax2)\n",
    "\n",
    "    date_time = datetime.now().strftime(\"%m_%d_%H_%M_%S\")\n",
    "    plt.savefig(f'report_{date_time}.png', dpi=300)\n",
    "    plt.show()\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: XGBClassifier\n",
      "Precision: 0.8\n",
      "Recall: 0.6666666666666666\n",
      "Accuracy: 0.9996539792387543\n",
      "-------------------------\n",
      "\n",
      "Model: Weighted XGBClassifier\n",
      "Precision: 0.6666666666666666\n",
      "Recall: 1.0\n",
      "Accuracy: 0.9996539792387543\n",
      "-------------------------\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 864x1008 with 2 Axes>",
      "image/png": "[encoded data, 38700 characters]\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "report([xgc0, xgc], x_test, y_test, model_names=['XGBClassifier', 'Weighted XGBClassifier'])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "For the best model let's see the features that influenced the decision the most:\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "outputs": [
    {
     "data": {
      "text/plain": "<AxesSubplot:title={'center':'Feature importance'}, xlabel='F score', ylabel='Features'>"
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "[encoded data, 26976 characters]\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_importance(xgc0)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "outputs": [],
   "source": [
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}