cea5eda985
* Created two services: recommendation training and recommendation serving * Deleted Docker temporary * Added features based in signals information * Added method to get sessions features using PG * Added same utils and core elements into ml_trainer * Added checks before training models, added handler for model serving * Updated serving API and recommendation functions to use frontend signals features * reorganized modules to have base image and for both serving and training * Added Dockerfiles and base Dockerfile * Solved issue while ordering sessions by relevance * Added method to save user feedback of recommendations * Added security authorization * Updated Dockerfile * fixed issues with secret insertion to API * Updated feedback structure * Added git for dags * Solved issue of insertion on recommendation feedback * Changed update method from def to async def and it is called during startup * Solved issues of airflow running mlflow in dag * Changes sanity checks and added middleware params * base path renaming * Changed update method to a interval method which loads one model each 10s if there are models to download * Added sql files for recommendation service and trainer * Cleaned files and added documentation for methods and classes * Added README file * Renamed endpoints, changed None into empty array and updated readme * refactor(recommendation): optimized query * style(recommendation): changed import to top file, renamed endpoints parameters, function optimization * refactor(recommendation): .gitignore * refactor(recommendation): .gitignore * refactor(recommendation): Optimized Dockerfiles * refactor(recommendation): changed imports * refactor(recommendation): optimized requests * refactor(recommendation): optimized requests * Fixed boot for fastapi, updated some queries * Fixed issues while downloading models and while returning json response from API * limited number of recommendations and set a minimum score to present recommendations * fix(recommendation): fixed some queries and updated prediction method * Added env value to control number of predictions to make * docs(recommendation): Added third party libraries used in recommendation service * frozen requirements * Update base_crons.py added `misfire_grace_time` to recommendation crons --------- Co-authored-by: Taha Yassine Kraiem <tahayk2@gmail.com>
24 lines
848 B
Python
24 lines
848 B
Python
from utils.declarations import CountryValue, DeviceValue
|
|
|
|
|
|
def _add_to_dict(element, index, dictionary):
|
|
if element not in dictionary.keys():
|
|
dictionary[element] = [index]
|
|
else:
|
|
dictionary[element].append(index)
|
|
|
|
|
|
def _process_pg_response(res, _X, _Y, X_project_ids, X_users_ids, X_sessions_ids, label=None):
|
|
for i in range(len(res)):
|
|
x = res[i]
|
|
_add_to_dict(x.pop('project_id'), i, X_project_ids)
|
|
_add_to_dict(x.pop('session_id'), i, X_sessions_ids)
|
|
_add_to_dict(x.pop('user_id'), i, X_users_ids)
|
|
if label is None:
|
|
_Y.append(int(x.pop('train_label')))
|
|
else:
|
|
_Y.append(label)
|
|
|
|
x['country'] = CountryValue(x['country']).get_int_val()
|
|
x['device_type'] = DeviceValue(x['device_type']).get_int_val()
|
|
_X.append(list(x.values()))
|