* fix(connector): set env variable for replace and set it to one minute default
* style(connector): different log for redshift and pg
* style(connector): different log for redshift select and replace
* fix(connector): replacing from oldest to newest to avoid blocking
* fix(connector): empty string changed to NN and fixed str issue for cron job
* Changing methods in connector
* fix(connectors): solved issues when replacing with null, reduced number of queries to redshift
* fix(connectors): fixed save method
* fix(connectors): fixed issue while saving event object
* feat(backend): moved sql requests related to sessions table to one place
* feat(backend): refactoring in db.Saver handler
* feat(backend): hude refactoring in db/postgres module
* fix(backend): workable feature flags
* fix(backend): workable integrations
* fix(backend): workable sessions and projects modules
* fix(backend): added missed projects module to sessions
* feat(backend): renaming
* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels
* feat(backend): moved project struct to projects module
* feat(backend): added projects model
* feat(backend): implemented new in memory cache for sessions and projects
* feat(backend): implemented new cache in projects
* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()
* feat(backend): added cache update operations
* fix(backend): fixed import cycle
* fix(backend): fixed panic in db message handler
* fix(backend): fixed panic in projects module
* fix(backend): fixed panic in sessions.GetDuration
* feat(backend): added direct call to get session duration if session is already in cache
* feat(backend): used pg pool everywhere except db service
* fix(backend): added missing part after rebase
* fix(backend): removed old sessions file
* feat(backend): added refactored redis client with produce/consume options
* feat(backend): added cache layer for projects
* fix(backend): added missing redis config
* fix(backend): added missing method for producer
* feat(backend): cache integration for sessions
* feat(backend): temporary method to get session directly from db
* feat(backend): adapt EE version of message handler
* fix(backend): fixed issue in fts realisation
* fix(backend): added redis cache to sessions module
* fix(backend): set 0 duration or hesitation time for inputs without focus event
* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation
* feat(backend): debug log
* feat(backend): more debug log
* feat(backend): removed debug log
* fix(backend): fixed an issue of tracking input events with empty label
* fix(backend): disabled debug log in projects cache
* fix(backend): renamed session updater
* fix(backend): fixed closed pool issue in DB service
* fix(backend): fixed dead lock in db Stop() method
* fix(backend): fixed panic in heuristics service
* feat(backend): enabled redis cache in projects
* feat(backend): clear cache on each update operation
* feat(backend): fully integrated cache layer with auto switch
* feat(backend): small refactoring in session updates
* fix(backend): fixed wrong events counter issue
* feat(backend): enabled full cache support in ender and http services
* fix(backend/ee): added missed import
* feat(backend): added second cache layer for db to speed up the service
* feat(backend): disable redis cache
* feat(backend): moved redis cache to ee
* fix for memory cleansing and added setup files for msgcodec build
* Kafka reader now calling only once to PG to verify sessionIds, updated fill missing user_ids script
* fix(connector): small error while reading from pg
---------
Co-authored-by: MauricioGarciaS <47052044+MauricioGarciaS@users.noreply.github.com>
* Updated pg connector
* fix(player): fix first 8 byte checker
* fix(player): fix commit conflict
* Added pylint
* Removed pylint for incompatible license
* change(ui): check for sessions records state
* Patch/api v1.12.0 (#1299)
* fix(chalice): include metadata in sessions exp search
* fix(chalice): fixed sessions exp search wrong col name
* fix(chalice): removed cookies
* fix(chalice): changed base image to support SSO/xmlsec
* fix(chalice): changed Dockerfile to support SSO/xmlsec
* fix(chalice): changed Dockerfile to support SSO/xmlsec
(cherry picked from commit 4b8cf9742c)
* fix(ui): project fallback to recorded variable
* Patch/api v1.12.0 (#1301)
* fix(chalice): changed base image to support SSO/xmlsec
* fix(chalice): fixed exp search null metadata
(cherry picked from commit ab000751d2)
* change(ui): assist no content message styles and icons
* change(ui): revert menu disable
* fix(connector): Added method to save state in s3 for redshift if sigterm arise
* Rewriting python code in cython
* Added pyx module for messages
* Auto create pyx files
* Updated and fixed msgcodec.pyx
* Added new module to connector code
* Updated kafka lib for base image
* cleaned Docker and updated base image version for pandas
* cleaned prints
* Added code to fetch data from db and add it into redshift
* Updated consumer reading method. Async multithreading over sessionId
* Added split for country (Country,State,City)
* Fixed decoding issue for uint
* Created service able to fix data from redshift by reading from db
* Handle when process ended because of lost connection to pg, country set to country value only
* fix(backend): fixed issue with wrong expiration timeout for projects in cache layer
* fix(backend): removed direct call db.GetProject()
* feat(backend): set default PROJECT_EXPIRATION to 10 minutes
* feat(backend): implemented azure blob storage support
* feat(backend): added azure implementation to backend services
* feat(backend): added azure blob storage support to chalice service
* fix(backend): removed prev version of s3
* feat(backend): moved azure support to ee subfolder
* feat(backend): prepared ee code for new utils.objects package
* feat(backend): added missed modules to go.mod
* feat(backend): added missed modules to go.sum
* feat(backend): go mod tidy
* feat(backend): temporary made s3 vars are not required
* feat(backend): added azure lib to ee chalice
* feat(api): changed azure env var name
* feat(api): added new object store and extra methods to chalice ee
* fix(api): added azure blob lib to alerts
* fix(api): fixed incorrect call in sessions_devtool
* fix(crons): added azure blob storage library to requirements list
* chore(build): Error message for not providing flag.
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): removed ios headers and object store for ios messages
* feat(backend): object config refactoring
* chore(helm): Update BUCKET_NAME
* fix(backend): removed object storage usage in http
* feat(backend): added debug logs to azure upload method
* fix(backend): fixed empty folder issue
* fix(backend): removed extra debug log
* chore(helm): Adding global variables for crons
* chore(helm): Remove clickhouse resource limit
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* fix(backend): removed assets debug log
* feat(api): use ABC class instead of empty interface
* feat(api): renamed helpers to generators
* feat(api): changed prep/clean dev scripts
* feat(api): changed name obj_store -> StorageClient
* feat(api): some changes after code review
* fix(api): removed unnecesery packages in oss api
* feat(backend): moved azure implementation to ee folder
---------
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* Created two services: recommendation training and recommendation serving
* Deleted Docker temporary
* Added features based in signals information
* Added method to get sessions features using PG
* Added same utils and core elements into ml_trainer
* Added checks before training models, added handler for model serving
* Updated serving API and recommendation functions to use frontend signals features
* reorganized modules to have base image and for both serving and training
* Added Dockerfiles and base Dockerfile
* Solved issue while ordering sessions by relevance
* Added method to save user feedback of recommendations
* Added security authorization
* Updated Dockerfile
* fixed issues with secret insertion to API
* Updated feedback structure
* Added git for dags
* Solved issue of insertion on recommendation feedback
* Changed update method from def to async def and it is called during startup
* Solved issues of airflow running mlflow in dag
* Changes sanity checks and added middleware params
* base path renaming
* Changed update method to a interval method which loads one model each 10s if there are models to download
* Added sql files for recommendation service and trainer
* Cleaned files and added documentation for methods and classes
* Added README file
* Renamed endpoints, changed None into empty array and updated readme
* refactor(recommendation): optimized query
* style(recommendation): changed import to top file, renamed endpoints parameters, function optimization
* refactor(recommendation): .gitignore
* refactor(recommendation): .gitignore
* refactor(recommendation): Optimized Dockerfiles
* refactor(recommendation): changed imports
* refactor(recommendation): optimized requests
* refactor(recommendation): optimized requests
* Fixed boot for fastapi, updated some queries
* Fixed issues while downloading models and while returning json response from API
* limited number of recommendations and set a minimum score to present recommendations
* fix(recommendation): fixed some queries and updated prediction method
* Added env value to control number of predictions to make
* docs(recommendation): Added third party libraries used in recommendation service
* frozen requirements
* Update base_crons.py
added `misfire_grace_time` to recommendation crons
---------
Co-authored-by: Taha Yassine Kraiem <tahayk2@gmail.com>
