* feat(backend): handlers for mobile messages
* feat(backend): new service template
* feat(backend): save mobile session start and send batches to kafka
* feat(backend): added mobile support to sink, ender and storage
* helm(videostorage): added helm chart for a new service videostorage
* fix(backend): added pointer to streing for userBrowser (because it's null for mobile sessions)
* feat(backend): added MsgIOSBatchMeta handler to message iterator's logic
* feat(backend): added ios ts parser to ender
* feat(backend): enabled producing batch of messages to queue
* feat(backend): removed iosstart from mob files
* feat(backend): added new ios message types
* feat(backend): added iosStart and iosEnd
* fix(backend): fixed log issue
* feat(backend): send tar.gz archives to special queue topic
* feat(backend): read raw archives from kafka
* fix(backend): added missing file
* fix(backend): removed the second file reading
* fix(backend): fixed wrong queue topic name
* fix(backend): fixed mobile trigger topic name
* feat(backend): added tar.gz extractor and iOSSessionEnd handler
* feat(backend): debug logs on message uploading
* fix(backend): added raw-images topic consumption
* feat(backend): now sink send iosSessionEnd to video-storage
* feat(backend): added dir creator for new sessions
* feat(backend): added correct command to execute
* feat(backend): added overwrite option
* feat(backend): added s3 uploader for video session replays
* feat(backend): new consumer group for mobile sessions
* feat(backend): debug logs for uploader
* feat(backend): removed unused log
* feat(backend): fixed s3 key for video replays
* feat(backend): removed debug code
* feat(backend): fixed video-storage message filter
* fix(backend): added mobileSessionEnd to SessionEnd converter
* feat(backend): added first version if db support for mobile events
* fix(backend): added swipe events to mob file
* feat(backend): added swipe event to pg
* feat(backend): split logic into 2 services: image-storage and video-storage
* feat(backend): added helm chart for image-storage service
* fix(backend): fixed table name for mobile taps
* feat(backend): added metadata handler for mobile message parser + fix message filters
* feat(backend): added iosRawTopic to DB message consumer
* fix(backend): removed value from mobile inputs
* feat(backend): removed debug log from iterator
* feat(backend): added new apple devices to iOS device parser
* fix(backend): added real projectID instead of 0
* feat(backend): extended a list of simulators for device detector
* feat(backend): updated networkCall mobile message
* fix(backend): added new way to define is network call successed or not
* feat(backend): added timezone support for mobile start request
* feat(backend): added 2 mobile events Input and Click to mob file
* feat(backend): refactored image storage
* feat(backend): video storage with 2 workers
* feat(backend): added project's platform support
* feat(backend): added memory size field for mobile start request
* feat(backend): changed video preset for ultrafast
* feat(backend): added debug log to http /late handler
* feat(backend): added debug log to db service for iosCrash messages
* feat(backend): added tapRage event handler to heuristics
* feat(backend): changed table and field names for ios crashes
* feat(backend): added payload for tapRage events
* feat(backend): added TapRage events insert to DB
* feat(backend): added fps value to /mobile/start response
* feat(backend): added image quality parameter to /mobile/start response
* feat(backend): added ScreenLeave handler
* feat(backend): removed screenEnter and screenLeave events, added new viewComponent event
---------
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): removed legacy InputEvent builder from heuristics service
* feat(backend): removed InputEvent handler from DB service
* feat(backend): removed InputEvent from ClickHouse connector
* feat(backend): added extra env variable for brotli compression algorithm
* feat(backend): redshift connector draft
* fix(backend): fixed memory leak, empty string ddos
* feat(backend): draft for sessions part
* feat(backend): session handler
* fix(backend): fixed wrong columns list in sessionToCSV convertor
* feat(backend): load session info from db/cache if there is nothing in memory when sessionEnd event recieved
* feat(backend): added filters for connector
* feat(backend): memory leak fix + extra cache for sessions
* feat(backend): moved table names as an env variable
* fix(backend): added timeout for last session messages to avoid memory leak
* fix(backend): fixed last memory leak
* feat(backend): moved redshift connector to ee folder
* feat(backend): new session end detection logic + several fixes
* feat(backend): support partitions managment in ender
* feat(backend): added new consumer support to redis
* feat(backend): added support for new consumer in kafka
* feat(backend): added new consumer support to redis (ee)
* feat(backend): small refactoring in ender
* fix(chalice): cast note's session_id to string
* fix(chalice): cast all session_id to string
* fix(chalice): fixed content-length after cast all session_id to string
* fix(connectors): added try method for redshift replace in fill
* fix(connectors): adding userId only if not empty string
* fix(connectors): Added fix as a worker
* fix(connector): set env variable for replace and set it to one minute default
* style(connector): different log for redshift and pg
* style(connector): different log for redshift select and replace
* fix(connector): replacing from oldest to newest to avoid blocking
* fix(connector): empty string changed to NN and fixed str issue for cron job
* Changing methods in connector
* fix(connectors): solved issues when replacing with null, reduced number of queries to redshift
* fix(connectors): fixed save method
* fix(connectors): fixed issue while saving event object
* feat(backend): moved sql requests related to sessions table to one place
* feat(backend): refactoring in db.Saver handler
* feat(backend): hude refactoring in db/postgres module
* fix(backend): workable feature flags
* fix(backend): workable integrations
* fix(backend): workable sessions and projects modules
* fix(backend): added missed projects module to sessions
* feat(backend): renaming
* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels
* feat(backend): moved project struct to projects module
* feat(backend): added projects model
* feat(backend): implemented new in memory cache for sessions and projects
* feat(backend): implemented new cache in projects
* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()
* feat(backend): added cache update operations
* fix(backend): fixed import cycle
* fix(backend): fixed panic in db message handler
* fix(backend): fixed panic in projects module
* fix(backend): fixed panic in sessions.GetDuration
* feat(backend): added direct call to get session duration if session is already in cache
* feat(backend): used pg pool everywhere except db service
* fix(backend): added missing part after rebase
* fix(backend): removed old sessions file
* feat(backend): added refactored redis client with produce/consume options
* feat(backend): added cache layer for projects
* fix(backend): added missing redis config
* fix(backend): added missing method for producer
* feat(backend): cache integration for sessions
* feat(backend): temporary method to get session directly from db
* feat(backend): adapt EE version of message handler
* fix(backend): fixed issue in fts realisation
* fix(backend): added redis cache to sessions module
* fix(backend): set 0 duration or hesitation time for inputs without focus event
* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation
* feat(backend): debug log
* feat(backend): more debug log
* feat(backend): removed debug log
* fix(backend): fixed an issue of tracking input events with empty label
* fix(backend): disabled debug log in projects cache
* fix(backend): renamed session updater
* fix(backend): fixed closed pool issue in DB service
* fix(backend): fixed dead lock in db Stop() method
* fix(backend): fixed panic in heuristics service
* feat(backend): enabled redis cache in projects
* feat(backend): clear cache on each update operation
* feat(backend): fully integrated cache layer with auto switch
* feat(backend): small refactoring in session updates
* fix(backend): fixed wrong events counter issue
* feat(backend): enabled full cache support in ender and http services
* fix(backend/ee): added missed import
* feat(backend): added second cache layer for db to speed up the service
* feat(backend): disable redis cache
* feat(backend): moved redis cache to ee
* fix for memory cleansing and added setup files for msgcodec build
* Kafka reader now calling only once to PG to verify sessionIds, updated fill missing user_ids script
* fix(connector): small error while reading from pg
---------
Co-authored-by: MauricioGarciaS <47052044+MauricioGarciaS@users.noreply.github.com>
* Updated pg connector
* fix(player): fix first 8 byte checker
* fix(player): fix commit conflict
* Added pylint
* Removed pylint for incompatible license
* change(ui): check for sessions records state
* Patch/api v1.12.0 (#1299)
* fix(chalice): include metadata in sessions exp search
* fix(chalice): fixed sessions exp search wrong col name
* fix(chalice): removed cookies
* fix(chalice): changed base image to support SSO/xmlsec
* fix(chalice): changed Dockerfile to support SSO/xmlsec
* fix(chalice): changed Dockerfile to support SSO/xmlsec
(cherry picked from commit 4b8cf9742c)
* fix(ui): project fallback to recorded variable
* Patch/api v1.12.0 (#1301)
* fix(chalice): changed base image to support SSO/xmlsec
* fix(chalice): fixed exp search null metadata
(cherry picked from commit ab000751d2)
* change(ui): assist no content message styles and icons
* change(ui): revert menu disable
* fix(connector): Added method to save state in s3 for redshift if sigterm arise
* Rewriting python code in cython
* Added pyx module for messages
* Auto create pyx files
* Updated and fixed msgcodec.pyx
* Added new module to connector code
* Updated kafka lib for base image
* cleaned Docker and updated base image version for pandas
* cleaned prints
* Added code to fetch data from db and add it into redshift
* Updated consumer reading method. Async multithreading over sessionId
* Added split for country (Country,State,City)
* Fixed decoding issue for uint
* Created service able to fix data from redshift by reading from db
* Handle when process ended because of lost connection to pg, country set to country value only
