* [Backend] Canvas support (#1705)
* feat(http): added new parameters to start response and new endpoint for canvas screenshorts
* fix(http): added new topic to dockerfile
* feat(http): try different multipart parser
* feat(image-storage): reused the same workflow for canvas topic handler
* feat(video-storage): new canvas parser and ffmpeg script
* feat(video-storage): use correct replay name for canvas
* feat(backend): added new message (CanvasNode)
* feat(backend): add canvas record events to db
* Async chalice.core.tenants:tenants_exists, and propagate
* rework after review
* chore(api): asyncify `/projects` and propagate.
Impact on the following routes:
/projects
/v1/projects
/passowrd/reset
/metadata/session_search
* fix(api): there is no cnx.mogrify method.
In psycopg v3, the mogrify method is only available on cursor objects.
In other words, just use cnx.execute(query, kwargs), except when
opening an "explicit cursor session".
ref: https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.ClientCursor.mogrify
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* feat(http): added new parameters to start response and new endpoint for canvas screenshorts
* fix(http): added new topic to dockerfile
* feat(http): try different multipart parser
* feat(image-storage): reused the same workflow for canvas topic handler
* feat(video-storage): new canvas parser and ffmpeg script
* feat(video-storage): use correct replay name for canvas
* feat(backend): added new message (CanvasNode)
* feat(backend): add canvas record events to db
* feat(backend): implemented missing new method for azure
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* feat(backend): added ux-testing support
* feat(backend): added ux-testing module
* feat(http): added bucket name for http service
* feat(backend): fixed small typos in http router
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* fix(ui): fix up mobile recordings display
* fix(ui): some messages
* fix(ui): some messages
* fix(player): fix msg generation for ios messages
* feat(player): add generic mmanager interface for ios player impl
* feat(player): mobile player and message manager; touch manager; videoplayer
* feat(player): add iphone shells, add log panel,
* feat(player): detect ios sessions and inject correct player
* feat(player): move screen mapper to utils
* feat(player): events panel for mobile, map shell sizes to device type data,
* feat(player): added network tab to mobile player; unify network message (ios and web)
* feat(player): resize canvas up to phone screen size, fix capitalize util
* feat(player): some general changes to support mobile events and network entries
* feat(player): remove swipes from timeline
* feat(player): more stuff for list walker
* fix(ui): performance tab, mobile project typings and form
* fix(ui):some ui touches for ios replayer shell
* fix(ui): more fixes for ui, new onboarding screen for mobile projects
* feat(ui): mobile overview panel (xray)
* feat(ui): fixes for phone shell and tap events
* fix(tracker): change phone shells and sizes
* fix(tracker): fix border on replay screen
* feat(ui): use crashes from db to show in session
* feat(ui): use event name for xray
* feat(ui): some overall ui fixes
* feat(ui): IOS -> iOS
* feat(ui): change tags to ant d
* fix(ui): fast fix
* fix(ui): fix for capitalizer
* fix(ui): fix for browser display
* fix(ui): fix for note popup
* fix(ui): change exceptions display
* fix(ui): add click rage to ios xray
* fix(ui): some icons and resizing
* fix(ui): fix ios context menu overlay, fix console logs creation for ios
* feat(ui): added icons
* feat(ui): performance warnings
* feat(ui): performance warnings
* feat(ui): different styles
* feat(ui): rm debug true
* feat(ui): fix warnings display
* feat(ui): some styles for animation
* feat(ui): add some animations to warnings
* feat(ui): move perf warnings to performance graph
* feat(ui): hide/show warns dynamically
* feat(ui): new mobile touch animation
* fix(tracker): update msg for ios
* fix(ui): taprage fixes
* fix(ui): regenerate icons and messages
* fix(ui): fix msgs
* fix(backend): fix events gen
* fix(backend): fix userid msg
* feat(backend): handlers for mobile messages
* feat(backend): new service template
* feat(backend): save mobile session start and send batches to kafka
* feat(backend): added mobile support to sink, ender and storage
* helm(videostorage): added helm chart for a new service videostorage
* fix(backend): added pointer to streing for userBrowser (because it's null for mobile sessions)
* feat(backend): added MsgIOSBatchMeta handler to message iterator's logic
* feat(backend): added ios ts parser to ender
* feat(backend): enabled producing batch of messages to queue
* feat(backend): removed iosstart from mob files
* feat(backend): added new ios message types
* feat(backend): added iosStart and iosEnd
* fix(backend): fixed log issue
* feat(backend): send tar.gz archives to special queue topic
* feat(backend): read raw archives from kafka
* fix(backend): added missing file
* fix(backend): removed the second file reading
* fix(backend): fixed wrong queue topic name
* fix(backend): fixed mobile trigger topic name
* feat(backend): added tar.gz extractor and iOSSessionEnd handler
* feat(backend): debug logs on message uploading
* fix(backend): added raw-images topic consumption
* feat(backend): now sink send iosSessionEnd to video-storage
* feat(backend): added dir creator for new sessions
* feat(backend): added correct command to execute
* feat(backend): added overwrite option
* feat(backend): added s3 uploader for video session replays
* feat(backend): new consumer group for mobile sessions
* feat(backend): debug logs for uploader
* feat(backend): removed unused log
* feat(backend): fixed s3 key for video replays
* feat(backend): removed debug code
* feat(backend): fixed video-storage message filter
* fix(backend): added mobileSessionEnd to SessionEnd converter
* feat(backend): added first version if db support for mobile events
* fix(backend): added swipe events to mob file
* feat(backend): added swipe event to pg
* feat(backend): split logic into 2 services: image-storage and video-storage
* feat(backend): added helm chart for image-storage service
* fix(backend): fixed table name for mobile taps
* feat(backend): added metadata handler for mobile message parser + fix message filters
* feat(backend): added iosRawTopic to DB message consumer
* fix(backend): removed value from mobile inputs
* feat(backend): removed debug log from iterator
* feat(backend): added new apple devices to iOS device parser
* fix(backend): added real projectID instead of 0
* feat(backend): extended a list of simulators for device detector
* feat(backend): updated networkCall mobile message
* fix(backend): added new way to define is network call successed or not
* feat(backend): added timezone support for mobile start request
* feat(backend): added 2 mobile events Input and Click to mob file
* feat(backend): refactored image storage
* feat(backend): video storage with 2 workers
* feat(backend): added project's platform support
* feat(backend): added memory size field for mobile start request
* feat(backend): changed video preset for ultrafast
* feat(backend): added debug log to http /late handler
* feat(backend): added debug log to db service for iosCrash messages
* feat(backend): added tapRage event handler to heuristics
* feat(backend): changed table and field names for ios crashes
* feat(backend): added payload for tapRage events
* feat(backend): added TapRage events insert to DB
* feat(backend): added fps value to /mobile/start response
* feat(backend): added image quality parameter to /mobile/start response
* feat(backend): added ScreenLeave handler
* feat(backend): removed screenEnter and screenLeave events, added new viewComponent event
---------
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): removed legacy InputEvent builder from heuristics service
* feat(backend): removed InputEvent handler from DB service
* feat(backend): removed InputEvent from ClickHouse connector
* feat(backend): added extra env variable for brotli compression algorithm
* feat(backend): redshift connector draft
* fix(backend): fixed memory leak, empty string ddos
* feat(backend): draft for sessions part
* feat(backend): session handler
* fix(backend): fixed wrong columns list in sessionToCSV convertor
* feat(backend): load session info from db/cache if there is nothing in memory when sessionEnd event recieved
* feat(backend): added filters for connector
* feat(backend): memory leak fix + extra cache for sessions
* feat(backend): moved table names as an env variable
* fix(backend): added timeout for last session messages to avoid memory leak
* fix(backend): fixed last memory leak
* feat(backend): moved redshift connector to ee folder
* feat(backend): new session end detection logic + several fixes
* feat(backend): support partitions managment in ender
* feat(backend): added new consumer support to redis
* feat(backend): added support for new consumer in kafka
* feat(backend): added new consumer support to redis (ee)
* feat(backend): small refactoring in ender
* feat(backend): moved sql requests related to sessions table to one place
* feat(backend): refactoring in db.Saver handler
* feat(backend): hude refactoring in db/postgres module
* fix(backend): workable feature flags
* fix(backend): workable integrations
* fix(backend): workable sessions and projects modules
* fix(backend): added missed projects module to sessions
* feat(backend): renaming
* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels
* feat(backend): moved project struct to projects module
* feat(backend): added projects model
* feat(backend): implemented new in memory cache for sessions and projects
* feat(backend): implemented new cache in projects
* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()
* feat(backend): added cache update operations
* fix(backend): fixed import cycle
* fix(backend): fixed panic in db message handler
* fix(backend): fixed panic in projects module
* fix(backend): fixed panic in sessions.GetDuration
* feat(backend): added direct call to get session duration if session is already in cache
* feat(backend): used pg pool everywhere except db service
* fix(backend): added missing part after rebase
* fix(backend): removed old sessions file
* feat(backend): added refactored redis client with produce/consume options
* feat(backend): added cache layer for projects
* fix(backend): added missing redis config
* fix(backend): added missing method for producer
* feat(backend): cache integration for sessions
* feat(backend): temporary method to get session directly from db
* feat(backend): adapt EE version of message handler
* fix(backend): fixed issue in fts realisation
* fix(backend): added redis cache to sessions module
* fix(backend): set 0 duration or hesitation time for inputs without focus event
* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation
* feat(backend): debug log
* feat(backend): more debug log
* feat(backend): removed debug log
* fix(backend): fixed an issue of tracking input events with empty label
* fix(backend): disabled debug log in projects cache
* fix(backend): renamed session updater
* fix(backend): fixed closed pool issue in DB service
* fix(backend): fixed dead lock in db Stop() method
* fix(backend): fixed panic in heuristics service
* feat(backend): enabled redis cache in projects
* feat(backend): clear cache on each update operation
* feat(backend): fully integrated cache layer with auto switch
* feat(backend): small refactoring in session updates
* fix(backend): fixed wrong events counter issue
* feat(backend): enabled full cache support in ender and http services
* fix(backend/ee): added missed import
* feat(backend): added second cache layer for db to speed up the service
* feat(backend): disable redis cache
* feat(backend): moved redis cache to ee
* fix(backend): fixed issue with wrong expiration timeout for projects in cache layer
* fix(backend): removed direct call db.GetProject()
* feat(backend): set default PROJECT_EXPIRATION to 10 minutes
* feat(backend): implemented azure blob storage support
* feat(backend): added azure implementation to backend services
* feat(backend): added azure blob storage support to chalice service
* fix(backend): removed prev version of s3
* feat(backend): moved azure support to ee subfolder
* feat(backend): prepared ee code for new utils.objects package
* feat(backend): added missed modules to go.mod
* feat(backend): added missed modules to go.sum
* feat(backend): go mod tidy
* feat(backend): temporary made s3 vars are not required
* feat(backend): added azure lib to ee chalice
* feat(api): changed azure env var name
* feat(api): added new object store and extra methods to chalice ee
* fix(api): added azure blob lib to alerts
* fix(api): fixed incorrect call in sessions_devtool
* fix(crons): added azure blob storage library to requirements list
* chore(build): Error message for not providing flag.
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): removed ios headers and object store for ios messages
* feat(backend): object config refactoring
* chore(helm): Update BUCKET_NAME
* fix(backend): removed object storage usage in http
* feat(backend): added debug logs to azure upload method
* fix(backend): fixed empty folder issue
* fix(backend): removed extra debug log
* chore(helm): Adding global variables for crons
* chore(helm): Remove clickhouse resource limit
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* fix(backend): removed assets debug log
* feat(api): use ABC class instead of empty interface
* feat(api): renamed helpers to generators
* feat(api): changed prep/clean dev scripts
* feat(api): changed name obj_store -> StorageClient
* feat(api): some changes after code review
* fix(api): removed unnecesery packages in oss api
* feat(backend): moved azure implementation to ee folder
---------
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend/http): added city and state support for geo extractor
* feat(backend): use new GeoLite db in Dockerfile
* feat(backend/http): added unit test
