* feat(server): moved an http server object into a pkg subdir to be reusable for http, spots, and integrations
* feat(web): isolated web module (server, router, middleware, utils) used in spots and new integrations
* feat(web): removed possible panic
* feat(web): split all handlers from http service into different packages for better management.
* feat(web): changed router's method signature
* feat(web): added missing handlers interface
* feat(web): added health middleware to remove unnecessary checks
* feat(web): customizable middleware set for web servers
* feat(web): simplified the handler's structure
* feat(web): created an unified server.Run method for all web services (http, spot, integrations)
* feat(web): fixed a json size limit issue
* feat(web): removed Keys and PG connection from router
* feat(web): simplified integration's main file
* feat(web): simplified spot's main file
* feat(web): simplified http's main file (builder)
* feat(web): refactored audit trail functionality
* feat(web): added ee version of audit trail
* feat(web): added ee version of conditions module
* feat(web): moved ee version of some web session structs
* feat(web): new format of web metrics
* feat(web): added new web metrics to all handlers
* feat(web): added justExpired feature to web ingest handler
* feat(web): added small integrations improvements
* feat(http): small prep for task implementation
* feat(http): added assist-only check to ee
* feat(backend): using a new license check url
* feat(backend): packages upgrade
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* feat(backend): added new trigger topic for mobile video replay maker
* feat(backend): mobile video maker with mix list support
* feat(backend): fixed panic
* feat(backend): removed commented code
* feat(ui/tracker): start tag n watch
* fix(tracker): test coverage, fix some watcher api
* fix(tracker): add intersectionobserver, adjust tests
* feat(tracker): relay + apollo plugins
* feat(ui): tags search
* feat(ui): tags name edit
* feat(ui): tags search icon
* feat(ui): icons for tabs in player
* feat(ui): save and find button
* feat(tracker): save tags in session storage (just in case)
* feat(ui): improve loading
* feat(ui): fix icon names gen
* feat(ui): fix typo
* feat(backend): added TagAndWatch backend support
* feat(backend): use msgIndex as a seqIndex
* feat(backend): try to use truncSqlIdx for seq_index
* feat(backend): added tags cache and tags filter to DB service
* feat(backend): removed debug logs
---------
Co-authored-by: nick-delirium <nikita@openreplay.com>
* [Backend] Canvas support (#1705)
* feat(http): added new parameters to start response and new endpoint for canvas screenshorts
* fix(http): added new topic to dockerfile
* feat(http): try different multipart parser
* feat(image-storage): reused the same workflow for canvas topic handler
* feat(video-storage): new canvas parser and ffmpeg script
* feat(video-storage): use correct replay name for canvas
* feat(backend): added new message (CanvasNode)
* feat(backend): add canvas record events to db
* Async chalice.core.tenants:tenants_exists, and propagate
* rework after review
* chore(api): asyncify `/projects` and propagate.
Impact on the following routes:
/projects
/v1/projects
/passowrd/reset
/metadata/session_search
* fix(api): there is no cnx.mogrify method.
In psycopg v3, the mogrify method is only available on cursor objects.
In other words, just use cnx.execute(query, kwargs), except when
opening an "explicit cursor session".
ref: https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.ClientCursor.mogrify
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* feat(http): added new parameters to start response and new endpoint for canvas screenshorts
* fix(http): added new topic to dockerfile
* feat(http): try different multipart parser
* feat(image-storage): reused the same workflow for canvas topic handler
* feat(video-storage): new canvas parser and ffmpeg script
* feat(video-storage): use correct replay name for canvas
* feat(backend): added new message (CanvasNode)
* feat(backend): add canvas record events to db
* feat(backend): implemented missing new method for azure
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* feat(backend): added ux-testing support
* feat(backend): added ux-testing module
* feat(http): added bucket name for http service
* feat(backend): fixed small typos in http router
---------
Co-authored-by: Alexander <zavorotynskiy@pm.me>
* feat(backend): handlers for mobile messages
* feat(backend): new service template
* feat(backend): save mobile session start and send batches to kafka
* feat(backend): added mobile support to sink, ender and storage
* helm(videostorage): added helm chart for a new service videostorage
* fix(backend): added pointer to streing for userBrowser (because it's null for mobile sessions)
* feat(backend): added MsgIOSBatchMeta handler to message iterator's logic
* feat(backend): added ios ts parser to ender
* feat(backend): enabled producing batch of messages to queue
* feat(backend): removed iosstart from mob files
* feat(backend): added new ios message types
* feat(backend): added iosStart and iosEnd
* fix(backend): fixed log issue
* feat(backend): send tar.gz archives to special queue topic
* feat(backend): read raw archives from kafka
* fix(backend): added missing file
* fix(backend): removed the second file reading
* fix(backend): fixed wrong queue topic name
* fix(backend): fixed mobile trigger topic name
* feat(backend): added tar.gz extractor and iOSSessionEnd handler
* feat(backend): debug logs on message uploading
* fix(backend): added raw-images topic consumption
* feat(backend): now sink send iosSessionEnd to video-storage
* feat(backend): added dir creator for new sessions
* feat(backend): added correct command to execute
* feat(backend): added overwrite option
* feat(backend): added s3 uploader for video session replays
* feat(backend): new consumer group for mobile sessions
* feat(backend): debug logs for uploader
* feat(backend): removed unused log
* feat(backend): fixed s3 key for video replays
* feat(backend): removed debug code
* feat(backend): fixed video-storage message filter
* fix(backend): added mobileSessionEnd to SessionEnd converter
* feat(backend): added first version if db support for mobile events
* fix(backend): added swipe events to mob file
* feat(backend): added swipe event to pg
* feat(backend): split logic into 2 services: image-storage and video-storage
* feat(backend): added helm chart for image-storage service
* fix(backend): fixed table name for mobile taps
* feat(backend): added metadata handler for mobile message parser + fix message filters
* feat(backend): added iosRawTopic to DB message consumer
* fix(backend): removed value from mobile inputs
* feat(backend): removed debug log from iterator
* feat(backend): added new apple devices to iOS device parser
* fix(backend): added real projectID instead of 0
* feat(backend): extended a list of simulators for device detector
* feat(backend): updated networkCall mobile message
* fix(backend): added new way to define is network call successed or not
* feat(backend): added timezone support for mobile start request
* feat(backend): added 2 mobile events Input and Click to mob file
* feat(backend): refactored image storage
* feat(backend): video storage with 2 workers
* feat(backend): added project's platform support
* feat(backend): added memory size field for mobile start request
* feat(backend): changed video preset for ultrafast
* feat(backend): added debug log to http /late handler
* feat(backend): added debug log to db service for iosCrash messages
* feat(backend): added tapRage event handler to heuristics
* feat(backend): changed table and field names for ios crashes
* feat(backend): added payload for tapRage events
* feat(backend): added TapRage events insert to DB
* feat(backend): added fps value to /mobile/start response
* feat(backend): added image quality parameter to /mobile/start response
* feat(backend): added ScreenLeave handler
* feat(backend): removed screenEnter and screenLeave events, added new viewComponent event
---------
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): moved sql requests related to sessions table to one place
* feat(backend): refactoring in db.Saver handler
* feat(backend): hude refactoring in db/postgres module
* fix(backend): workable feature flags
* fix(backend): workable integrations
* fix(backend): workable sessions and projects modules
* fix(backend): added missed projects module to sessions
* feat(backend): renaming
* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels
* feat(backend): moved project struct to projects module
* feat(backend): added projects model
* feat(backend): implemented new in memory cache for sessions and projects
* feat(backend): implemented new cache in projects
* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()
* feat(backend): added cache update operations
* fix(backend): fixed import cycle
* fix(backend): fixed panic in db message handler
* fix(backend): fixed panic in projects module
* fix(backend): fixed panic in sessions.GetDuration
* feat(backend): added direct call to get session duration if session is already in cache
* feat(backend): used pg pool everywhere except db service
* fix(backend): added missing part after rebase
* fix(backend): removed old sessions file
* feat(backend): added refactored redis client with produce/consume options
* feat(backend): added cache layer for projects
* fix(backend): added missing redis config
* fix(backend): added missing method for producer
* feat(backend): cache integration for sessions
* feat(backend): temporary method to get session directly from db
* feat(backend): adapt EE version of message handler
* fix(backend): fixed issue in fts realisation
* fix(backend): added redis cache to sessions module
* fix(backend): set 0 duration or hesitation time for inputs without focus event
* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation
* feat(backend): debug log
* feat(backend): more debug log
* feat(backend): removed debug log
* fix(backend): fixed an issue of tracking input events with empty label
* fix(backend): disabled debug log in projects cache
* fix(backend): renamed session updater
* fix(backend): fixed closed pool issue in DB service
* fix(backend): fixed dead lock in db Stop() method
* fix(backend): fixed panic in heuristics service
* feat(backend): enabled redis cache in projects
* feat(backend): clear cache on each update operation
* feat(backend): fully integrated cache layer with auto switch
* feat(backend): small refactoring in session updates
* fix(backend): fixed wrong events counter issue
* feat(backend): enabled full cache support in ender and http services
* fix(backend/ee): added missed import
* feat(backend): added second cache layer for db to speed up the service
* feat(backend): disable redis cache
* feat(backend): moved redis cache to ee
* feat(backend): implemented azure blob storage support
* feat(backend): added azure implementation to backend services
* feat(backend): added azure blob storage support to chalice service
* fix(backend): removed prev version of s3
* feat(backend): moved azure support to ee subfolder
* feat(backend): prepared ee code for new utils.objects package
* feat(backend): added missed modules to go.mod
* feat(backend): added missed modules to go.sum
* feat(backend): go mod tidy
* feat(backend): temporary made s3 vars are not required
* feat(backend): added azure lib to ee chalice
* feat(api): changed azure env var name
* feat(api): added new object store and extra methods to chalice ee
* fix(api): added azure blob lib to alerts
* fix(api): fixed incorrect call in sessions_devtool
* fix(crons): added azure blob storage library to requirements list
* chore(build): Error message for not providing flag.
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend): removed ios headers and object store for ios messages
* feat(backend): object config refactoring
* chore(helm): Update BUCKET_NAME
* fix(backend): removed object storage usage in http
* feat(backend): added debug logs to azure upload method
* fix(backend): fixed empty folder issue
* fix(backend): removed extra debug log
* chore(helm): Adding global variables for crons
* chore(helm): Remove clickhouse resource limit
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
* fix(backend): removed assets debug log
* feat(api): use ABC class instead of empty interface
* feat(api): renamed helpers to generators
* feat(api): changed prep/clean dev scripts
* feat(api): changed name obj_store -> StorageClient
* feat(api): some changes after code review
* fix(api): removed unnecesery packages in oss api
* feat(backend): moved azure implementation to ee folder
---------
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
* feat(backend/http): added city and state support for geo extractor
* feat(backend): use new GeoLite db in Dockerfile
* feat(backend/http): added unit test
