Commit graph

62 commits

Author SHA1 Message Date
Alexander
b92177a151
Assist-only feature (#2214)
* feat(http): small prep for task implementation

* feat(http): added assist-only check to ee

* feat(backend): using a new license check url

* feat(backend): packages upgrade
2024-05-28 09:41:16 +02:00
Alexander
2bf9298836
Mobile renaming (protocol + backend) (#2078)
* feat(backend): renamed ios to mobile

* feat(backend): added missing changes to iterator

* feat(backend): added missing changes to ender

* feat(backend): as usual auto deleted import in ch connector

* feat(backend): updated ee/conditions func calls
2024-04-12 09:30:24 +02:00
Alexander
486a7115d7 feat(backend): added session's screen width/height to PG 2024-04-08 13:49:29 +02:00
Alexander
daddedebcc feat(backend): fixed an issue with no mobile replays for android tracker 2024-04-05 17:00:45 +02:00
Alexander
54156fc81d feat(backend): moved batch info to query param 2024-04-05 16:46:02 +02:00
Alexander
37276006fb feat(backend): added missing mobile log 2024-04-05 14:54:48 +02:00
Alexander
75ccacf576 feat(backend): added missing sessionID to log in start request + batch info 2024-04-05 09:42:55 +02:00
Alexander
5356d72417 feat(backend): tmp print screen size in logs 2024-04-02 17:43:13 +02:00
Alexander
eb3952666e feat(backend): moved mobile screen size to browser version 2024-04-02 17:35:37 +02:00
Alexander
8c84fd6550 feat(backend): added new fields to mobile start request 2024-03-25 09:14:00 +01:00
Alexander
fa24030f0b feat(backend): added more details to http logs 2024-03-14 15:43:57 +01:00
Alexander
a453256f23 feat(backend): moved log to prevent error code loss 2024-03-14 14:30:46 +01:00
Alexander
45c956c489
Json logs format (#1952)
* feat(backend): try a new approach for logs formatting (http)

* feat(backend): added logger module

* feat(backend): added project/session info to /i endpoint

* feat(backend): found a solution for correct caller information

* feat(backend): finished logs for http handlers

* feat(backend): finished logs for mobile http handlers

* feat(backend): finished ender

* feat(backend): finished assets

* feat(backend): finished heuristics

* feat(backend): finished image-storage

* feat(backend): finished sink

* feat(backend): finished storage

* feat(backend): formatted logs in all services

* feat(backend): finished foss part

* feat(backend): added missed foss part

* feat(backend): fixed panic in memory manager and sink service

* feat(backend): connectors
2024-03-14 12:51:14 +01:00
Alexander
68d4090e15 feat(backend): added projectID to mobile session start responce 2024-03-01 16:42:49 +01:00
Alexander
a73bf81902 feat(backend): added support for conditional recording feature (mobile sessions) 2024-02-26 15:13:51 +01:00
Eng Zer Jun
76f0f3ff3d
refactor(backend): move from io/ioutil to io and os packages (#1784)
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2024-01-29 16:37:59 +01:00
Alexander
df2ee71bbb feat(backend): fixed issue in start request 2024-01-26 14:25:44 +01:00
Alexander
354f071fb0
Conditions (ee version) (#1858)
* feat(backend): moved conditions to ee folder

* feat(backend): fixed typo
2024-01-26 14:03:29 +01:00
Alexander
255f879476
Mobile replay (#1853)
* feat(backend): added new trigger topic for mobile video replay maker

* feat(backend): mobile video maker with mix list support

* feat(backend): fixed panic

* feat(backend): removed commented code
2024-01-25 14:30:40 +01:00
Alexander
9e3548e9ce
Conditional recording (#1849)
* feat(backend): added mocked endpoint for conditional recordings

* feat(backend): disabled project check

* feat(backend): adapt getConditions endpoint for API implementation

* feat(backend): added condition option to start request

* feat(backend): added missing file

* feat(backend): debug log
2024-01-22 14:52:02 +01:00
Alexander
622b189d39
Tag and watch backend support (#1838)
* feat(ui/tracker): start tag n watch

* fix(tracker): test coverage, fix some watcher api

* fix(tracker): add intersectionobserver, adjust tests

* feat(tracker): relay + apollo plugins

* feat(ui): tags search

* feat(ui): tags name edit

* feat(ui): tags search icon

* feat(ui): icons for tabs in player

* feat(ui): save and find button

* feat(tracker): save tags in session storage (just in case)

* feat(ui): improve loading

* feat(ui): fix icon names gen

* feat(ui): fix typo

* feat(backend): added TagAndWatch backend support

* feat(backend): use msgIndex as a seqIndex

* feat(backend): try to use truncSqlIdx for seq_index

* feat(backend): added tags cache and tags filter to DB service

* feat(backend): removed debug logs

---------

Co-authored-by: nick-delirium <nikita@openreplay.com>
2024-01-19 15:21:10 +01:00
Alexander
7284818af9 feat(backend): added isOffline parameter to startRequest 2023-12-20 17:29:45 +01:00
Alexander
1c6290a312
Small http fixes (#1767)
* fix(backend): removed issue with beta versions of tracker

* feat(backend): added filter for getUxTest request
2023-12-11 15:19:25 +01:00
Alexander
7e8de3cb1d feat(backend): added new arg to start request for condrecs 2023-12-08 16:56:35 +01:00
Kraiem Taha Yassine
7368072db2
Revert "chore(api): asyncify GET /projects, and propagate (#1701)" (#1747)
This reverts commit 23015f5567.

Co-authored-by: ⵄⵎⵉⵔⵓⵛ <amirouche.boubekki@gmail.com>
2023-12-06 16:15:50 +01:00
ⵄⵎⵉⵔⵓⵛ
23015f5567
chore(api): asyncify GET /projects, and propagate (#1701)
* [Backend] Canvas support (#1705)

* feat(http): added new parameters to start response and new endpoint for canvas screenshorts

* fix(http): added new topic to dockerfile

* feat(http): try different multipart parser

* feat(image-storage): reused the same workflow for canvas topic handler

* feat(video-storage): new canvas parser and ffmpeg script

* feat(video-storage): use correct replay name for canvas

* feat(backend): added new message (CanvasNode)

* feat(backend): add canvas record events to db

* Async chalice.core.tenants:tenants_exists, and propagate

* rework after review

* chore(api): asyncify `/projects` and propagate.

Impact on the following routes:

  /projects
  /v1/projects
  /passowrd/reset
  /metadata/session_search

* fix(api): there is no cnx.mogrify method.

In psycopg v3, the mogrify method is only available on cursor objects.

In other words, just use cnx.execute(query, kwargs), except when
opening an "explicit cursor session".

ref: https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.ClientCursor.mogrify

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>
2023-12-06 16:09:55 +01:00
Alexander
3de478650e feat(backend): added projectID check for getUXTest endpoint 2023-12-01 14:30:10 +01:00
Alexander
5a66a6b5c3 feat(backend): added env variables for canvas configuration 2023-11-29 13:29:21 +01:00
Alexander
d49a377110 feat(backend): returned ua to sessions table because of sql constraints 2023-11-28 12:31:20 +01:00
ⵄⵎⵉⵔⵓⵛ
6ae3ee1927
Canvas support (#1711)
* feat(http): added new parameters to start response and new endpoint for canvas screenshorts

* fix(http): added new topic to dockerfile

* feat(http): try different multipart parser

* feat(image-storage): reused the same workflow for canvas topic handler

* feat(video-storage): new canvas parser and ffmpeg script

* feat(video-storage): use correct replay name for canvas

* feat(backend): added new message (CanvasNode)

* feat(backend): add canvas record events to db

* feat(backend): implemented missing new method for azure

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>
2023-11-27 16:22:03 +01:00
ⵄⵎⵉⵔⵓⵛ
f321fccc11
Ux testing backend (#1709)
* feat(backend): added ux-testing support

* feat(backend): added ux-testing module

* feat(http): added bucket name for http service

* feat(backend): fixed small typos in http router

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>
2023-11-27 15:58:36 +01:00
Alexander
d573d5ad97 feat(backend): removed commented code from mobile imageHandler 2023-11-09 16:00:42 +01:00
Alexander
d22e4e138e
Backend for mobile tracker support (#1456)
* feat(backend): handlers for mobile messages

* feat(backend): new service template

* feat(backend): save mobile session start and send batches to kafka

* feat(backend): added mobile support to sink, ender and storage

* helm(videostorage): added helm chart for a new service videostorage

* fix(backend): added pointer to streing for userBrowser (because it's null for mobile sessions)

* feat(backend): added MsgIOSBatchMeta handler to message iterator's logic

* feat(backend): added ios ts parser to ender

* feat(backend): enabled producing batch of messages to queue

* feat(backend): removed iosstart from mob files

* feat(backend): added new ios message types

* feat(backend): added iosStart and iosEnd

* fix(backend): fixed log issue

* feat(backend): send tar.gz archives to special queue topic

* feat(backend): read raw archives from kafka

* fix(backend): added missing file

* fix(backend): removed the second file reading

* fix(backend): fixed wrong queue topic name

* fix(backend): fixed mobile trigger topic name

* feat(backend): added tar.gz extractor and iOSSessionEnd handler

* feat(backend): debug logs on message uploading

* fix(backend): added raw-images topic consumption

* feat(backend): now sink send iosSessionEnd to video-storage

* feat(backend): added dir creator for new sessions

* feat(backend): added correct command to execute

* feat(backend): added overwrite option

* feat(backend): added s3 uploader for video session replays

* feat(backend): new consumer group for mobile sessions

* feat(backend): debug logs for uploader

* feat(backend): removed unused log

* feat(backend): fixed s3 key for video replays

* feat(backend): removed debug code

* feat(backend): fixed video-storage message filter

* fix(backend): added mobileSessionEnd to SessionEnd converter

* feat(backend): added first version if db support for mobile events

* fix(backend): added swipe events to mob file

* feat(backend): added swipe event to pg

* feat(backend): split logic into 2 services: image-storage and video-storage

* feat(backend): added helm chart for image-storage service

* fix(backend): fixed table name for mobile taps

* feat(backend): added metadata handler for mobile message parser + fix message filters

* feat(backend): added iosRawTopic to DB message consumer

* fix(backend): removed value from mobile inputs

* feat(backend): removed debug log from iterator

* feat(backend): added new apple devices to iOS device parser

* fix(backend): added real projectID instead of 0

* feat(backend): extended a list of simulators for device detector

* feat(backend): updated networkCall mobile message

* fix(backend): added new way to define is network call successed or not

* feat(backend): added timezone support for mobile start request

* feat(backend): added 2 mobile events Input and Click to mob file

* feat(backend): refactored image storage

* feat(backend): video storage with 2 workers

* feat(backend): added project's platform support

* feat(backend): added memory size field for mobile start request

* feat(backend): changed video preset for ultrafast

* feat(backend): added debug log to http /late handler

* feat(backend): added debug log to db service for iosCrash messages

* feat(backend): added tapRage event handler to heuristics

* feat(backend): changed table and field names for ios crashes

* feat(backend): added payload for tapRage events

* feat(backend): added TapRage events insert to DB

* feat(backend): added fps value to /mobile/start response

* feat(backend): added image quality parameter to /mobile/start response

* feat(backend): added ScreenLeave handler

* feat(backend): removed screenEnter and screenLeave events, added new viewComponent event

---------

Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-09 15:02:20 +02:00
Alexander Zavorotynskiy
fe2e935012 feat(backend): added new column timezone to PG and CH session tables 2023-08-21 10:27:31 +02:00
Alexander
4b8f3bee25
Sessions refactoring (#1371)
* feat(backend): moved sql requests related to sessions table to one place

* feat(backend): refactoring in db.Saver handler

* feat(backend): hude refactoring in db/postgres module

* fix(backend): workable feature flags

* fix(backend): workable integrations

* fix(backend): workable sessions and projects modules

* fix(backend): added missed projects module to sessions

* feat(backend): renaming

* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels

* feat(backend): moved project struct to projects module

* feat(backend): added projects model

* feat(backend): implemented new in memory cache for sessions and projects

* feat(backend): implemented new cache in projects

* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()

* feat(backend): added cache update operations

* fix(backend): fixed import cycle

* fix(backend): fixed panic in db message handler

* fix(backend): fixed panic in projects module

* fix(backend): fixed panic in sessions.GetDuration

* feat(backend): added direct call to get session duration if session is already in cache

* feat(backend): used pg pool everywhere except db service

* fix(backend): added missing part after rebase

* fix(backend): removed old sessions file

* feat(backend): added refactored redis client with produce/consume options

* feat(backend): added cache layer for projects

* fix(backend): added missing redis config

* fix(backend): added missing method for producer

* feat(backend): cache integration for sessions

* feat(backend): temporary method to get session directly from db

* feat(backend): adapt EE version of message handler

* fix(backend): fixed issue in fts realisation

* fix(backend): added redis cache to sessions module

* fix(backend): set 0 duration or hesitation time for inputs without focus event

* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation

* feat(backend): debug log

* feat(backend): more debug log

* feat(backend): removed debug log

* fix(backend): fixed an issue of tracking input events with empty label

* fix(backend): disabled debug log in projects cache

* fix(backend): renamed session updater

* fix(backend): fixed closed pool issue in DB service

* fix(backend): fixed dead lock in db Stop() method

* fix(backend): fixed panic in heuristics service

* feat(backend): enabled redis cache in projects

* feat(backend): clear cache on each update operation

* feat(backend): fully integrated cache layer with auto switch

* feat(backend): small refactoring in session updates

* fix(backend): fixed wrong events counter issue

* feat(backend): enabled full cache support in ender and http services

* fix(backend/ee): added missed import

* feat(backend): added second cache layer for db to speed up the service

* feat(backend): disable redis cache

* feat(backend): moved redis cache to ee
2023-07-06 10:55:43 +02:00
Alexander
db7d624b3b
[Backend] feature flags (#1354)
* feat(backend): added ff mock

* feat(backend): added feature flag pg method

* fix(backend): fixed ff request field

* feat(backend): added multivariant support

* feat(backend): added logic handler for feature flag filters

* fix(backend): correct sql request for multivariants flags

* feat(backend): added new fields to sessionStart response

* feat(backend): removed unused fields from getFeatureFlags request

* fix(backend): removed comments

* feat(backend): added debug logs

* feat(backend): added single type case for arrayToNum parser

* feat(backend): added unit tests for feature flags
2023-06-21 13:41:06 +02:00
Alexander
f89d84232a
[DB] Added support for city/state autocomplete (#1317)
* feat(backend): added support for city/state autocomplete

* fix(backend): removed unused http handlers
2023-06-08 15:59:53 +02:00
Alexander
111d6a1b54
Azure support (#1270)
* feat(backend): implemented azure blob storage support

* feat(backend): added azure implementation to backend services

* feat(backend): added azure blob storage support to chalice service

* fix(backend): removed prev version of s3

* feat(backend): moved azure support to ee subfolder

* feat(backend): prepared ee code for new utils.objects package

* feat(backend): added missed modules to go.mod

* feat(backend): added missed modules to go.sum

* feat(backend): go mod tidy

* feat(backend): temporary made s3 vars are not required

* feat(backend): added azure lib to ee chalice

* feat(api): changed azure env var name

* feat(api): added new object store and extra methods to chalice ee

* fix(api): added azure blob lib to alerts

* fix(api): fixed incorrect call in sessions_devtool

* fix(crons): added azure blob storage library to requirements list

* chore(build): Error message for not providing flag.

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* feat(backend): removed ios headers and object store for ios messages

* feat(backend): object config refactoring

* chore(helm): Update BUCKET_NAME

* fix(backend): removed object storage usage in http

* feat(backend): added debug logs to azure upload method

* fix(backend): fixed empty folder issue

* fix(backend): removed extra debug log

* chore(helm): Adding global variables for crons

* chore(helm): Remove clickhouse resource limit

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* fix(backend): removed assets debug log

* feat(api): use ABC class instead of empty interface

* feat(api): renamed helpers to generators

* feat(api): changed prep/clean dev scripts

* feat(api): changed name obj_store -> StorageClient

* feat(api): some changes after code review

* fix(api): removed unnecesery packages in oss api

* feat(backend): moved azure implementation to ee folder

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-08 13:31:54 +02:00
Alexander
691c84d5f4
[HTTP] added city and state support for geo extractor (#1312)
* feat(backend/http): added city and state support for geo extractor

* feat(backend): use new GeoLite db in Dockerfile

* feat(backend/http): added unit test
2023-06-08 09:07:08 +02:00
Alexander
fe0840ee84
feat(backend): added support for new env variable to enable/disable Access-Control-* headers (#1308) 2023-06-06 16:54:56 +02:00
Alexander Zavorotynskiy
72ccac1aed fix(backend): added COMPRESSION_THRESHOLD env variable to Dockerfile 2023-05-09 17:50:16 +02:00
Alexander Zavorotynskiy
2c6f9a15ab fix(backend): added Content-Encoding to CORS for http service 2023-05-09 17:35:17 +02:00
Alexander
742c038ecd
New env variable CLOUD (aws by default) (#1232)
* feat(backend): added new env variable CLOUD (aws by default)

* chore(backend): Adding env variable for CLOUD

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-04 16:48:48 +02:00
Alexander Zavorotynskiy
8aa14a68c9 fix(backend): disabled debug log in http handler 2023-05-02 16:59:57 +02:00
Alexander
19fc925462
Improved session compression (#1200)
* feat(backend): implemented new compression
2023-04-25 10:25:25 +02:00
Alexander Zavorotynskiy
e22845049d feat(backend): removed debug logs in http methods 2023-04-11 16:19:35 +02:00
Alexander Zavorotynskiy
f465d16110 feat(backend): removed debug logs in http methods 2023-04-11 16:19:35 +02:00
Alexander Zavorotynskiy
2a9c64ef16 feat(backend): debug log for body content 2023-04-11 16:19:35 +02:00
Alexander Zavorotynskiy
05e47e978b feat(backend): added compression threshold to start request 2023-04-11 16:19:35 +02:00
Alexander Zavorotynskiy
33fbdb149f feat(backend): more debug logs 2023-04-11 16:19:35 +02:00