Commit graph

170 commits

Author SHA1 Message Date
Alexander
b2cb874a2a
Save the last batch (#2748)
* feat(backend): fix to save the latest message tracker just after the token has been expired

* feat(http): return 401 even after successfully saved batch for JustExpired case
2024-11-15 11:49:37 +01:00
Alexander
bbf1dbcc23 feat(http): ff and uxt are disabled by default 2024-09-03 11:41:24 +02:00
Alexander
345f316b27
Spots (#2305)
* feat(spot): first version to test http endpoints

* fix(helm): changed nginx path prefix

* fix(spots): added missing BUCKET_NAME env var

* fix(spots): added services init check

* feat(spots): removed geo module

* feat(spots): removed uaparser

* feat(spots): added more detailed authorization error log

* feat(spots): changed the authorization middleware

* feat(spots): extended http body size limit to 128kb

* feat(spots): added s3 error log

* feat(spots): added new handler for uploaded event

* feat(backend): small api changes in spot service

* feat(backend): rewrote request parameters grabber for getSpot handler

* feat(backend): added tenantID to auth struct

* feat(backend): added pre-signed download urls for preview, mob et video files

* feat(backend): added user's email to spots table, and getSpot responses

* feat(backend): returning spotID as a string

* feat(spot): added transcoder pipeline

* fix(spot): return spotID as a string

* feat(spot): added volume mount to spot service

* feat(spot): fixed volume mounting

* feat(spot): helm fix

* feat(spot): helm another fix

* fix(spot): correct video.webm path

* fix(spot): correct pre-signed url for download original video

* feat(spot): added PATCH and DELETE methods to CORS

* feat(spot): use string format for spotIDs in delete method

* feat(spot): added public key implemented

* fix(spot): correct public-key parser

* fix(spot): fixed query params issue + user's tenantID

* fix(spot): use 1 as a default tenant

* feat(spot): added correct total spots calculation

* fix(spot): fixed offset calculation

* feat(spot): added extra check in auth method

* fix(spot): removed / from video file name

* fix(spot): devided codec flag into 2 parts

* feat(spot): use fixed tenantID = 1 for oss users

* feat(spot): return 404 for public key not found issue

* feat(spots): added spots folder to minio path rule

* feat(spot): added spot video streaming support

* fix(spot): fixed an sql request for spot streams

* feat(spot): return playlist file in getSpot responce

* feat(spot): try to use aac audio codec

* feat(spot): added permissions support (oss/ee)

* feat(spot): added authorizer method

* feat(spot): added license check

* feat(spot): added spot preview for get response

* fix(spot): fixed a problem with permissions

* feat(spot): added crop feature

* feat(spot): upload cropped video back to s3

* feat(spot): manage expired modified playlist file

* feat(backend): hack with video formats

* feat(backend): removed space

* feat(spot): req tracing

* feat(spot): manual method's name mapping

* feat(spot): added a second method to public key auth support

* feat(spot): metrics

* feat(spot): added rate limiter per user

* feat(spot): added ping endpoint for spot jwt token check

* feat(spot): getStatus endpoint

* feat(spot): added missing import

* feat(spot): transcoding issue fix

* feat(spot): temp remove tasks

* feat(spot): better error log message

* feat(spot): set default jwt_secret value

* feat(spot): debug auth

* feat(spot): 2 diff jwt tokens support

* feat(spot): pg tasks with process status

* feat(spot): more logs

* feat(spot): improved defer for GetTask method

* feat(spot): keep only failed tasks

* feat(spot): removing temp dir with spot files

* feat(spot): added several workers for transcoding module

* feat(spot): fixed spot path for temp video files

* feat(spot): use custom statusWriter to track response code in middleware

* feat(spot): added body and parameter parser for auditrail feature

* feat(spot): fixed IsAuth method signature

* feat(spot): fixed ee service builder

* feat(spot): added import

* feat(spot): fix data type for payload and parameters jsonb fields

* feat(spot): typo fix

* feat(spot): moved out consts

* feat(spot): new table's name

* feat(spot): added missing imports in go.mod

* feat(spot): added a check for the number of comments (20 by default)
2024-08-29 16:08:33 +02:00
Alexander
1637e5b5ec feat(backend): changed mobile autocomplete naming 2024-07-25 10:19:36 +02:00
Alexander
b3d04d76d4
Heatmaps fix (float coordinates) (#2403)
* feat(spot): use float click coordinates instead of ints in PG

* feat(db): added support for float clicks in CH

* feat(db): fix float instead of uint8

* feat(mobile): new naming for mobile autocomplete types
2024-07-18 12:32:44 +02:00
Alexander
75eb6924c1 feat(connector): added s3 batches support for CH connector 2024-07-05 16:16:02 +02:00
Alexander
16fe6674f3 feat(assets): use UA header every get call 2024-07-02 12:20:07 +02:00
Alexander
b92177a151
Assist-only feature (#2214)
* feat(http): small prep for task implementation

* feat(http): added assist-only check to ee

* feat(backend): using a new license check url

* feat(backend): packages upgrade
2024-05-28 09:41:16 +02:00
Alexander
993b6b58ab
feat(backend): added utm data to PG (#2152)
* feat(backend): added utm data to PG

* feat(backend): added utm data to CH

* feat(db): removed nullableString() calls
2024-05-24 08:33:28 +02:00
Alexander
4c96cbcef3 feat(connector): added new events + projectID filter 2024-05-23 12:09:59 +02:00
Alexander
de9ffeb6a9 feat(connector): added s3 connector + small improvements 2024-05-22 15:23:45 +02:00
Alexander
2bf9298836
Mobile renaming (protocol + backend) (#2078)
* feat(backend): renamed ios to mobile

* feat(backend): added missing changes to iterator

* feat(backend): added missing changes to ender

* feat(backend): as usual auto deleted import in ch connector

* feat(backend): updated ee/conditions func calls
2024-04-12 09:30:24 +02:00
Alexander
bdb3f37c56 feat(backend): added es connector 2024-04-09 14:58:47 +02:00
Alexander
486a7115d7 feat(backend): added session's screen width/height to PG 2024-04-08 13:49:29 +02:00
Alexander
daddedebcc feat(backend): fixed an issue with no mobile replays for android tracker 2024-04-05 17:00:45 +02:00
Alexander
54156fc81d feat(backend): moved batch info to query param 2024-04-05 16:46:02 +02:00
Alexander
37276006fb feat(backend): added missing mobile log 2024-04-05 14:54:48 +02:00
Alexander
75ccacf576 feat(backend): added missing sessionID to log in start request + batch info 2024-04-05 09:42:55 +02:00
Alexander
f7744a0c2c
Mob split (#2015)
* feat(backend): try to split mob files by ts

* feat(backend): removed debug code and used worker pool
2024-04-03 17:10:44 +02:00
Alexander
5356d72417 feat(backend): tmp print screen size in logs 2024-04-02 17:43:13 +02:00
Alexander
eb3952666e feat(backend): moved mobile screen size to browser version 2024-04-02 17:35:37 +02:00
Alexander
8c84fd6550 feat(backend): added new fields to mobile start request 2024-03-25 09:14:00 +01:00
Alexander
fa24030f0b feat(backend): added more details to http logs 2024-03-14 15:43:57 +01:00
Alexander
a453256f23 feat(backend): moved log to prevent error code loss 2024-03-14 14:30:46 +01:00
Alexander
45c956c489
Json logs format (#1952)
* feat(backend): try a new approach for logs formatting (http)

* feat(backend): added logger module

* feat(backend): added project/session info to /i endpoint

* feat(backend): found a solution for correct caller information

* feat(backend): finished logs for http handlers

* feat(backend): finished logs for mobile http handlers

* feat(backend): finished ender

* feat(backend): finished assets

* feat(backend): finished heuristics

* feat(backend): finished image-storage

* feat(backend): finished sink

* feat(backend): finished storage

* feat(backend): formatted logs in all services

* feat(backend): finished foss part

* feat(backend): added missed foss part

* feat(backend): fixed panic in memory manager and sink service

* feat(backend): connectors
2024-03-14 12:51:14 +01:00
Alexander
89752b2acc
Canvas archives (#1938)
* feat(api): added second pre-signed urls for canvases

* feat(api): fixed old key format

* feat(backend): draft version for new approach for canvases

* feat(deploy): fixed some issues

* feat(backend): debug logs for bash command

* feat(backend): removed wrong parameter from bash pipeline

* feat(backend): removed canvas-maker service + small refactoring
2024-03-11 17:51:49 +01:00
Alexander
68d4090e15 feat(backend): added projectID to mobile session start responce 2024-03-01 16:42:49 +01:00
Alexander
7c58ece936 feat(backend): added separate config for canvas-handler 2024-02-28 13:21:19 +01:00
Alexander
21c4ebd6e3 feat(backend): upgraded user-agent for assist service 2024-02-27 13:54:45 +01:00
Alexander
36fb1a07f7
Worker pool for image processing services (#1912)
* feat(backend): added worker pool for screenshot handler

* feat(backend): added worker pool to canvas handler

* feat(backend): added worker pool to canvas maker

* feat(backend): added missing method to canvas-maker service
2024-02-27 13:48:27 +01:00
Alexander
a73bf81902 feat(backend): added support for conditional recording feature (mobile sessions) 2024-02-26 15:13:51 +01:00
Alexander
a073ce498d
No ffmpeg solution (#1905)
* feat(video-storage): added zstd library to the machine

* feat(video-storage): added new method to pack screenshots into compressed tar arch

* feat(video-storage): try to split command into 2

* feat(video-storage): try a new approad to avoid no file error

* feat(api): added support to a new pre-signed url for screenshots archive

* feat(video-storage): fixed an issue in extension check

* feat(video-storage): correct file name

* feat(backend): removed video-storage and splitted logic

* feat(backend): removed video-storage from helm charts

* feat(backend): split canvas and screenshot handlers

* feat(canvas): clean up canvas-handler

* feat(api): changed mobile replay url (screenshots instead of video)

* feat(backend): removed msg.SessID() call

* feat(backend): clean up code in imagestorage main
2024-02-26 14:16:43 +01:00
Alexander
88ea925263 feat(backend): added new flag USE_S3_TAGS to disable s3 tags 2024-01-29 16:50:08 +01:00
Eng Zer Jun
76f0f3ff3d
refactor(backend): move from io/ioutil to io and os packages (#1784)
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2024-01-29 16:37:59 +01:00
Alexander
df2ee71bbb feat(backend): fixed issue in start request 2024-01-26 14:25:44 +01:00
Alexander
354f071fb0
Conditions (ee version) (#1858)
* feat(backend): moved conditions to ee folder

* feat(backend): fixed typo
2024-01-26 14:03:29 +01:00
Alexander
db17366aae feat(backend): fixed no mobile sessions issue in CH 2024-01-26 13:46:17 +01:00
Alexander
255f879476
Mobile replay (#1853)
* feat(backend): added new trigger topic for mobile video replay maker

* feat(backend): mobile video maker with mix list support

* feat(backend): fixed panic

* feat(backend): removed commented code
2024-01-25 14:30:40 +01:00
Alexander
acc21fe82d feat(backend): fixed typo in canvas group name 2024-01-23 11:37:47 +01:00
Alexander
78a9146144 feat(backend): added new consumer groups for new canvas pipeline 2024-01-22 17:03:15 +01:00
Alexander
9e3548e9ce
Conditional recording (#1849)
* feat(backend): added mocked endpoint for conditional recordings

* feat(backend): disabled project check

* feat(backend): adapt getConditions endpoint for API implementation

* feat(backend): added condition option to start request

* feat(backend): added missing file

* feat(backend): debug log
2024-01-22 14:52:02 +01:00
Alexander
622b189d39
Tag and watch backend support (#1838)
* feat(ui/tracker): start tag n watch

* fix(tracker): test coverage, fix some watcher api

* fix(tracker): add intersectionobserver, adjust tests

* feat(tracker): relay + apollo plugins

* feat(ui): tags search

* feat(ui): tags name edit

* feat(ui): tags search icon

* feat(ui): icons for tabs in player

* feat(ui): save and find button

* feat(tracker): save tags in session storage (just in case)

* feat(ui): improve loading

* feat(ui): fix icon names gen

* feat(ui): fix typo

* feat(backend): added TagAndWatch backend support

* feat(backend): use msgIndex as a seqIndex

* feat(backend): try to use truncSqlIdx for seq_index

* feat(backend): added tags cache and tags filter to DB service

* feat(backend): removed debug logs

---------

Co-authored-by: nick-delirium <nikita@openreplay.com>
2024-01-19 15:21:10 +01:00
Alexander
b4b6ceb025
Canvas refactoring (#1836)
* feat(backend): added new topic for canvasToVideo communication, moved some logic

* feat(backend): enabled canvas recording

* feat(backend): fixed canvas service main logic

* feat(backend): fixed sessionEnd detector

* feat(backend): send canvas mix lists instead of just a sessID

* feat(backend): enabled canvas recording

* feat(backend): removed old logs from video-storage

* feat(backend): default low setting for canvas recording
2024-01-17 14:39:12 +01:00
Alexander
7284818af9 feat(backend): added isOffline parameter to startRequest 2023-12-20 17:29:45 +01:00
Alexander
f8a8dfa459
[Integrations] small refactoring (#1752)
* feat(backend): small refactoring in integrations

* feat(backend): ignore context timeout error
2023-12-14 11:28:05 +01:00
Alexander
1c6290a312
Small http fixes (#1767)
* fix(backend): removed issue with beta versions of tracker

* feat(backend): added filter for getUxTest request
2023-12-11 15:19:25 +01:00
Alexander
7e8de3cb1d feat(backend): added new arg to start request for condrecs 2023-12-08 16:56:35 +01:00
Kraiem Taha Yassine
7368072db2
Revert "chore(api): asyncify GET /projects, and propagate (#1701)" (#1747)
This reverts commit 23015f5567.

Co-authored-by: ⵄⵎⵉⵔⵓⵛ <amirouche.boubekki@gmail.com>
2023-12-06 16:15:50 +01:00
ⵄⵎⵉⵔⵓⵛ
23015f5567
chore(api): asyncify GET /projects, and propagate (#1701)
* [Backend] Canvas support (#1705)

* feat(http): added new parameters to start response and new endpoint for canvas screenshorts

* fix(http): added new topic to dockerfile

* feat(http): try different multipart parser

* feat(image-storage): reused the same workflow for canvas topic handler

* feat(video-storage): new canvas parser and ffmpeg script

* feat(video-storage): use correct replay name for canvas

* feat(backend): added new message (CanvasNode)

* feat(backend): add canvas record events to db

* Async chalice.core.tenants:tenants_exists, and propagate

* rework after review

* chore(api): asyncify `/projects` and propagate.

Impact on the following routes:

  /projects
  /v1/projects
  /passowrd/reset
  /metadata/session_search

* fix(api): there is no cnx.mogrify method.

In psycopg v3, the mogrify method is only available on cursor objects.

In other words, just use cnx.execute(query, kwargs), except when
opening an "explicit cursor session".

ref: https://www.psycopg.org/psycopg3/docs/api/cursors.html#psycopg.ClientCursor.mogrify

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>
2023-12-06 16:09:55 +01:00
Alexander
a68351d509
[Backend] Zstd support for mob files (#1732)
* feat(backend): added zstd compression support to storage service

* feat(backend): try to ignore content-encoding for zstd compression
2023-12-04 16:00:59 +01:00