Commit graph

63 commits

Author SHA1 Message Date
Alexander
e84bdb5eef
Patch/patch ch clicks with url (#2698)
* added url and url_path to click events

* added app_crash support

* fixed an url in click event
2024-10-28 18:47:31 +01:00
Alexander
d9fe2b5bb8
added url and url_path to click events (#2697)
* added url and url_path to click events

* added app_crash support
2024-10-28 18:36:22 +01:00
Alexander
90d13c69ab
Revert "feat(clickhouse): added host/url to click events (#2680)" (#2693)
This reverts commit a56b94ba92.
2024-10-28 16:38:56 +01:00
Alexander
a56b94ba92
feat(clickhouse): added host/url to click events (#2680) 2024-10-25 14:25:20 +02:00
Alexander
b3d04d76d4
Heatmaps fix (float coordinates) (#2403)
* feat(spot): use float click coordinates instead of ints in PG

* feat(db): added support for float clicks in CH

* feat(db): fix float instead of uint8

* feat(mobile): new naming for mobile autocomplete types
2024-07-18 12:32:44 +02:00
Alexander
c6c46c6486 fix(backend): fixed naming issue in CH connector 2024-07-03 16:48:13 +02:00
Alexander
b9dc87884c fix(backend): fixed an empty url_path for request events in CH 2024-07-03 16:40:44 +02:00
Alexander
939d661cdc fix(backend): changed normalized_x/y from uint32 to uint8 2024-07-03 16:34:39 +02:00
Alexander
2cd210fbaa fix(backend): fixed an issue in url_path builder 2024-06-27 13:00:23 +02:00
Delirium
960da9f037
Tracker 14.x.x changes (#2240)
* feat tracker: add document titles to tabs

* feat: titles for tabs

* feat tracker: send initial title, parse titles better

* feat ui: tab name styles

* feat tracker: update changelogs

* fix tracker: fix tests

* fix tracker: fix failing tests, add some coverage

* fix tracker: fix failing tests, add some coverage

* Heatmaps  (#2264)

* feat ui: start heatmaps ui and tracker update

* fix ui: drop clickmap from session

* fix ui: refactor heatmap painter

* fix ui: store click coords as int percent

* feat(backend): insert normalized x and y to PG

* feat(backend): insert normalized x and y to CH

* feat(connector): added missing import

* feat(backend): fixed different uint type issue

* fix tracker: use max scrollable size for doc

* fix gen files

* fix ui: fix random crash, remove demo data generator

* fix ui: rm some dead code

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>

* fix tracker: add heatmap changelog to tracker CHANGELOG.md

* fix tracker: fix peerjs version to 1.5.4 (was 1.5.1)

* fix document height calculation

* Crossdomain tracking (#2277)

* feat tracker: crossdomain tracking (start commit)

* catch crossdomain messages, add nodeid placeholder

* click scroll

* frame placeholder number -> dynamic

* click rewriter, fix scroll prop

* some docs

* some docs

* fix options merging

* CHANGELOG.md update

* checking that crossdomain will not fire automatically

* simplify func declaration

* update test data

* change clickmap document height calculation to scrollheight (which should be true height)

---------

Co-authored-by: Alexander <zavorotynskiy@pm.me>
2024-06-24 13:49:26 +02:00
Alexander
400b3d2e61 feat(backend): added url_path inserts into CH 2024-05-29 13:25:33 +02:00
Alexander
55b6cd7dcd feat(db): insert full url to pages table 2024-05-24 09:42:27 +02:00
Alexander
17d7f127bb Revert "feat(db): insert url path to CH pages table"
This reverts commit dee1ef1618.
2024-05-24 09:40:04 +02:00
Alexander
dee1ef1618 feat(db): insert url path to CH pages table 2024-05-24 09:27:42 +02:00
Alexander
993b6b58ab
feat(backend): added utm data to PG (#2152)
* feat(backend): added utm data to PG

* feat(backend): added utm data to CH

* feat(db): removed nullableString() calls
2024-05-24 08:33:28 +02:00
Alexander
035d116fb2 feat(backend): insert click's selector into CH 2024-04-18 13:08:33 +02:00
Alexander
2bf9298836
Mobile renaming (protocol + backend) (#2078)
* feat(backend): renamed ios to mobile

* feat(backend): added missing changes to iterator

* feat(backend): added missing changes to ender

* feat(backend): as usual auto deleted import in ch connector

* feat(backend): updated ee/conditions func calls
2024-04-12 09:30:24 +02:00
Alexander
b6d71194d7 feat(backend): autodeleted include 2024-03-19 14:48:05 +01:00
Alexander
6c12af6125 feat(backend): fixed an issue in CH connector 2024-03-19 12:20:46 +01:00
Alexander
4711bf28f0 feat(backend): small improvements in CH client 2024-01-26 13:48:03 +01:00
Alexander
07d8b4e30f fix(backend): added type convertor for TransferredBodySize field in CH connector 2023-11-08 10:57:33 +01:00
nick-delirium
91f5c16e28 fix(backend): msg name 2023-10-27 12:17:11 +02:00
Alexander
7ed2967361 feat(backend): added mobile event handlers for CH db 2023-10-26 17:11:19 +02:00
Delirium
c7e5145282
feat(tracker): add beacon proxy and request body size (#1389)
* feat(tracker): add beacon proxy and body size

* feat(ui): remove unused components

* feat(ui): generate new messages, add body size to resource parser

* feat(ui): fix tooltip text, fix size detection (ts safe)

* feat(ui): cover resource with tests

* feat(ui): enable test coverage for player, utils and mstore

* fix(tracker): adjust test to support new message

* fix(tracker): fix tracker version for back compat

* feat(backend): added new column to network requests

* fix(tracker): fix unit tests

* fix(backend): fix msg gen

* fix(tracker): ci fun

* fix(tracker): changelog

* fix(tracker): fix some test

---------

Co-authored-by: Alexander Zavorotynskiy <zavorotynskiy@pm.me>
2023-10-17 12:25:20 +02:00
Alexander Zavorotynskiy
15b55c837b fix(backend): added missed import for clickhouse connector 2023-10-10 11:51:59 +02:00
Alexander
bb7c086c5b
Performance patch (#1508)
* feat(backend): removed legacy InputEvent builder from heuristics service

* feat(backend): removed InputEvent handler from DB service

* feat(backend): removed InputEvent from ClickHouse connector

* feat(backend): added extra env variable for brotli compression algorithm
2023-10-09 13:54:53 +02:00
Alexander Zavorotynskiy
fe2e935012 feat(backend): added new column timezone to PG and CH session tables 2023-08-21 10:27:31 +02:00
Alexander
4b8f3bee25
Sessions refactoring (#1371)
* feat(backend): moved sql requests related to sessions table to one place

* feat(backend): refactoring in db.Saver handler

* feat(backend): hude refactoring in db/postgres module

* fix(backend): workable feature flags

* fix(backend): workable integrations

* fix(backend): workable sessions and projects modules

* fix(backend): added missed projects module to sessions

* feat(backend): renaming

* feat(backend): moved session struct to sessions module and split methods into interface, cache and storage levels

* feat(backend): moved project struct to projects module

* feat(backend): added projects model

* feat(backend): implemented new in memory cache for sessions and projects

* feat(backend): implemented new cache in projects

* feat(backend): there are 2 methods in cache module now: Get() and GetAndRefresh()

* feat(backend): added cache update operations

* fix(backend): fixed import cycle

* fix(backend): fixed panic in db message handler

* fix(backend): fixed panic in projects module

* fix(backend): fixed panic in sessions.GetDuration

* feat(backend): added direct call to get session duration if session is already in cache

* feat(backend): used pg pool everywhere except db service

* fix(backend): added missing part after rebase

* fix(backend): removed old sessions file

* feat(backend): added refactored redis client with produce/consume options

* feat(backend): added cache layer for projects

* fix(backend): added missing redis config

* fix(backend): added missing method for producer

* feat(backend): cache integration for sessions

* feat(backend): temporary method to get session directly from db

* feat(backend): adapt EE version of message handler

* fix(backend): fixed issue in fts realisation

* fix(backend): added redis cache to sessions module

* fix(backend): set 0 duration or hesitation time for inputs without focus event

* feat(backend): added cache for session updates and failover mechanism for batch.Insert() operation

* feat(backend): debug log

* feat(backend): more debug log

* feat(backend): removed debug log

* fix(backend): fixed an issue of tracking input events with empty label

* fix(backend): disabled debug log in projects cache

* fix(backend): renamed session updater

* fix(backend): fixed closed pool issue in DB service

* fix(backend): fixed dead lock in db Stop() method

* fix(backend): fixed panic in heuristics service

* feat(backend): enabled redis cache in projects

* feat(backend): clear cache on each update operation

* feat(backend): fully integrated cache layer with auto switch

* feat(backend): small refactoring in session updates

* fix(backend): fixed wrong events counter issue

* feat(backend): enabled full cache support in ender and http services

* fix(backend/ee): added missed import

* feat(backend): added second cache layer for db to speed up the service

* feat(backend): disable redis cache

* feat(backend): moved redis cache to ee
2023-07-06 10:55:43 +02:00
Alexander Zavorotynskiy
00ca3d6184 feat(backend): added user's state and city to CH 2023-06-22 14:26:31 +02:00
Alexander Zavorotynskiy
853ac7c277 feat(backend): added CH creds support 2023-03-21 17:18:31 +01:00
Alexander Zavorotynskiy
d6526464c6 fix(backend): fixed field names 2023-03-15 16:26:46 +01:00
Alexander Zavorotynskiy
a7ca8ac54f feat(backend): implemented new events for CH 2023-03-15 16:21:44 +01:00
Alexander
c6aac11cbf
Heuristics refactoring (#987)
* feat(backend): refactored heuristics service

* feat(backend): refactored db service (moved several events to heuristics)
2023-03-09 09:54:12 +01:00
Alexander
fdd28dbc4a
Draft: New metrics module (#982)
* feat(backend): created new metrics module
2023-02-20 16:37:55 +01:00
Alexander
a40ce7c376
[DB] async db inserts for PG and CH (#967)
* feat(backend): added async workers for db service
2023-02-13 13:58:03 +01:00
Rajesh Rajendran
93f27fe459
clickhouse password (#972)
* chore(helm): Adding clickhouse password
* chore(backend): Support clickhouse password
* chore(api): Adding clickhouse password support
* chore(helm): inject clickhouse password
* chore(helm): Updating clickhouse image
* fix(clickhouse): Removing duplicate country code

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-02-08 18:47:32 +01:00
Kraiem Taha Yassine
a68d2f2a58
Merge pull request #905 from openreplay/db_limits
[DB] limits for some columns
2023-01-20 17:47:58 +01:00
Alexander
da7d2038ef
[DB] added url field for click rage issues (#925)
* feat(backend): added url field for click rage issues
2023-01-18 13:38:10 +01:00
Alexander Zavorotynskiy
fa3655079d feat(backend): added CH limits for some columns 2022-12-30 11:19:13 +01:00
Alexander
205fdcba42
fix(backend/clickhouse): fixed batch insert panic because of wrong error source (#900) 2022-12-24 20:53:11 +01:00
Alex Kaminskii
ff80386aff feat(backend,mobs):replace Fetch with NetworkRequest 2022-12-09 15:48:47 +01:00
Alex K
e861bd9641
refactor(backend):remove legacy message wrapping (#854)
* refactor(backend):remove legacy message wrapping & few backend-only messages
2022-12-06 11:54:36 +01:00
Alexander Zavorotynskiy
3b54ddb642 fix(backend): fixed panic from CH lib on incorrect issue type 2022-11-11 11:26:21 +01:00
Alex K
974afa454b
feat(backend): insert error tags+ (#768)
* feat(backend): transform legacy messages

* refactor(backend/postgres):remove seqIndex transformation

* feat(backend/db): parse & insert errors_tags

* chore(backend): remove building folder after build

* fix(backend/db):remove message types from filter that do not go through kafka 

* fix(backend/assets):use JSException instead of ErrorEvent

* refactor(backend/db):ErrorEvent is no longer a message

* feat(backend/db):insert error tags in clickhouse

* refactor(backend):remove unused LongTask code
2022-10-19 10:31:13 +02:00
Alexander Zavorotynskiy
35a58edec2 feat(backend): removed context from issue event 2022-10-18 14:12:19 +02:00
Alexander
bdcc98492d
Issues table for ClickHouse (#764)
* feat(backend): added batch for issues table to clickhouse connector
2022-10-11 10:25:45 +02:00
Alexander Zavorotynskiy
4f8e000de3 fix(backend): removed old method from CH interface 2022-09-19 11:06:39 +02:00
Alexander Zavorotynskiy
7502ad76df feat(backend): removed optimize call 2022-09-16 18:34:06 +02:00
Alexander Zavorotynskiy
348a323a41 feat(backend): added extra checks for urlMethod and resourceType in CH connector 2022-09-02 22:28:57 +02:00
Alexander Zavorotynskiy
c526d611f5 fix(backend): fixed some issues in db service 2022-09-02 18:16:10 +02:00