Commit graph

148 commits

Author SHA1 Message Date
Alexander
56c216d2d5
V1.9.0 sink improvements (#848)
* feat(backend/sink): write big messages directly to file

* feat(backend/sink): manually sync all files on partitions rebalanced event (#847)

* feat(backend): fix vulnerability in /x/net
2022-11-29 11:59:48 +01:00
Jorgen Evens
e0a88666af feat(redis): add suppport for credentials and TLS
By using `ParseURL` it is now possible to
- Supply login credentials for the redis instance
- Use an encrypted TLS connection by setting the scheme to `rediss://`
2022-11-25 19:16:01 +01:00
Alexander
9e319ed27c
[Sink] Improved files sync algo (#831)
* feat(backend): use channel of changed sessions instead of sync.Map

* feat(backend): avoid memory alloc for message body in message iterator

* feat(backend): removed unnecessary locks in file syncer

* feat(backend): sync.Map with prev updates

* feat(backend): improved write algorith (added bufio.Writer)

* feat(backend): session writer refactoring

* feat(backend): removed unnecessary type definition

* feat(backend): added write retrier to avoid data losing

* feat(backend): refactoring

* feat(backend): added session file implementation
2022-11-25 17:25:55 +01:00
Alex Kaminskii
1e17c5d2f9 fix(backend):hash payload to ID only in js-sourced errors 2022-11-17 16:48:15 +01:00
Alex K
822ca53980
Dev aleksk (#795)
*feat(tracker;backend;frontend/player): focus in elem
*feat(tracker;frontend/player): FontFace load
2022-11-04 11:33:29 +01:00
Alexander Zavorotynskiy
df8aed3464 feat(backend): get fresh session from db only on SessionEnd message 2022-11-04 09:40:11 +01:00
Alexander Zavorotynskiy
920c2523d6 fix(backend): fixed bug in sessions cache layer and in sessionEnd handler 2022-11-03 17:41:43 +01:00
Alexander
2abf063ba2
Sessions cache layer (#789)
* feat(backend/db): cache layer for sessions with auto deleting logic
2022-11-02 17:00:41 +01:00
Alexander Zavorotynskiy
7a299fabc0 fix(backend): fixed possible panic in parseTags method 2022-11-02 12:45:38 +01:00
Alex Kaminskii
e1cd349cb9 fix(backend/db): truncate messageIndex to Int32 before insertion 2022-10-31 11:12:27 +01:00
Alex K
c7520166f3
fix(backend): use first frame for error identification (#790) 2022-10-27 16:29:48 +02:00
Alexander Zavorotynskiy
7e53bcf17c feat(backend/db): removed unnecessary logs 2022-10-26 11:52:36 +02:00
Alexander Zavorotynskiy
a286b0b2d2 fix(backend): fixed method call in redis consumer 2022-10-25 18:52:34 +02:00
Alexander
51b6414cb6
feat(backend): added convertor for old sessionEnd message (#786) 2022-10-25 14:50:02 +02:00
Alexander
c4f0252677
feat(backend/sink): added cache layer for assets url rewriter (#785) 2022-10-25 12:16:45 +02:00
Alex Kaminskii
0a8ece3573 feat(backend/http):store Delay in sess-token 2022-10-19 16:21:58 +02:00
Alex K
974afa454b
feat(backend): insert error tags+ (#768)
* feat(backend): transform legacy messages

* refactor(backend/postgres):remove seqIndex transformation

* feat(backend/db): parse & insert errors_tags

* chore(backend): remove building folder after build

* fix(backend/db):remove message types from filter that do not go through kafka 

* fix(backend/assets):use JSException instead of ErrorEvent

* refactor(backend/db):ErrorEvent is no longer a message

* feat(backend/db):insert error tags in clickhouse

* refactor(backend):remove unused LongTask code
2022-10-19 10:31:13 +02:00
Alexander Zavorotynskiy
35a58edec2 feat(backend): removed context from issue event 2022-10-18 14:12:19 +02:00
Alexander
a166482227
Session mod files encryption (#766)
* feat(backend): added session mod files encryption
2022-10-18 12:50:36 +02:00
Alex Kaminskii
d48a580d62 feat(mobs;backend): negative filtering for mob-file messages (no backend involved in plugin creation) 2022-10-17 10:29:47 +02:00
sylenien
117b1aca96 change(tracker/backend): change msg format for exceptions 2022-10-12 12:44:05 +02:00
sylenien
55abe72412 change: fix message format 2022-10-11 12:20:33 +02:00
sylenien
c7fa1e7fa5 feat(tracker): remove tags key 2022-10-11 12:20:33 +02:00
sylenien
d7e01e21af feat(backend/tracker): fix error type 2022-10-11 12:20:33 +02:00
sylenien
6e3e03dd46 feat(backend/tracker): add error messages 2022-10-11 12:20:33 +02:00
Alex K
c3fcda45d4
Devtools separation (#752)
* feat (backend+frontend/player): writing  devtools-related messages into a separate file
2022-10-07 16:20:48 +02:00
Alexander
617c78f224
Speed up Ender (#762)
* feat(backend): process only Timestamp messages in ender's messageHandler

* feat(backend): moved stats logger to session ender
2022-10-05 13:30:43 +02:00
Alexander
56ed06ed17
Message processing refactoring (#743)
* feat(backend): refactored message processing logic and cleaned up previous changes
2022-09-28 13:41:50 +02:00
Alexander Zavorotynskiy
9ff555cc64 feat(backend): set seek pointer at the end of batch for avoiding memory leaks 2022-09-19 10:59:00 +02:00
Alexander Zavorotynskiy
3b045fa864 feat(backend): call sessions table optimise method only in one db service 2022-09-13 15:48:23 +02:00
sylenien
f360d8416d feat(tracker): add zustand support 2022-09-09 11:55:31 +02:00
Alexander Zavorotynskiy
0eed5b58d4 feat(backend): extra checks to avoid nil cast panic 2022-09-08 16:21:41 +02:00
Alexander Zavorotynskiy
bef74d5284 feat(backend): skip corrupted batch 2022-09-06 14:35:16 +02:00
Alexander Zavorotynskiy
348a323a41 feat(backend): added extra checks for urlMethod and resourceType in CH connector 2022-09-02 22:28:57 +02:00
Alexander Zavorotynskiy
c526d611f5 fix(backend): fixed some issues in db service 2022-09-02 18:16:10 +02:00
Alexander Zavorotynskiy
a621dd39b4 feat(backend): added message_id to events table in CH 2022-09-02 11:59:13 +02:00
Alexander
3904509f18
ClickHouse bugs fix (#707)
* fix(backend): fixed issues in CH connector
2022-09-01 17:45:28 +02:00
Alex K
f699b03117
fix(backend/assets): once-a-day versioning; once-a-day re-uploading (#699) 2022-09-01 10:52:13 +02:00
Alexander Zavorotynskiy
6a956da100 fix(backend): fixed import error in db pg connector 2022-08-31 19:13:47 +02:00
Alexander
7421154939
Ch improvements (#705)
* feat(backend): in-memory batches for click house
* feat(backend): new scheme for ClickHouse tables
2022-08-31 19:00:26 +02:00
Alex K
f19a7df354
Merge pull request #694 from openreplay/adopted-style-sheets
Tracker 3.6.0 and message schema update
* feat (tracker, backend, player): Adopted Style Sheets maintenance
* refactor(tracker,player): compact messages representation (as array)
* feat (tracker,backend): Use real sessionStart timestamp decoded from token on start
* fix (frontend/assist): Fix activity timeouts logic in assist
* fix (tracker): maintain scroll, mousemove, mouseclick, exceptions inside iFrames
* fix (tracker): img module url resolving
* fix (tracker): critical bug in observer (missing nodes)
* feat (tracker): sessionHash returned on stop can be used for continuing session on start
2022-08-26 16:00:31 +02:00
Alexander Zavorotynskiy
95cff55e6c feat(backend): skip raw batch when get incorrect batch version 2022-08-24 15:25:16 +02:00
Alex Kaminskii
c4b3310c58 fix(backend): library use in message template 2022-08-24 10:49:46 +02:00
Alex Kaminskii
6fa68e7990 Merge branch 'dev' into adopted-style-sheets 2022-08-23 16:27:18 +02:00
Alexander
5887ab5ddb
Message protocol update (message size) (#663)
* feat(tracker): update message schema with BatchMetadata; separate message-related responsibilities; add message size
* chore(docker): removing edge busybox, as the main repo updated
* feat(backend): updated message protocol templates
* feat(backend): added support of message size
* feat(backend): implemented iterator for new message protocol (with message size)

Co-authored-by: Alex Kaminskii <alex@openreplay.com>
2022-08-22 12:32:48 +02:00
Alex Kaminskii
fe11fb06dc feat(backend):rewrite adoptedStyleSheet style messages 2022-08-17 20:05:35 +02:00
Alexander Zavorotynskiy
27806054e0 fix(backend): removed redundant files 2022-08-08 15:49:49 +02:00
Alexander Zavorotynskiy
e86ce25382 fix(backend): removed ch files from pkg 2022-08-08 13:29:33 +02:00
Alexander Zavorotynskiy
f3b431d64b feat(backend): implemented iterator for new message protocol (with message size) 2022-08-05 19:49:18 +02:00
Alexander Zavorotynskiy
d06bd8769f feat(backend): added support of message size 2022-08-04 16:33:51 +02:00