Commit graph

499 commits

Author SHA1 Message Date
ⵄⵎⵉⵔⵓⵛ
a75dc75aff
Chore small setup doc fixes (#1533) 2023-10-19 09:01:00 -04:00
rjshrjndrn
2b40fc3c6b fix(helm): frontend service monitor
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-19 00:42:26 +02:00
Rajesh Rajendran
88928ad6bc refactor(helm): skip db migration, in conditions (#1501)
1. if there are no version change
2. if skipMigration variable is set
3. if forceMigration variable is not set

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-11 12:17:51 +02:00
rjshrjndrn
cdefcc3540 chore(helm): Create domain name with custom port
else use standard scheme.

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-11 12:17:24 +02:00
Alexander
d22e4e138e
Backend for mobile tracker support (#1456)
* feat(backend): handlers for mobile messages

* feat(backend): new service template

* feat(backend): save mobile session start and send batches to kafka

* feat(backend): added mobile support to sink, ender and storage

* helm(videostorage): added helm chart for a new service videostorage

* fix(backend): added pointer to streing for userBrowser (because it's null for mobile sessions)

* feat(backend): added MsgIOSBatchMeta handler to message iterator's logic

* feat(backend): added ios ts parser to ender

* feat(backend): enabled producing batch of messages to queue

* feat(backend): removed iosstart from mob files

* feat(backend): added new ios message types

* feat(backend): added iosStart and iosEnd

* fix(backend): fixed log issue

* feat(backend): send tar.gz archives to special queue topic

* feat(backend): read raw archives from kafka

* fix(backend): added missing file

* fix(backend): removed the second file reading

* fix(backend): fixed wrong queue topic name

* fix(backend): fixed mobile trigger topic name

* feat(backend): added tar.gz extractor and iOSSessionEnd handler

* feat(backend): debug logs on message uploading

* fix(backend): added raw-images topic consumption

* feat(backend): now sink send iosSessionEnd to video-storage

* feat(backend): added dir creator for new sessions

* feat(backend): added correct command to execute

* feat(backend): added overwrite option

* feat(backend): added s3 uploader for video session replays

* feat(backend): new consumer group for mobile sessions

* feat(backend): debug logs for uploader

* feat(backend): removed unused log

* feat(backend): fixed s3 key for video replays

* feat(backend): removed debug code

* feat(backend): fixed video-storage message filter

* fix(backend): added mobileSessionEnd to SessionEnd converter

* feat(backend): added first version if db support for mobile events

* fix(backend): added swipe events to mob file

* feat(backend): added swipe event to pg

* feat(backend): split logic into 2 services: image-storage and video-storage

* feat(backend): added helm chart for image-storage service

* fix(backend): fixed table name for mobile taps

* feat(backend): added metadata handler for mobile message parser + fix message filters

* feat(backend): added iosRawTopic to DB message consumer

* fix(backend): removed value from mobile inputs

* feat(backend): removed debug log from iterator

* feat(backend): added new apple devices to iOS device parser

* fix(backend): added real projectID instead of 0

* feat(backend): extended a list of simulators for device detector

* feat(backend): updated networkCall mobile message

* fix(backend): added new way to define is network call successed or not

* feat(backend): added timezone support for mobile start request

* feat(backend): added 2 mobile events Input and Click to mob file

* feat(backend): refactored image storage

* feat(backend): video storage with 2 workers

* feat(backend): added project's platform support

* feat(backend): added memory size field for mobile start request

* feat(backend): changed video preset for ultrafast

* feat(backend): added debug log to http /late handler

* feat(backend): added debug log to db service for iosCrash messages

* feat(backend): added tapRage event handler to heuristics

* feat(backend): changed table and field names for ios crashes

* feat(backend): added payload for tapRage events

* feat(backend): added TapRage events insert to DB

* feat(backend): added fps value to /mobile/start response

* feat(backend): added image quality parameter to /mobile/start response

* feat(backend): added ScreenLeave handler

* feat(backend): removed screenEnter and screenLeave events, added new viewComponent event

---------

Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-09 15:02:20 +02:00
Rajesh Rajendran
5bb0e8bc82 Changes for clickhouse compression (#1500)
* feat(helm): Update clickhouse image

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* chore(helm): disabling ch_compression for chalice

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-04 19:51:45 +02:00
rjshrjndrn
aa40945597 feat(helm): enable minio init, only if enabled or forceInit
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-04 15:53:50 +02:00
Shekar Siri
8cd6b6216e
feat(api): assist stats (#1488)
* feat(api): assist stats

* refactor(helm): Updating helm code

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* fix(actions): build

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* feat(actions): deploy image to ee also

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-10-02 15:20:49 +02:00
rjshrjndrn
193661607c Revert "refactor(cli): Bind k3s api server to local port"
This reverts commit ebeedae502.
Sometimes proxy connection to the node fails. The ideal way to secure is
to have a security group only allowing access to the http/https ip to
the node.
2023-09-20 11:57:32 +02:00
rjshrjndrn
45f28ec33b feat(ingress): Hardening security headers
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-09-19 18:45:09 +02:00
rjshrjndrn
644da8999a feat(api): disable Caching
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-09-19 18:13:40 +02:00
rjshrjndrn
ebeedae502 refactor(cli): Bind k3s api server to local port
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-09-19 15:22:29 +02:00
rjshrjndrn
e9123fc6c2 chore(release): version update 2023-07-06 14:34:39 +02:00
rjshrjndrn
4522947ceb fix(helm): kafka persistence variable
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-26 16:09:00 +02:00
Rajesh Rajendran
8054abfb91 chore(cli): Allow override or only upgrade (#1377)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-24 00:50:16 +02:00
rjshrjndrn
5b8fd99290 chore(helm): Uninstall tooling because upgrade the chart is flaky
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-24 00:35:28 +02:00
Rajesh Rajendran
643cd07bea chore(cli): Don't upgrade tooling on regular patches (#1370)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-23 16:08:07 +02:00
Rajesh Rajendran
075bf95951 chore(cli): Cleaningup cli only on release upgrade (#1369)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-23 16:08:05 +02:00
rjshrjndrn
b777381f97 Posix compliant path decalaration 2023-06-23 01:02:09 +02:00
Rajesh Rajendran
12e8d84c18 chore(cli): Adding warn log (#1284)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-23 00:58:26 +02:00
rjshrjndrn
1c706f09f8 chore(cli): Clean up toolings
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-12 14:15:24 +02:00
rjshrjndrn
4180780b13 Updating kyverno 2023-06-09 18:13:40 +02:00
rjshrjndrn
5ad1dd58f1 chore(helm): Skip migration for reload
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-09 18:06:38 +02:00
rjshrjndrn
99b196156c chore(release): Updated version 2023-06-08 18:04:24 +02:00
Alexander
111d6a1b54
Azure support (#1270)
* feat(backend): implemented azure blob storage support

* feat(backend): added azure implementation to backend services

* feat(backend): added azure blob storage support to chalice service

* fix(backend): removed prev version of s3

* feat(backend): moved azure support to ee subfolder

* feat(backend): prepared ee code for new utils.objects package

* feat(backend): added missed modules to go.mod

* feat(backend): added missed modules to go.sum

* feat(backend): go mod tidy

* feat(backend): temporary made s3 vars are not required

* feat(backend): added azure lib to ee chalice

* feat(api): changed azure env var name

* feat(api): added new object store and extra methods to chalice ee

* fix(api): added azure blob lib to alerts

* fix(api): fixed incorrect call in sessions_devtool

* fix(crons): added azure blob storage library to requirements list

* chore(build): Error message for not providing flag.

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* feat(backend): removed ios headers and object store for ios messages

* feat(backend): object config refactoring

* chore(helm): Update BUCKET_NAME

* fix(backend): removed object storage usage in http

* feat(backend): added debug logs to azure upload method

* fix(backend): fixed empty folder issue

* fix(backend): removed extra debug log

* chore(helm): Adding global variables for crons

* chore(helm): Remove clickhouse resource limit

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>

* fix(backend): removed assets debug log

* feat(api): use ABC class instead of empty interface

* feat(api): renamed helpers to generators

* feat(api): changed prep/clean dev scripts

* feat(api): changed name obj_store -> StorageClient

* feat(api): some changes after code review

* fix(api): removed unnecesery packages in oss api

* feat(backend): moved azure implementation to ee folder

---------

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
Co-authored-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-08 13:31:54 +02:00
rjshrjndrn
1c21c80ae8 chore(helm): Adding cors as value header 2023-06-06 15:51:38 +02:00
rjshrjndrn
6d1d9d159d chore(helm): frontend ingress name override
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-06 11:13:30 +02:00
rjshrjndrn
35cc8e0f52 chore(helm): Adding frontend ingress overrid
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-06-06 11:01:07 +02:00
rjshrjndrn
8121ddfb45 chore(cli): Don't exit if kubeconfig doesn't found
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-25 14:39:20 +02:00
rjshrjndrn
49253f7a4d fix(helm): SITE_URL injection 2023-05-22 13:47:14 +02:00
Rajesh Rajendran
9678a17ffa chore(cli): Adding info on which kubeconfig is getting used (#1261)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-17 18:55:02 +02:00
Rajesh Rajendran
df7af56dab fix(cli): KUBECONFIG PATH override (#1266)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-17 18:55:02 +02:00
rjshrjndrn
6d0fef3c67 refactor(cli): In old clusters kyverno upgrade won't work.
So we'll have to upgrade OR only.

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-16 14:36:21 +02:00
rjshrjndrn
e0f7522752 chore(helm): enablig pg_stat for metrics
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-16 14:36:19 +02:00
Rajesh Rajendran
68272293b4 chore(helm): Updating frontend image release (#1243) 2023-05-09 17:58:51 +02:00
Rajesh Rajendran
4cfa516ce3 fix(cli): Override cli options (#1239)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-09 12:43:06 +02:00
rjshrjndrn
8f7b25bab1 chore(cli): Adding separate query for ee cleanup
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-05 19:29:38 +02:00
Kraiem Taha Yassine
52afb6e525
chore(minio): changed lifecycle rules to support delete-jobs (#1235) 2023-05-04 19:43:33 +02:00
rjshrjndrn
dce19801b7 chore(helm): enable kafka jmx metrics
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-04 12:37:35 +02:00
rjshrjndrn
e676dd0727 fix(helm): typo
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-04 12:05:20 +02:00
rjshrjndrn
693449e739 chore(helm): Update kafka chart
Enable metrics and increased storage
2023-05-04 11:26:06 +02:00
rjshrjndrn
874570baa3 chore(cli): Override GH repo
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-04 05:43:11 +02:00
rjshrjndrn
158299334e Updating kyverno 2023-05-04 05:34:50 +02:00
Taha Yassine Kraiem
1bdfc2ecfc Merge remote-tracking branch 'origin/api-v1.12.0' into dev
# Conflicts:
#	api/chalicelib/core/health.py
#	api/chalicelib/core/jobs.py
#	api/chalicelib/core/sessions_devtool.py
#	api/chalicelib/core/sessions_mobs.py
#	api/chalicelib/utils/s3.py
#	api/routers/crons/core_dynamic_crons.py
#	ee/api/chalicelib/core/assist_records.py
#	ee/api/chalicelib/core/health.py
#	ee/api/chalicelib/core/sessions_devtool.py
#	ee/api/chalicelib/core/traces.py
#	ee/api/requirements-crons.txt
#	ee/api/routers/crons/core_dynamic_crons.py
#	ee/api/schemas_ee.py
2023-05-02 15:37:20 +02:00
rjshrjndrn
2826cb4c4a fix(build): Parallel build
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-02 14:42:43 +02:00
Rajesh Rajendran
ae8e22709a Revert "Moving cli to scripts folder (#1196)" (#1197)
This reverts commit c947e48d99.
2023-05-02 11:18:47 +02:00
Rajesh Rajendran
0316999528 Moving cli to scripts folder (#1196) 2023-05-02 11:18:47 +02:00
Rajesh Rajendran
00e136f9cb Changing default encryption to false (#1161) 2023-05-02 10:57:13 +02:00
Rajesh Rajendran
cb64fcb3f0 chore(helm): Enabling redis string for helm template variable (#1159)
fix #1158

Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-02 10:55:57 +02:00
Rajesh Rajendran
6317d4b1cb chore(helm): Adding option for records bucket (#1146)
Signed-off-by: rjshrjndrn <rjshrjndrn@gmail.com>
2023-05-02 10:55:57 +02:00