16efb1316c
* feature(intelligent-search): Added API to connect to Llama.cpp in EC2 and filter the response into OR filters * updated sql to filter script and added init.sql for tables * feature(intelligent-search): Changed llama.cpp for llama in GPU now contained in API * Updated Dockerfile to use GPU and download LLM from S3 * Added link to facebook/research/llama * Updated Dockerfile * Updated requirements and Dockerfile base images * fixed minor issues: Not used variables, updated COPY and replace values * fix(intelligent-search): Fixed WHERE statement filter * feature(smart-charts): Added method to create charts using llama. style(intelligent-search): Changed names for attributes to match frontend format. fix(intelligent-search): Fixed vulnerability in requiments and small issues fix * Added some test before deploying the service * Added semaphore to handle concurrency --------- Co-authored-by: EC2 Default User <ec2-user@ip-10-0-2-226.eu-central-1.compute.internal>
11 lines
413 B
Python
11 lines
413 B
Python
from decouple import config
|
|
from typing import Optional
|
|
|
|
ckpt_dir: str = config('CHECKPOINT_DIR')
|
|
tokenizer_path: str = config('TOKENIZER_PATH')
|
|
temperature: float = config('TEMPERATURE', default=0.6)
|
|
top_p: float = config('TOP_P', default=0.9)
|
|
max_seq_len: int = config('MAX_SEQ_LEN', default=4098)
|
|
max_gen_len: int = config('MAX_GEN_LEN', default=256)
|
|
max_batch_size: int = config('MAX_BATCH_SIZE', default=4)
|
|
|