Serverless API
#1
by alexanderniebuhr - opened
I'm trying to understand how to deploy this as an serverless API, so I can submit an image as an Input and receive the embeddings as an Output. Could you help with that?
For example, using a function that you can deploy via, say, FastAPI, or some serverless setup. This just creates embeddings for the specified page, and returns that as a JSON string:
images: list[Image.Image] = convert_from_bytes(pdf_data, dpi=300, first_page=page_number, last_page=page_number)
image = images[0]
embeddings = generate_embeddings(colqwen2_model, colqwen2_processor.process_images([image]))[0]
page_embeddings = [binary_quantize(e) for e in embeddings]
embeddings_list = [e.to_text() for e in page_embeddings]
return {"embeddings": json.dumps(embeddings_list)}
https://github.com/athrael-soju/fastapi-nextjs-colpali-template
The colpali folder is a standalong Colpali server running colqwen-0.2 as a standalone.