Sagemaker get embeddings

How to create embeddings¶

In this notebook, we will show you how to create embeddings for your own text data and and open source model from Hugging Face hosted as an endpoint on Amazon SageMaker.

0. Setup¶

Before you can use easyllm with Amazon SageMaker you need to deploy the model to a SageMaker endpoint. You can do this by following one of the bloh posts below:

• Creating document embeddings with Hugging Face's Transformers & Amazon SageMaker

Once you have your endpoint deploy copy the endpoint name of it. The endpoint name will be our model paramter. You can get the endpoint name in the AWS management console for Amazon SageMaker under "Inference" -> "Endpoints" -> "Name" or when you deployed your model using the sagemaker sdk you can get it from the predictor.endpoint_name attribute.

1. Import the easyllm library¶

In [ ]:

# if needed, install and/or upgrade to the latest version of the OpenAI Python library
%pip install --upgrade easyllm

# if needed, install and/or upgrade to the latest version of the OpenAI Python library %pip install --upgrade easyllm

In [ ]:

# import the EasyLLM Python library for calling the EasyLLM API
import easyllm

# import the EasyLLM Python library for calling the EasyLLM API import easyllm

2. An example chat API call¶

A embedding API call has two required inputs:

• model: the name of the model you want to use (e.g., sentence-transformers/all-MiniLM-L6-v2) or leave it empty to just call the api
• input: a string or list of strings you want to embed

Let's look at an example API calls to see how the chat format works in practice.

In [3]:

import os 
# set env for prompt builder
os.environ["HUGGINGFACE_PROMPT"] = "llama2" # vicuna, wizardlm, stablebeluga, open_assistant
os.environ["AWS_REGION"] = "us-east-1"  # change to your region
# os.environ["AWS_ACCESS_KEY_ID"] = "XXX" # needed if not using boto3 session
# os.environ["AWS_SECRET_ACCESS_KEY"] = "XXX" # needed if not using boto3 session

from easyllm.clients import sagemaker

embedding = sagemaker.Embedding.create(
    model="SageMakerModelEmbeddingEndpoint24E49D09-64prhjuiWUtE",
    input="That's a nice car.",
)

len(embedding["data"][0]["embedding"])

import os # set env for prompt builder os.environ["HUGGINGFACE_PROMPT"] = "llama2" # vicuna, wizardlm, stablebeluga, open_assistant os.environ["AWS_REGION"] = "us-east-1" # change to your region # os.environ["AWS_ACCESS_KEY_ID"] = "XXX" # needed if not using boto3 session # os.environ["AWS_SECRET_ACCESS_KEY"] = "XXX" # needed if not using boto3 session from easyllm.clients import sagemaker embedding = sagemaker.Embedding.create( model="SageMakerModelEmbeddingEndpoint24E49D09-64prhjuiWUtE", input="That's a nice car.", ) len(embedding["data"][0]["embedding"])

Out[3]:

768

In [ ]: