How to use Text (Instruction) Completion clients¶

EasyLLM can be used as an abstract layer to replace text-davinci-003 with open source models.

You can change your own applications from the OpenAI API, by simply changing the client.

Chat models take a series of messages as input, and return an AI-written message as output.

This guide illustrates the chat format with a few example API calls.

1. Import the easyllm library¶

In [ ]:

# if needed, install and/or upgrade to the latest version of the OpenAI Python library
%pip install --upgrade easyllm

# if needed, install and/or upgrade to the latest version of the OpenAI Python library %pip install --upgrade easyllm

In [6]:

# import the EasyLLM Python library for calling the EasyLLM API
import easyllm

# import the EasyLLM Python library for calling the EasyLLM API import easyllm

2. An example chat API call¶

A text API call has two required inputs:

• model: the name of the model you want to use (e.g., meta-llama/Llama-2-70b-chat-hf) or leave it empty to just call the api
• prompt: a text prompt that is sent to the model to generate the text

Compared to OpenAI api is the huggingface module also exposing a prompt_builder and stop_sequences parameter you can use to customize the prompt and stop sequences. The EasyLLM package comes with build in popular methods for both of these parameters, e.g. llama2_prompt_builder and llama2_stop_sequences.

Let's look at an example chat API calls to see how the chat format works in practice.

In [1]:

from easyllm.clients import huggingface

# Example EasyLLM Python library request
MODEL = "meta-llama/Llama-2-70b-chat-hf"
huggingface.prompt_builder = "llama2"

# The module automatically loads the HuggingFace API key from the environment variable HUGGINGFACE_TOKEN or from the HuggingFace CLI configuration file.
# huggingface.api_key="hf_xxx"

response = huggingface.Completion.create(
    model=MODEL,
    prompt="What is the meaning of life?",
    temperature=0.9,
    top_p=0.6,
    max_tokens=1024,
)
response

from easyllm.clients import huggingface # Example EasyLLM Python library request MODEL = "meta-llama/Llama-2-70b-chat-hf" huggingface.prompt_builder = "llama2" # The module automatically loads the HuggingFace API key from the environment variable HUGGINGFACE_TOKEN or from the HuggingFace CLI configuration file. # huggingface.api_key="hf_xxx" response = huggingface.Completion.create( model=MODEL, prompt="What is the meaning of life?", temperature=0.9, top_p=0.6, max_tokens=1024, ) response

Out[1]:

{'id': 'hf-ZK--Ndk30h',
 'object': 'text.completion',
 'created': 1691129933,
 'model': 'meta-llama/Llama-2-70b-chat-hf',
 'choices': [{'index': 0,
   'text': " The meaning of life is a question that has puzzled philosophers, theologians, and scientists for centuries. There are many different perspectives on what constitutes the meaning of life, and there is no one definitive answer. However, some common themes that people often associate with the meaning of life include:\n\n1. Purpose: Having a sense of purpose or direction in life, whether it be through work, relationships, or personal goals.\n2. Fulfillment: Feeling fulfilled and satisfied with one's experiences and achievements.\n3. Happiness: Pursuing happiness and well-being, whether through personal relationships, material possessions, or personal growth.\n4. Self-actualization: Realizing one's potential and living up to one's capabilities.\n5. Legacy: Leaving a lasting impact or legacy, whether through contributions to society, artistic or cultural achievements, or impacting the lives of others.\n6. Spirituality: Connecting with a higher power or a sense of something greater than oneself, and finding meaning and purpose through faith or spiritual practices.\n7. Love: Finding and experiencing love, whether it be through romantic relationships, friendships, or family.\n8. Personal growth: Continuously learning, growing, and improving oneself.\n9. Community: Building and being a part of a community, whether it be through work, volunteering, or social connections.\n10. Making a difference: Making a positive impact in the world and leaving it a better place than when you arrived.\n\nUltimately, the meaning of life is a deeply personal and subjective question, and what gives meaning and purpose to one person's life may be different for another. It's a question that each person must answer for themselves, and it may change throughout their life as they grow and evolve.",
   'finish_reason': 'eos_token'}],
 'usage': {'prompt_tokens': 11, 'completion_tokens': 406, 'total_tokens': 417}}

As you can see, the response object has a few fields:

• id: the ID of the request
• object: the type of object returned (e.g., text.completion)
• created: the timestamp of the request
• model: the full name of the model used to generate the response
• usage: the number of tokens used to generate the replies, counting prompt, completion, and total
• choices: a list of completion objects (only one, unless you set n greater than 1)
  • text: the generated text
  • finish_reason: the reason the model stopped generating text (either stop, eos_token, or length if max_tokens limit was reached)
  • logprobs: optional the log probs of each generated token.

Extract just the reply with:

In [2]:

print(response['choices'][0]['text'])

print(response['choices'][0]['text'])

 The meaning of life is a question that has puzzled philosophers, theologians, and scientists for centuries. There are many different perspectives on what constitutes the meaning of life, and there is no one definitive answer. However, some common themes that people often associate with the meaning of life include:

1. Purpose: Having a sense of purpose or direction in life, whether it be through work, relationships, or personal goals.
2. Fulfillment: Feeling fulfilled and satisfied with one's experiences and achievements.
3. Happiness: Pursuing happiness and well-being, whether through personal relationships, material possessions, or personal growth.
4. Self-actualization: Realizing one's potential and living up to one's capabilities.
5. Legacy: Leaving a lasting impact or legacy, whether through contributions to society, artistic or cultural achievements, or impacting the lives of others.
6. Spirituality: Connecting with a higher power or a sense of something greater than oneself, and finding meaning and purpose through faith or spiritual practices.
7. Love: Finding and experiencing love, whether it be through romantic relationships, friendships, or family.
8. Personal growth: Continuously learning, growing, and improving oneself.
9. Community: Building and being a part of a community, whether it be through work, volunteering, or social connections.
10. Making a difference: Making a positive impact in the world and leaving it a better place than when you arrived.

Ultimately, the meaning of life is a deeply personal and subjective question, and what gives meaning and purpose to one person's life may be different for another. It's a question that each person must answer for themselves, and it may change throughout their life as they grow and evolve.

In [ ]: