Cookbook: Observability for Groq Models (Python)

import os
 
# Get keys for your project from https://cloud.langfuse.com
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..." # Private Project
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." # Private Project
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"  # 🇪🇺 EU region
# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com"  # 🇺🇸 US region
 
# Your Groq API key
os.environ["GROQ_API_KEY"] = "gsk_..."

%pip install langfuse openai --upgrade

# Instead of: import openai
from langfuse.openai import OpenAI

client = OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key=os.environ.get("GROQ_API_KEY")
)

completion = client.chat.completions.create(
    model="llama3-8b-8192",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": "Write a poem about language models"
        }
    ]
)
print(completion.choices[0].message.content)

In silicon halls, a dream takes form,
A machine that speaks, yet lacks a norm.
A language model, born of code and might,
Generates prose, both wondrous and bright.

With algorithms fine, it weaves its spell,
Converting data into words to tell.
It learns from texts, both old and new,
And synthesizes meaning anew.

Its vocabulary vast, its syntax grand,
It crafts a language, at its command.
It chatters fast, with wit and fluency,
A fluent friend, for humanity.

It mimics tone, and echoes the heart,
A master of words, a work of art.
Yet, as we converse, we pause to think,
Can it truly understand, or just blink?

For language is a dance, both subtle and grand,
That requires nuance, and a human hand.
The model's skill, though impressive to see,
Falls short, at times, of humanity.

Still, as we shape its future course,
It brings us closer, to the universe.
A tool, to aid our creative spark,
A language model, that leaves its mark.

Let us guide it, with care and wit,
To discover new paths, day and night.
For in its realm, the possibilities abound,
A world where language, and machines entwine, profound.

%pip install groq langfuse

from groq import Groq
 
# Initialize Groq client
groq_client = Groq(api_key=os.environ["GROQ_API_KEY"])

from langfuse.decorators import langfuse_context, observe
 
# Function to handle Groq chat completion calls, wrapped with @observe to log the LLM interaction
@observe(as_type="generation")
def groq_chat_completion(**kwargs):
    # Clone kwargs to avoid modifying the original input
    kwargs_clone = kwargs.copy()
 
    # Extract relevant parameters from kwargs
    messages = kwargs_clone.pop('messages', None)
    model = kwargs_clone.pop('model', None)
    temperature = kwargs_clone.pop('temperature', None)
    max_tokens = kwargs_clone.pop('max_tokens', None)
    top_p = kwargs_clone.pop('top_p', None)
 
    # Filter and prepare model parameters for logging
    model_parameters = {
        "max_tokens": max_tokens,
        "temperature": temperature,
        "top_p": top_p
    }
    model_parameters = {k: v for k, v in model_parameters.items() if v is not None}
 
    # Log the input and model parameters before calling the LLM
    langfuse_context.update_current_observation(
        input=messages,
        model=model,
        model_parameters=model_parameters,
        metadata=kwargs_clone,
    )
 
    # Call the Groq model to generate a response
    response = groq_client.chat.completions.create(**kwargs)
 
    # Log the usage details and output content after the LLM call
    choice = response.choices[0]
    langfuse_context.update_current_observation(
        usage_details={
            "input": len(str(messages)),
            "output": len(choice.message.content)
        },
        output=choice.message.content
    )
 
    # Return the model's response object
    return response

@observe()
def find_best_painter_from(country="France"):
    response = groq_chat_completion(
        model="llama3-70b-8192",
        max_tokens=1024,
        temperature=0.4,
        messages=[
            {
                "role": "user",
                "content": f"this is a test"
            }
        ]
    )
    return response.choices[0].message.content
 
print(find_best_painter_from())

It looks like you're testing to see if I'm working properly! That's perfectly fine. I'm happy to report that I'm functioning as intended and ready to assist you with any questions or topics you'd like to discuss. Is there anything specific you'd like to chat about or ask?

from langfuse.decorators import langfuse_context, observe
 
@observe()
def find_best_painting_from(country="France"):
    response = groq_chat_completion(
        model="llama3-70b-8192",
        max_tokens=1024,
        temperature=0.1,
        messages=[
            {
                "role": "user",
                "content": f"Who is the best painter from {country}? Only provide the name."
            }
        ]
    )
    painter_name = response.choices[0].message.content.strip()
 
    response = groq_chat_completion(
        model="llama3-70b-8192",
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": f"What is the most famous painting of {painter_name}? Answer in one short sentence."
            }
        ]
    )
    return response.choices[0].message.content
 
print(find_best_painting_from("Germany"))

Albrecht Dürer's most famous painting is "Melencolia I" (1514), a complex and enigmatic work that is widely regarded as one of the most iconic and influential prints of the Renaissance.

from langfuse.decorators import langfuse_context, observe
 
@observe(as_type="generation")
def stream_groq_chat_completion(**kwargs):
    kwargs_clone = kwargs.copy()
    messages = kwargs_clone.pop('messages', None)
    model = kwargs_clone.pop('model', None)
    temperature = kwargs_clone.pop('temperature', None)
    max_tokens = kwargs_clone.pop('max_tokens', None)
    top_p = kwargs_clone.pop('top_p', None)
 
    model_parameters = {
        "max_tokens": max_tokens,
        "temperature": temperature,
        "top_p": top_p
    }
    model_parameters = {k: v for k, v in model_parameters.items() if v is not None}
 
    langfuse_context.update_current_observation(
        input=messages,
        model=model,
        model_parameters=model_parameters,
        metadata=kwargs_clone,
    )
 
    stream = groq_client.chat.completions.create(stream=True, **kwargs)
    final_response = ""
    for chunk in stream:
        content = str(chunk.choices[0].delta.content)
        final_response += content
        yield content
 
    langfuse_context.update_current_observation(
        usage_details={
            "total_tokens": len(final_response.split())
        },
        output=final_response
    )

@observe()
def stream_find_best_five_painter_from(country="France"):
    response_chunks = stream_groq_chat_completion(
        model="llama3-70b-8192",
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": f"Who are the best five painters from {country}? Give me a list of names and their most famous painting."
            }
        ]
    )
    final_response = ""
    for chunk in response_chunks:
        final_response += str(chunk)
        print(chunk, end="")
 
    return final_response
 
stream_find_best_five_painter_from("Spain")

What a great question! Spain has a rich artistic heritage, and choosing just five painters is a challenge. However, here's a list of five of the most renowned Spanish painters, along with their most famous works:

1. **Diego Velázquez** (1599-1660)
	* Most famous painting: "Las Meninas" (1656) - a masterpiece of Baroque art, showcasing the Spanish royal family and the artist himself.

2. **Pablo Picasso** (1881-1973)
	* Most famous painting: "Guernica" (1937) - a powerful anti-war artwork responding to the bombing of the town of Guernica during the Spanish Civil War.

3. **El Greco** (1541-1614)
	* Most famous painting: "The Burial of the Count of Orgaz" (1588) - a stunning example of Mannerist art, blending Spanish and Greek influences.

4. **Francisco Goya** (1746-1828)
	* Most famous painting: "The Third of May 1808" (1814) - a haunting depiction of the brutal suppression of a rebellion against Napoleon's army.

5. **Joan Miró** (1893-1983)
	* Most famous painting: "The Birth of the World" (1925) - a seminal work of Surrealist art, showcasing the artist's unique blend of abstract forms and vibrant colors.

These five painters have not only contributed significantly to the development of Spanish art but also left an indelible mark on the global art scene.None




'What a great question! Spain has a rich artistic heritage, and choosing just five painters is a challenge. However, here\'s a list of five of the most renowned Spanish painters, along with their most famous works:\n\n1. **Diego Velázquez** (1599-1660)\n\t* Most famous painting: "Las Meninas" (1656) - a masterpiece of Baroque art, showcasing the Spanish royal family and the artist himself.\n\n2. **Pablo Picasso** (1881-1973)\n\t* Most famous painting: "Guernica" (1937) - a powerful anti-war artwork responding to the bombing of the town of Guernica during the Spanish Civil War.\n\n3. **El Greco** (1541-1614)\n\t* Most famous painting: "The Burial of the Count of Orgaz" (1588) - a stunning example of Mannerist art, blending Spanish and Greek influences.\n\n4. **Francisco Goya** (1746-1828)\n\t* Most famous painting: "The Third of May 1808" (1814) - a haunting depiction of the brutal suppression of a rebellion against Napoleon\'s army.\n\n5. **Joan Miró** (1893-1983)\n\t* Most famous painting: "The Birth of the World" (1925) - a seminal work of Surrealist art, showcasing the artist\'s unique blend of abstract forms and vibrant colors.\n\nThese five painters have not only contributed significantly to the development of Spanish art but also left an indelible mark on the global art scene.None'