Available Built-in Methods

The TruffleClient object is your app's primary interface for interacting with the Truffle OS backend and client UI. It allows you to access core functionalities like model inference, built-in tools (like Perplexity search), user interaction, and more.

Import:

from truffle.client import TruffleClient

Initialization (typically within your app's __init__):

class MyApp:
    def __init__(self):
        self.client = TruffleClient()
        # ... rest of your init

---

Core Methods

.perplexity_search(...)

Accesses Perplexity AI's search capabilities directly through the Truffle backend. You don't need your own Perplexity API key.

# Example usage within a @truffle.tool decorated method:
query = "Explain quantum entanglement simply"
self.client.tool_update(f"Searching Perplexity for: '{query[:30]}...'") # Inform user
try:
    response_content = self.client.perplexity_search(
        query=query,
        model="sonar-pro" # Available: "sonar", "sonar-pro", "sonar-reasoning"
        # response_fmt="markdown", # Optional: specify format
        # system_prompt="You are a helpful assistant." # Optional
    )
    self.client.tool_update("Search complete.")
    return response_content # Return result to LLM
except Exception as e:
    self.client.tool_update("Search failed.")
    raise RuntimeError(f"Perplexity search error: {e}")

• Args:
  • query (str): The search query.
  • model (str): The Perplexity model to use (e.g., "sonar-pro").
  • response_fmt (str, optional): Desired response format (e.g., "markdown", "json").
  • system_prompt (str, optional): Custom system prompt for the Perplexity model.
• Returns: (str) The content of the Perplexity response.
• Raises: RuntimeError if the backend call fails.

---

.tool_update(...)

Sends a status message that is displayed transiently in the Truffle client UI, typically while a tool is executing. This provides real-time feedback to the user.

# Example: Before a long operation
self.client.tool_update("Processing large file...")
# ... perform long operation ...
self.client.tool_update("File processing complete.")

• Args:
  • message (str): The text to display in the client UI status area.
• Raises: RuntimeError on communication error.

---

.ask_user(...)

Pauses tool execution and prompts the user for input via the Truffle client UI. Execution resumes only after the user provides a response.

# Example: Getting confirmation or missing information
try:
    user_response = self.client.ask_user(
        message="Please enter your GitHub API token:",
        reason="Needed to access private repository data." # Optional context
    )
    token = user_response.get("response") # Response is a dict
    if token:
        # ... use the token ...
        return "API token received and used."
    else:
        return "User did not provide a token."
except Exception as e:
    raise RuntimeError(f"Failed to ask user: {e}")

• Args:
  • message (str): The question or prompt displayed to the user.
  • reason (str, optional): Context shown to the user explaining why the input is needed.
• Returns: (dict) A dictionary containing the user's input under the key "response". Example: {'response': 'user-typed-value'}.
• Raises: RuntimeError on communication error.

---

.infer(...)

Performs inference using one of the LLMs available on the user's Truffle instance. This allows your app to leverage generative AI capabilities directly. It supports streaming responses.

# Example: Generating a summary within a tool
prompt = "Summarize the following text: [long text here]"
summary = ""
try:
    # model_id=0 usually refers to the primary model
    stream = self.client.infer(prompt, model_id=0, max_tokens=100)
    self.client.tool_update("Generating summary...")
    for token in stream:
        print(token, end="") # Can process token by token
        summary += token
    self.client.tool_update("Summary generated.")
    return summary
except Exception as e:
    self.client.tool_update("Inference failed.")
    raise RuntimeError(f"Inference error: {e}")

• Args:
  • prompt (str): The input prompt for the model.
  • model_id (int, optional): The index of the model to use (from .get_models(), defaults usually to 0).
  • system_prompt (str, optional): System message for the inference.
  • context_idx (int, optional): Use existing chat history context (advanced).
  • max_tokens (int, optional): Maximum tokens to generate.
  • temperature (float, optional): Sampling temperature.
  • format_type (str, optional): Enforce output format ("TEXT", "JSON", "EBNF").
  • schema (str, optional): Schema definition (required for "JSON" or "EBNF").
• Returns: A generator yielding strings (tokens) as they are generated by the model.
• Raises: RuntimeError on backend/SDK communication errors.

---

.get_models()

Retrieves a list of the language models currently available and loaded on the user's Truffle instance.

try:
    available_models = self.client.get_models()
    if not available_models:
        print("No models found.")
        return

    print("Available Models:")
    for i, model_desc in enumerate(available_models):
        # Access protobuf fields like .name, .id, etc.
        print(f"  ID {i}: {model_desc.name} (Family: {model_desc.family})")

    # Example: Find the ID of a specific model
    llama3_id = -1
    for i, m in enumerate(available_models):
        if "llama3" in m.name.lower():
            llama3_id = i
            break
    # Use llama3_id in client.infer(..., model_id=llama3_id)

except Exception as e:
    raise RuntimeError(f"Failed to get models: {e}")

• Returns: (list) A list of ModelDescription objects (protobufs), each containing details about a model (like name, family, ID/index).
• Raises: RuntimeError on communication error.

---

.query_embed(...)

Performs semantic search over a list of documents provided by your app. It uses Truffle's embedding capabilities to find documents most similar to the given query.

# Example: Searching through meeting notes stored in the app
documents = [
    "Meeting 1: Discussed project alpha budget.",
    "Meeting 2: Brainstormed marketing strategy for beta.",
    "User Guide: How to set up project alpha.",
]
query = "information about alpha budget"

try:
    self.client.tool_update(f"Searching documents for: '{query}'")
    # Returns list of (document, score) tuples, sorted by similarity
    results = self.client.query_embed(query=query, documents=documents)

    if not results:
        return "No relevant documents found."

    # Format top results for the LLM
    top_results_str = "Found relevant documents:\n"
    for doc, score in results[:2]: # Get top 2
        top_results_str += f"- (Score: {score:.2f}) {doc}\n"

    self.client.tool_update("Document search complete.")
    return top_results_str

except Exception as e:
    self.client.tool_update("Document search failed.")
    raise RuntimeError(f"Embedding query failed: {e}")

• Args:
  • query (str): The natural language query.
  • documents (list[str]): A list of strings representing the documents to search within.
• Returns: (list) A list of tuples (document, similarity_score), sorted from most similar to least similar.
• Raises: RuntimeError on communication or embedding errors.