Skip to main content
This quickstart demonstrates how to fine-tune an LLM using Ladder AI on a graph problem, either by loading an existing dataset or generating a new one from scratch.

​
1. Install Requirements

setup
pip install ladder-ai trl

​
2. Prepare Environment

Create a .env file in your project root with your HuggingFace and OpenAI API keys:
main.py
import os
os.environ["HF_TOKEN"] = os.environ.get("HF_TOKEN")
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY")

​
3. Generate Dataset

main.py

from ladder import  LadderConfig, create_dataset, finetune_model
from ladder.llms import OpenAIModel, OllamaModel

problem_description = "Your Problem Description here "

config = LadderConfig(
    # 1- Main LLM for dataset generation, ladder implementation
    instructor_llm = OpenAIModel(model="gpt-3.5-turbo", api_key=os.environ.get("OPENAI_API_KEY") ),

    # 2- To the run the same LLM for finetuning (make sure ollama is running locally)
    finetune_llm_runner = OllamaModel(model="llama3.2:latest",),

    # 3- Target LLM to finetune (hugginfacehub compatible)
    target_finetune_llm_id="meta-llama/Llama-3.1-8B-Instruct",

    # 4- Number of steps to finetune
    num_train_epochs=1,

    # 5- add more configs as needed
)

# Step1 - generate dataset (if u dont have one)
dataset = create_dataset(problem_description=problem_description, config=config, dataset_len=10)

​
4. Ladder

main.py

## VLadder Dataset
vladder_dataset = dataset.to_vladder()

# Step2 - Ladder Finetune
final_model = finetune_model(
    vladder_dataset=vladder_dataset,
    config=config,
    reward_funcs=[],
    use_ttrl=False
)

​
5. Next Steps

  • Explore the Finetuning and Engines docs for more options.
  • Try different problem domains or custom reward functions.
Effortless LLM fine-tuning—make small models smarter without data or supervision!