A student-built, offline-first AI assistant robot for the computer lab — from a desk companion to a fully mobile autonomous agent.
CLAW (Computer Lab Autonomous Workmate) is a real AI assistant that lives in the lab. It knows who students are, remembers past interactions, can answer questions, help debug code, and eventually move around the room. It runs entirely on local hardware — no student data ever leaves the building. The brain lives in the iMac. The body lives on a Raspberry Pi 5. Students build both.
Get the iMac running as the AI brain. Install Ollama, pull a local model, build the FastAPI server, lock down the network firewall. Every student runs this locally too on their machine to understand the stack before touching hardware.
Give CLAW a real memory. Build the vector database for semantic recall and SQLite for structured facts about people and places. This is the hardest conceptual phase — and the richest for teaching.
Give CLAW ears and a voice. Run OpenAI's Whisper model locally for speech-to-text, and a text-to-speech engine for output. The Pi handles audio capture and playback — the iMac handles transcription and generation.
CLAW learns to recognize students by face. When someone walks up, it greets them by name and recalls their last session. Built entirely with local models — no cloud vision APIs.
Build the physical enclosure and put CLAW on a desk. Servo-driven head movement, expressive LED eyes, a small screen for a face. The robot becomes a real presence in the lab.
Mount CLAW on a wheeled platform. Add navigation sensors. Implement basic autonomous movement and room mapping. This is advanced — a stretch goal that high-performing students can own.
Divide the class into four squads. Each squad owns a vertical slice of the system. Squads collaborate via the FastAPI interface — a real-world API contract between teams. Rotate students between squads each phase so everyone touches every layer.
Owns the iMac server. FastAPI endpoints, Ollama integration, prompt engineering, context window management, inference optimization.
Owns ChromaDB + SQLite. Schema design, embedding pipelines, RAG retrieval, entity extraction, memory summarization strategies.
Owns voice and vision. Whisper transcription, speaker output, face recognition, camera streaming, wake word detection.
Owns the Pi and physical hardware. GPIO, servo control, motors, sensors, Pygame face animations, enclosure design.
| Component | Runs On | Purpose |
|---|---|---|
| Ollama + Llama 3.3 | iMac | Local LLM inference — no cloud required |
| FastAPI | iMac | HTTP API server — Pi↔Brain communication |
| ChromaDB | iMac | Vector database for semantic memory retrieval |
| SQLite (stdlib) | iMac | Structured store — people, places, sessions, facts |
| openai-whisper | iMac | Offline speech-to-text transcription |
| deepface / face_recognition | iMac | Local face identification pipeline |
| PyAudio | Pi 5 | Microphone capture, audio buffering |
| pyttsx3 / espeak | Pi 5 | Offline text-to-speech output |
| picamera2 | Pi 5 | Camera feed capture and streaming |
| gpiozero / RPi.GPIO | Pi 5 | Servo, motor, LED, sensor GPIO control |
| Pygame | Pi 5 | Animated face display — teacher's home turf |
| ROS 2 | Pi 5 (Phase 5) | Navigation, SLAM, autonomous mobility |
┌─────────────────────────────────────────────────────────┐
│ iMAC — THE BRAIN │
│ │
│ FastAPI Server (:8000) │
│ POST /chat ← main conversation endpoint │
│ POST /transcribe ← audio → text via Whisper │
│ POST /identify ← image → person name │
│ GET /memory/{name} ← retrieve person's history │
│ GET /health ← status check from Pi │
│ │
│ [Ollama LLM] ← [Memory Retriever] ← [ChromaDB] │
│ ↓ │
│ [SQLite DB] │
│ │
│ Firewall: INBOUND open · OUTBOUND blocked │
└─────────────────────────────────────────────────────────┘
↕ WiFi (LAN only)
┌─────────────────────────────────────────────────────────┐
│ RASPBERRY PI 5 — THE BODY │
│ │
│ [Microphone] → buffer audio → POST /transcribe │
│ [Camera] → capture frame → POST /identify │
│ [Speaker] ← receive text ← speak response │
│ │
│ GPIO: │
│ Pan/Tilt Servos → head tracks speaker │
│ LED Matrix → expressive face/eyes │
│ Ultrasonic → obstacle detection (Phase 5) │
│ Motor HAT → wheel drive (Phase 5) │
│ │
│ Pygame display → animated face on small screen │
└─────────────────────────────────────────────────────────┘
This is Day 1 code for students. Get the brain talking before touching any hardware. Every student runs this on their own machine first.
# imac_brain/server.py — the minimal brain server # Run: uvicorn server:app --host 0.0.0.0 --port 8000 import ollama from fastapi import FastAPI from pydantic import BaseModel app = FastAPI(title="CLAW Brain") class ChatRequest(BaseModel): message: str speaker_name: str = "unknown" @app.post("/chat") async def chat(req: ChatRequest): response = ollama.chat( model="llama3.3", messages=[ {"role": "system", "content": f"You are CLAW, a helpful lab assistant. You are speaking with {req.speaker_name}."}, {"role": "user", "content": req.message} ] ) return {"reply": response["message"]["content"]} @app.get("/health") async def health(): return {"status": "online", "model": "llama3.3"}
# pi_body/client.py — the minimal Pi client # Students run this from their laptop first to test import requests BRAIN_URL = "http://imac.local:8000" def ask_claw(message: str, name: str = "Student") -> str: response = requests.post( f"{BRAIN_URL}/chat", json={"message": message, "speaker_name": name} ) return response.json()["reply"] # Test it if __name__ == "__main__": reply = ask_claw("Hey CLAW, what can you help me with?", "Alex") print(f"CLAW: {reply}")