Developer docs
HAi Robo Skills
Build something for a small robot that sits on a desk. A skill is a web page (HTML, CSS and JavaScript) that runs inside HAi Robo and can talk through it, move its eyes, colour its lights, turn its head, and read where your face is.
No native code, no build step, no SDK to install. If you can write a web page, you can write a skill.
await hairobo.voice.speak('Good morning');
await hairobo.led.set('#FF8A18');
await hairobo.eyes.look(0, 0.3);
Start here
- Open the simulator in a browser. It runs your skill with the robot APIs stubbed out, so you can build the whole thing before you ever touch a device.
- Download the kit, then copy
examples/starter/and editindex.html,plugin.jsandmanifest.json. You never include the SDK file itself: the app and the simulator both injecthairobofor you. - Upload the folder. We host it, so there is nothing for you to deploy.
- Open HAi Robo signed in with the same account. Your draft is at the top of Settings → Plugins → Browse the Plugin Gallery, marked DRAFT and visible only to you. Tap Try it to run it on your actual robot.
- Re-upload as often as you like. A draft is mutable and costs no version number. When it’s right, hit submit for review.
What a skill is made of
my-plugin/
manifest.json who you are, what you need → docs/manifest.md
index.html your entry point
plugin.js your code (call it whatever you like)
The script name is yours: the examples use plugin.js, breathe.js,
follow.js and rollcall.js. Only two names are fixed: manifest.json, and
whatever you set as entry.
Every file you want installed must be listed in the manifest’s files array.
There is no zip and no bundler. The app fetches exactly what you list.
Documentation
| manifest.md | Every field, which are required |
| api.md | The complete hairobo.* surface |
| permissions.md | What you can ask for, and what the user sees |
| limits.md | The sandbox. Read this before you build |
| publishing.md | Getting into the gallery |
Examples
All five ship inside the kit, under
examples/. Drop any of them on the simulator to
watch it run.
starter/ |
The smallest thing that runs |
hello/ |
Speaking and lights |
breathe/ |
A guided breathing session: timing, LEDs, voice |
follow-me/ |
Camera, events and head movement. The most complete example |
roll-call/ |
Multi-face and storage: a party game for a room of people |
character/ |
A character face. Pip, the example that ships in the app |
cab/ |
Owns a spoken phrase and opens the cab app. Ships in the app too |
Two things that catch people out
Your skill has no internet. The CSP blocks every outbound request. No fetch, no CDN, no analytics, no fonts. Everything ships in your folder. This is deliberate. See limits.md.
Permissions you ask for may be declined. Every robot call returns false
rather than throwing when the host declines it, and there may be no dock
connected at all. Write for that: a skill that only works with a robot
attached is a skill most people see broken.
manifest.json
{
"id": "com.yourname.myplugin",
"name": "My Plugin",
"version": "1.0.0",
"author": "Your Name",
"summary": "One line describing what this does.",
"entry": "index.html",
"mode": "session",
"permissions": ["voice", "eyes"],
"minAppBuild": 17,
"files": ["index.html", "plugin.js"]
}
| Field | Required | Notes |
|---|---|---|
id |
yes | Reverse-DNS, globally unique. This is your identity: installs, permissions and updates are keyed on it. Changing it makes a different skill. |
name |
yes | Shown in the gallery and the skill list. |
version |
yes | Semver. Bump it for every published change. |
author |
no | Shown to the user. |
summary |
no | One line, shown in the gallery. Write it for someone deciding whether to install. |
entry |
no | Defaults to index.html. |
mode |
no | session (default), ambient or face. See below. |
permissions |
no | Array of strings. See permissions.md. Empty means your skill is pure web content. |
intents |
no | Spoken phrases your skill claims, e.g. ["book a cab to the office"]. Tried after every built-in command. See commands.md. |
minAppBuild |
no | Refuse to install on older builds. Use it when you depend on a newer API. |
files |
no, but see below | Every file the skill needs, as a path relative to the manifest. Omit it and ONLY your entry file is installed. |
files
There is no zip anywhere in the pipeline: not in the upload, not on the
device. The app fetches each path in files individually.
files is optional in the code and effectively required in practice.
Leave it out and the installer falls back to [entry]: your entry file and
nothing else. Every stylesheet and script it references is silently absent, and
the skill installs cleanly and renders blank. Our own hello example shipped
exactly that bug. If your skill is one self-contained file you can omit it; the
moment there are two, list them.
Listing manifest.json itself is not needed. It is always fetched, and the
checker ignores it either way. Our examples list only the entry plus their
scripts.
So files is the definition of your skill. A file you upload but don’t list is
not installed, and a file you list but don’t upload is a blocking error at
upload time. A stylesheet missing from the list is the most common way a skill
works locally and ships blank.
mode
session: full screen, opened and closed by the user. Your page owns the
display; the robot’s face is not visible while you run.
ambient: replaces the idle face on the dock. Good for clocks, ambient
visuals, a custom face.
One thing to know about ambient: it yields the moment the robot leaves idle. If the user speaks to it, your skill gives up the screen until the conversation ends. Deep idle also always wins, for battery. Do not build an ambient skill that depends on running continuously.
face: a character. Your page IS the robot’s face, in every state: idle,
waking, listening, thinking, speaking, drowsy and asleep. Unlike ambient it
does not yield when the robot is busy, because drawing those moments is the
whole job. The app tells you what is happening through the state event; see
character.md.
Unknown fields and permissions
Unknown permission strings are dropped, not rejected: a skill built
against a newer SDK still runs on an older app, with the subset that build
understands. Combine that with minAppBuild and feature detection to decide
what “still runs” should mean for you.
The hairobo API
Injected automatically. You do not include a script. window.hairobo exists
before your code runs.
Everything returns a Promise. Calls resolve false rather than throwing when
the host declines them: no permission, no dock, rate limited. Write for that.
const spoke = await hairobo.voice.speak('Hello');
if (!spoke) { /* muted, busy, or not permitted: carry on */ }
voice (needs voice)
voice.speak(text) → Promise<bool>
Speaks through the robot in its own voice. Capped at 300 characters, and at
most one line every 0.4s; a call inside the gap resolves false and is
dropped, not queued. Resolves false while the robot is already talking.
eyes
In
mode: "session"your page covers the face, so eye movement is invisible while you run. These calls belong inmode: "ambient". Needseyes.
eyes.look(x, y) → Promise<bool>
Points the gaze. Both are roughly −1…1, x right-positive, y up-positive.
eyes.express(name) → Promise<bool>
One of happy, curious, thinking, sad, excited, calm. Unknown names
are ignored rather than erroring.
led (needs led)
led.set(color) → Promise<bool>
Hex string, e.g. '#37D2F0'. The dock’s ring.
led.reset() → Promise<bool>
Back to whatever the app would show. Call this when you finish, though teardown does it for you if you forget.
motor (needs motor · needs a dock)
Degrees are relative to where the head is now. The host clamps them, so ask for what you want and let it decide.
motor.pan(degrees) → Promise<bool>
Turns left/right. Clamped to ±20° per call.
Positive degrees turn the same way positive f.x lies. So to follow a face
you pass the offset straight through: motor.pan(f.x * step), not -f.x.
The negation belongs only to anything that MIRRORS the face, like drawing
pupils that look toward the user; the head turns toward it instead, which is
the opposite operation. Getting this backwards makes the robot turn away, keep
turning until it reaches its pan limit, and then sit there doing nothing.
motor.followFace(on) → Promise<bool>
If you want to follow a face, use this. It hands the job to the robot’s own tracking, the same loop behind the built-in follow-me.
await hairobo.motor.followFace(true); // start
await hairobo.motor.followFace(false); // stop
You send nothing else. No camera events, no targets, no timers. The app reads its face tracker directly and drives pan and lift itself.
Why this exists, and why followPan is not the same: anything you drive
from face events works from data that arrives at about 6 Hz and is already
~160 ms old. The app’s own loop is fed at frame rate with no round trip.
That gap cannot be tuned away: a skill reconstructing the servo will always
settle less cleanly than the loop that sees the face first-hand.
Needs camera as well as motor (it turns the camera on for you) and
stops when your skill closes. An update that adds a permission does not inherit
consent, so the user is asked again.
If you already hand-rolled a follower, DELETE IT. Every workaround built to fight the old stepping is now actively harmful:
- your stop band and settle logic
- sending a centred target once settled
- scaling speed down near centre
- adjusting gain when the face crosses the middle
- the
on('face')handler, if it exists only to drive the motor - any timer that re-sends targets
Two control loops arguing produce worse motion than either alone. Adding
followFace on top of existing tracking code is the most likely way to conclude
this API is broken. You send nothing: no events, no targets, no timers.
Use followPan below only when you need the head pointed at something that is
not a face: a target of your own choosing.
motor.followPan({targetX, smoothing, maxSpeed}) → Promise<{ok, errorDeg, settled}>
motor.stopFollowPan() → Promise<true>
For following a face, prefer motor.followFace above. Use this for a target
only your skill knows about.
motor.pan sends one pulse and stops, so following with it is visibly stepped
however often you call it, and the 0.35 s budget means most of your calls are
dropped anyway. followPan hands the robot a target instead and lets it run
the servo: smoothing, acceleration, speed limit, deadzone and the stop are all
the host’s.
hairobo.on('face', (f) => {
if (f.detected) hairobo.motor.followPan({ targetX: f.x });
else hairobo.motor.stopFollowPan();
});
targetX is where the face is, in the same −1…1 space as f.x. Pass it
straight through: no negation, exactly as with motor.pan.
smoothing |
fraction of the remaining error taken per host tick. 0.02 to 1, default 0.2. Lower is smoother and lazier. |
maxSpeed |
degrees per second, default 12, capped by the host at 30. A ceiling, not the speed it travels at. |
deadZone |
stop below this angular error, in degrees. Default 1.5. |
resumeZone |
do not move again until the error exceeds this for two consecutive ticks. Default 3.5, always greater than deadZone. |
Resolves { ok: true, errorDeg, settled }: the current angular error in
degrees, and whether the head has stopped. Resolves false with no dock or
while the robot is levelling.
It used to resolve exactly true. if (r) still works; r === true does
not.
What the host is doing on your behalf, since it decides how this feels:
- Your normalized
targetXis converted to a real angle using the camera’s own field of view, so the gain does not depend on the lens. - The loop runs at 20 Hz (servo rate, not the ~6 Hz camera rate), and the motion already commanded since your last target was measured is subtracted from the error. That dead-time term is why it stops cleanly: without it the loop re-corrects for a move still in flight, overshoots, and hunts.
- Speed tapers steeply near centre, so the last couple of degrees are a crawl while a large error still runs at full speed.
- Acceleration is limited, so it eases in and out rather than stepping.
- The dead zone has hysteresis and a two-tick confirmation, so a settled head needs a real movement to wake it rather than sensor noise.
Keep sending targets. Go quiet for more than about half a second and the host stops the motor. That is deliberate: the dock firmware has no watchdog of its own and keeps driving the last command it was given, so a skill that crashes or stalls mid-follow would otherwise leave the head turning until something physical stopped it. Silence has to mean stop.
You do not need to throttle your calls. A target update is not a movement (the host decides when the motor actually turns), so calling it on every camera event is exactly right.
There is no motor.lift: the third axis is the app’s
The dock has three motors: pan (left/right), lift (up/down) and tilt (roll). Only pan and tilt are here, and that is deliberate rather than an oversight. Two reasons, both structural:
Lift down is the one direction that can damage the hardware. It brings the dock’s base toward the phone and can knock it off the mount, so every lift-down command passes a safety check that reads live app state (dock geometry, what the robot is currently doing) and refuses when that state says no. A skill has none of that context, and an API that silently refuses most calls would be worse than no API.
There is no “move N degrees” primitive for lift. Pan and tilt take clamped relative degrees. Lift is pulse-and-hysteresis: short bursts with a rest between them, tuned as a closed loop against live face tracking. Exposing it would mean inventing an open-loop degree model that does not exist today.
It is also the axis the app uses for auto-centring, so a skill nudging it would be pulling against the app’s own correction in a way pan does not.
If you are watching the robot raise and lower itself and wondering which call does that: none of them. That is the app.
motor.tilt(degrees) → Promise<bool>
Rolls the head. Clamped to ±10°, and refused entirely if it would take the phone more than 25° from level. Past that, the dock’s base can strike the phone. Also refused if the app cannot currently read its own angle.
motor.center() → Promise<bool>
Return to level. Takes a second or two of visible movement.
At most one motor move every 0.35s. Anything faster resolves false and is
dropped, so an animation loop can call freely without building a backlog. The
gap is measured between moves that actually happened: a call that turns
nothing does not start the clock.
What false means
false is never an error. It means the robot did not turn, and there are
four reasons it might not have:
- there is no dock connected
- less than 0.35s since the last move
- the request rounded to less than half a degree (
motor.pan(0)and the sub-degree tail of an easing curve both land here) - the host refused it: tilt more than 25° from level, or an IMU reading too stale to know which way is up
You cannot tell these apart, deliberately. A skill that behaves differently
depending on why the robot declined is a skill fighting the host. Treat
false as “not this time” and carry on.
Movement reads best with a deadzone. Turning on every small change looks
nervous rather than attentive. See follow-me/.
camera (needs camera)
You get positions, never pixels. There is no API that returns a frame, and the image never leaves the phone.
camera.start() → Promise<bool> · camera.stop() → Promise<bool>
Start it, then listen. Stopped automatically when your skill closes.
Events arrive at roughly 6/second:
hairobo.on('face', (f) => {
if (!f.detected) return;
moveEyes(f.x, f.y); // the TRACKED face
});
hairobo.on('hand', (h) => { /* h.x only */ });
The event carries the whole room as well:
{ detected, x, y, count, faces: [{ x, y, size }, ...] }
x/y are the tracked face: the largest one, or whoever the robot is
holding a gaze lock on. faces is every face it can see, up to 8, each
with size = apparent width, the only depth cue there is: bigger means nearer.
Order is by x ascending, on the same axis as f.x. That is the axis motor.pan
uses, so hairobo.motor.pan(faces[i].x * 20) turns toward face i. Which
physical side index 0 sits on is deliberately unspecified: the camera feed is
mirrored, so “leftmost” depends on whose left you mean. Drive the motor from
the number; don’t label a seat in your UI.
Positions are not people
There is no face recognition in this API and there is not going to be one.
faces[2] is a seat, not a person: two people swapping chairs swap
indices, and someone leaving reindexes everyone after them. You can build
turn-taking, counting, “is anyone still here”, scan-and-pick. You cannot build
anything that remembers who someone is, and a skill that acts as if it can
will be wrong in front of a room full of people.
See roll-call/ for a party game built entirely within that limit.
x and y are positions, not corrections: they say where the face is,
not which way to move. Anything that mirrors the face (pupils) negates them;
anything that turns toward it (motor.pan) does not. See motor.pan above.
data (needs data)
data.weather() → Promise<object>
Current weather from the app. Your only source of outside information. You have no network of your own.
storage (needs NO permission)
Survives between runs. Scoped to your skill id: you read back only what you
wrote, no other skill can see it, and it is deleted when the user uninstalls
you. This is the only thing that persists. The WebView’s own localStorage is
wiped every run.
Values are strings. set stringifies; get does not parse.
storage.get(key) → Promise<string|null>
An absent key resolves null. That is the normal first-run case, not an error.
storage.set(key, value) → Promise<true>
storage.remove(key) → Promise<bool>
Resolves whether anything was actually there.
storage.keys() → Promise<string[]> · storage.clear() → Promise<true>
storage.usage() → Promise<{used, limit, keys, maxKeys, maxValueBytes}>
Bytes. Show your own usage rather than discovering the cap by hitting it.
if (!hairobo.storage) return; // older app, see below
const raw = await hairobo.storage.get('save');
const save = raw ? JSON.parse(raw) : { level: 1, best: 0 };
save.best = Math.max(save.best, score);
try {
await hairobo.storage.set('save', JSON.stringify(save));
} catch (e) {
// A cap was hit. Retrying will not help (see below).
}
A cap REJECTS the promise. It does not resolve false. Everywhere else
in this API false means “dropped, try again”: a rate limit that clears by
itself. A full store does not empty itself, so reporting it the same way would
have you retrying forever. Caps: 200 keys, 128-character keys, 16 KB a value,
256 KB in total.
Character state (mode face, needs NO permission)
Characters receive one event with everything they need to draw. It arrives on every change and once a second, so subscribing late still catches up.
hairobo.on('state', (s) => {
draw(s.state, s.mood, s.gazeX, s.gazeY);
});
state |
idle, drowsy, sleeping, listening, thinking, speaking, smile, happy, tracking, error |
mood |
calm, happy, curious, thinking, sad, excited |
gazeX, gazeY |
−1…1, where the eyes would look. Same space as f.x on the face event. |
idleSeconds |
since the person last did anything. The built-in eyes go drowsy at 45 and asleep at 70. |
docked |
the phone is on the robot |
dimmed |
the screen is dimmed for deep idle |
Needs "mode": "face" and app build 26. See character.md for
the rules a character has to follow.
open (needs open)
open.cab(to) → Promise<bool>
Opens the user’s cab app with to as the destination: a saved place name like
'office', or an address. The app resolves the place, finds the pickup point
and chooses the app. You never see the address or where the user is.
Resolves false if called more than once every few seconds.
There is no “open this link” and there will not be; see permissions.md.
Spoken commands (needs NO permission)
Claim phrases in your manifest with intents, then:
hairobo.on('command', (c) => { // c.text = what was said, c.phrase = yours
hairobo.command.done(); // finished; the app stops you anyway at 25s
});
Your skill runs with no screen, and the app shows your skill’s name on the face while it answers. See commands.md.
Always available
info.get() → Promise<object>
App build, apiVersion, whether a dock is connected, which permissions you
were granted, whether the screen is dimmed. Check info.get() before offering robot
features, rather than discovering there is no dock by being refused.
log(message)
Goes to the app’s debug log. Invisible in release builds: your own debugging, not a user-facing channel.
wait(ms) → Promise
on(event, fn)
version: the API generation this host provides. Currently 5.
2 was the original surface; 3 added storage.* and count/faces on the
face event; 4 added motor.followPan; 5 added motor.followFace. Prefer feature-detecting the namespace over comparing the number.
assistant (needs assistant)
assistant.ask({system, messages}) → Promise<string>
Ask HAi Robo’s assistant and resolve with its reply.
const reply = await hairobo.assistant.ask({
system: 'You are a patient English tutor. Correct gently, one point at a time.',
messages: [{ role: 'user', content: said }]
});
messages:{role: 'user'|'assistant', content}, most recent last, and it must contain at least oneuserturn. Capped at 12 turns, 2000 characters each;systemis capped at 1200.- One question in flight per skill. Asking again before the first resolves rejects.
- Rejects with
daily limit reachedwhen the user’s own allowance is spent. Do not retry. Tell them. - Write
systemas a literal string in your source. It is reviewed like code, and a prompt assembled at runtime cannot be read by the person deciding whether to publish your skill. Those submissions are rejected. Dropping the user’s words into an otherwise literal prompt is fine; building the instruction itself out of pieces is not. See CONTENT-POLICY.md.
mic (needs microphone)
mic.listen(maxSeconds?) → Promise<string|null>
Listen for one turn and resolve with what the user said.
const said = await hairobo.mic.listen(8);
if (!said) return; // silence, or the robot was busy
maxSeconds2 to 20, default 8.- Resolves null on silence. That is normal. Handle it rather than treating it as an error.
- If the robot is still speaking, this WAITS for it to finish rather than
refusing, so
speakthenlistenreads naturally.voice.speakresolves when the line is queued, not when it is spoken. Null after a 15 s wait means the robot never became free. - You get the transcript, never audio. The app’s own listening indicator is shown, so the user always knows.
The tutor loop
while (running) {
await hairobo.voice.speak('Your turn.');
const said = await hairobo.mic.listen(8);
if (!said) continue;
const reply = await hairobo.assistant.ask({ system: TUTOR, messages: [{ role: 'user', content: said }] });
await hairobo.voice.speak(reply);
}
Both need minAppBuild: 21. Declare it, or your skill will not run and
the reason will depend on how old the app is:
- Builds before 21 drop a permission they do not recognise and install the
skill under-granted. Your calls then fail at runtime with nothing on
screen to explain it. Nothing can be done about those builds now. They are
already released, which is exactly why
minAppBuildexists. - Build 21 and later refuse the skill outright and say which permission they did not understand. Loud, and much easier to diagnose.
The portal blocks a submission that uses assistant or microphone without
it, for the same reason.
Feature-detect anything new
On an older app, a newer namespace is undefined, and the property access
throws before your .catch can run, leaving your skill silently stuck:
if (!hairobo.camera) {
status.textContent = 'Needs a newer Hai Robo';
return;
}
Either that, or set minAppBuild and refuse to install at all.
For the motor-following calls, feature-detect. Do NOT set minAppBuild
yet. Neither motor.followFace nor motor.followPan is in a shipping build,
and there is currently no build number that means “has them”: the released
build and the development build carry the same number and different APIs. A
skill declaring one would be offered to users who cannot run it. Detect down
the ladder instead:
if (hairobo.motor.followFace) { /* best: the app's own tracking */ }
else if (hairobo.motor.followPan) { /* host servo from your targets */ }
else { /* discrete motor.pan steps */ }
This note goes away once a build ships with them.
Characters: your own face for the robot
A character replaces the robot’s eyes. Not the idle screen, the whole face: waking, listening, thinking, speaking, going drowsy, falling asleep. The app behaves exactly as it always does. Only the drawing is yours.
{ "mode": "face", "minAppBuild": 26 }
A character needs no permissions at all. It sees no camera, no microphone, no data. It is given what the robot is doing, and it draws.
The whole contract is one event
hairobo.on('state', (s) => {
draw(s);
});
state |
idle, drowsy, sleeping, listening, thinking, speaking, smile, happy, tracking, error |
mood |
calm, happy, curious, thinking, sad, excited |
gazeX, gazeY |
−1…1. Where the eyes would look, so your character can follow the person. |
idleSeconds |
since the person last did anything |
docked |
the phone is on the robot |
dimmed |
the screen is dimmed because nobody is there |
It arrives on every change and once a second, so subscribing late is fine, and
idleSeconds keeps ticking while nothing else happens.
Match the built-in timing unless you have a reason not to: the eyes go drowsy at 45 seconds and asleep at 70.
What the app keeps
The app still draws the listening indicator, the captions, the settings button and the dock badge, on top of you. A character cannot hide or fake them. That is what makes it safe to let a stranger’s code be the face.
Your page takes no touches. Tapping the face talks to the robot and a long press opens Settings, both of which belong to the app. Build the character to be watched, not tapped.
You will be taken off screen during games and camera routines, which draw their own screen, and in deep idle, which stops drawing entirely for battery. Do not depend on running continuously.
Speaking and moving
A character may also have manners. With voice it can greet someone; with
motor it can nod or turn the head. Both are ordinary permissions, declared in
the manifest and approved by the user at install, and the app keeps the motor
limits and the speaking rate limit.
Use them sparingly. A face that talks every time it is looked at is charming once and exhausting by the afternoon.
Battery, because a face runs all day
- Stop drawing when the page is hidden:
if (document.hidden) return; - Then redraw when it comes back. A hidden page gets no animation
frames at all, so nothing redraws it and the last frame you painted is
what the user sees on return. The example hit exactly this: it fell
asleep, the page was hidden, the state changed, and the sleeping marks
were still on screen afterwards. Listen for
visibilitychangeand ask for one frame when the page is visible again. - Animate with
requestAnimationFrame, never a busy loop. - Do not redraw a still frame. If nothing changed, draw nothing.
- Keep it to a few shapes. This runs for hours on a phone that is also listening, tracking a face and driving motors.
Anything that stutters or overheats is dropped back to the built-in eyes and can be suspended remotely, without an app update.
Review
On top of the normal checks, a character is looked at for:
- Flashing and strobing. This is a screen people sit in front of all day. Nothing that pulses hard at a few times a second.
- Battery over a long idle run.
- Nothing that imitates the app’s own interface, its permission prompts, or the listening indicator.
Try it
character/ in this kit is the example that ships inside the app, called Pip:
two eyes, a mouth, and about a hundred lines. Copy it, change the drawing, keep
the event handler.
On the phone: Settings, then Plugins, then tap your character in the list. Tap it again to go back to the built-in eyes.
Commands: your skill owns a phrase
A skill can claim spoken phrases. Say one, and your skill runs and does the work: no opening it, no tapping.
{
"intents": ["heading to the office", "off to the office"],
"permissions": ["open", "voice"],
"minAppBuild": 26
}
hairobo.on('command', (c) => { // c.text, c.phrase
hairobo.voice.speak('Getting you a cab to the office.');
hairobo.open.cab('office').then(() => hairobo.command.done());
});
That is the whole feature. cab/ in this kit is the working example.
Where your phrase sits
Every one of the app’s own commands is tried first. Yours is tried after them and before the assistant. You can add what the app cannot do; you can never take away what it can. “What’s the weather” stays the app’s, whatever you claim.
Between skills, the longest phrase wins, and a tie is broken the same way on every device rather than by who installed first. A suspended skill claims nothing.
The rules for a phrase
- At most 8 per skill.
- At least 6 characters, letters, numbers, spaces and apostrophes only.
- Claim whole phrases people say, not words. A skill claiming “cab” or “play” swallows half the app’s own commands the moment it is installed, and the user has no way to see why the robot stopped understanding them. Review will refuse it.
Your phrase is matched anywhere in what was said, lowercased, so “right, I am heading to the office now” matches “heading to the office”.
Phrases the app already owns
This is the mistake to avoid, and it is a quiet one.
The app answers “book a cab to the office” itself. If you claim that phrase, your skill is installed, the phrase looks claimed, and it never runs. The app books the cab, the user sees something happen, and nothing anywhere says your skill was skipped. It looks like your skill is broken.
The example in this kit claims “heading to the office” for exactly that reason. The app has no answer for it, so the phrase is really yours. The skill still ends up opening a cab, because that part is the app’s job either way.
So claim the phrase the app does NOT have. Before you publish, say your phrases to a robot with your skill uninstalled: anything that already works is not yours to take.
Written out in Config.swift terms, the app’s own travel commands cover
“book a cab”, “get me a ride”, “cab to”, “uber to”, “navigate to”, “take me
to”, “directions to” and their obvious relatives. That list is not the whole
app, which is the point of testing rather than reading.
How your skill runs
With no screen. It is started when the phrase is heard, receives the
command event, does its work, and stops when you call hairobo.command.done().
If you never call it, the app stops you after 25 seconds. There is nothing to
draw and nobody to look at it: do the work, say something, finish.
Everything you already have works here: voice.speak, storage.*,
assistant.ask, data.weather, and the rest, subject to the permissions you
declared.
The user is told it is you. While your skill is answering, the app shows your skill’s name on the robot’s face. You cannot draw it, hide it or change it. A skill speaking in the robot’s own voice without the user knowing which is which would be the whole trust problem in one feature, so the app answers it rather than asking you to.
Doing things: ask, do not reach
await hairobo.open.cab('office'); // needs "open"
You name a place. The app resolves it, finds the pickup point and opens the cab app. You never see the address, the coordinates, or which cab app they use.
There is deliberately no “open this link”. A skill has no network, and a link you chose would be the way around that: anything you know could be put in the URL and read by whoever opens it. So the list of things you can open grows as whole actions, one at a time, and each one is the app doing the job on your behalf.
Today that list is: a cab to a place. More will follow the same shape.
open.cab resolves false if you call it more than once every few seconds:
opening another app takes the screen away from the user, and a loop of that is
unusable.
Saving a preference
Your own store, as always, and it needs no permission:
await hairobo.storage.set('destination', 'office');
That is how a skill remembers “the office” without the app knowing what your skill means by it.
Try it
cab/ in this kit is the example, and it also ships inside the app. Say
“heading to the office” and watch the cab app open with the destination set.
Save a place called “office” in the app first, under Your stuff.
Say “book a cab to the office” instead and the cab still opens, but the app did it and your skill never ran. That is the difference worth seeing once.
Permissions
Declare what you need in manifest.json. The user is asked once, at install,
and sees your list before agreeing.
Your bridge is built from what was granted, not what you asked for. A
declined permission is not an error at install time. Your skill runs, and
those calls resolve false. Handle it.
| Permission | What the user is told | Grants |
|---|---|---|
voice |
“Speak” | voice.speak |
eyes |
“Move the eyes” | eyes.look, eyes.express |
led |
“Change the dock lights” | led.set, led.reset |
motor |
“Move the robot” | motor.pan, motor.tilt, motor.center |
camera |
“Use the camera” | camera.start/stop and face/hand events |
data |
“Read weather” | data.weather |
assistant |
“Ask the assistant” | assistant.ask |
microphone |
“Listen to you” | mic.listen |
open |
“Open other apps” | open.cab |
storage is not on this list and never will be. A skill keeping its own
data under its own id is the same trust level as localStorage on a website:
you read back only what you wrote, no other skill can see it, and there is no
network for it to leave by. Asking the user to approve that would be a prompt
with no decision in it, and prompts nobody can meaningfully answer are how
consent stops meaning anything for the permissions that do matter.
motor, camera, microphone and assistant are marked sensitive and
are presented more prominently. assistant is on that list because it spends
the user’s own daily allowance and puts your text in front of the model on
their behalf, not because it can reach their data. Ask for them only if you use them. A permission you request and
never call costs you installs.
What camera is not
It does not give you the picture. You receive face and hand positions; there is no API that returns a frame, and the image never leaves the phone.
It also does not give you identity. You get how many faces are in front of the robot and where each one is, never who. Face recognition exists elsewhere in the app, behind its own separate consent, and it is deliberately not wired to skills: a face embedding is biometric data, and handing it to third-party code is a different promise from the one on this screen.
Only the skill that called camera.start() receives these events.
The user still sees the camera indicator while your skill runs, so tell them why you need it.
Updates
An update that adds a permission does not inherit consent. The user is
asked again, and until they agree the new permission stays ungranted while the
rest keep working. Adding camera in v2 will not silently gain you a camera.
Users can change their mind
Permissions can be reviewed and revoked per skill at any time in Settings.
Check info.get() at startup rather than assuming what you had last run.
eyes only does something in ambient mode
A session skill is presented over the whole screen, and on this device the
screen is the face. hairobo.eyes.look() still runs and still returns true:
it moves a face that is underneath your page, invisible until you close.
So in session mode, animate your own DOM. Your page is the display.
In ambient mode you replace the idle face, and eyes is exactly the right
tool.
Uploading a session skill that asks for eyes gets a warning, not an error.
There are legitimate reasons to set an expression that outlives your session.
But if you are animating eyes to be watched, check your mode first.
What assistant is not
It is not an API key, and it is not your budget.
The app makes the call with its own credentials on the signed-in user’s
token, so the request is billed to the same daily allowance their ordinary
conversations use. You never see a key, and you can never obtain more assistant
than the user already has. When they run out, assistant.ask rejects with
daily limit reached. That is not a transient failure to retry around. Show
it and stop.
Your system string sets a persona or a topic. It reaches the model as
skill-authored text inside the user turn, explicitly fenced and labelled, not
as a system instruction. So it cannot override the assistant’s own rules, and
review reads it as carefully as it reads your code.
What microphone is not
It is not a recording. You receive what the user said, as text, and never audio.
mic.listen borrows the app’s own listening turn, which has two consequences
worth designing around. The app’s listening indicator and eye state are shown
while it runs, so you cannot listen without the user seeing it. And it only
starts when the robot is idle: ask while it is speaking or thinking and you
get null, not a queued turn.
null also comes back when the user simply said nothing. Silence is an
ordinary outcome, not an error: a tutor that treats it as a failure will be
wrong several times a session.
What open is not
It is not “open a link”. You never give a URL. You name what you want, the app builds it and opens it.
That is not a formality. Your skill has no network, and a link you chose would
be the way around it: anything you know could be put in the URL and read by
whoever opens it. So each thing you can open is added as a whole action that
the app performs on your behalf, and the app keeps what is private to the user.
open.cab('office') opens their cab app with the office set, and you never see
the address, the coordinates, or which app it was.
Opening another app takes the screen away from the user, so calls are limited
to one every few seconds; the extra ones resolve false.
The sandbox
Read this before you build. Every limit here is deliberate, and none of them are configurable.
No network at all.
A Content Security Policy is injected before your code runs:
default-src 'self' 'unsafe-inline' data: blob:; connect-src 'none';
script-src 'self' 'unsafe-inline'
connect-src 'none' means no fetch, no XHR, no WebSocket, no beacon. No
CDN scripts, no Google Fonts, no analytics, no error reporting, no API you own.
Everything ships in your folder. Inline your CSS, embed images as data URIs, bundle your own fonts.
This is the single most important thing to design around, and the most common reason a skill that worked in a browser does nothing on the device. The simulator does not enforce it. Test on a device before you publish.
The only outside information available to you is data.weather().
You cannot leave your folder
File access is scoped to your own skill directory. You cannot read another skill’s files, the app’s files, or anything else on the device.
Navigation away from your own bundle is refused: links, redirects,
window.location. Your skill is one page, for its whole life.
localStorage does not persist, hairobo.storage does
The WebView’s own storage is non-persistent: localStorage, cookies and
caches are cleared between runs, and always will be. Anything you keep there
is gone the moment your skill closes.
Use hairobo.storage instead. It survives between runs, is scoped to your
skill id, needs no permission, and is deleted when the user uninstalls you.
Full API in api.md. Its caps:
| keys | 200 |
| key length | 128 characters |
| one value | 16 KB |
| whole store | 256 KB |
Values are strings, so JSON.stringify your own objects.
Hitting a cap REJECTS the promise; it does not resolve false. That is
the opposite of the rate limits below, and deliberately so: a rate limit clears
on its own and retrying is correct, whereas a full store stays full until you
remove something. Catch it.
It is not a database. If you need more than 256 KB, this is not the API for what you are building.
Rate limits
voice.speak |
one per 0.4s, 300 characters |
motor.* |
one move per 0.35s |
| camera events | about 6 per second, up to 8 faces |
Exceeded calls resolve false and are dropped, not queued. A backlog of
stale commands is worse than a missed one, especially for movement.
Ambient skills yield
An ambient skill gives up the screen the moment the robot leaves idle, and
whenever the display deep-dims for battery. Do not depend on running
continuously.
What you can rely on
Your page is a full modern WebView. Canvas, WebGL, CSS animation, Web Audio, touch: all of it works. The limits above are about the outside world, not about what you can build on screen.
The simulator fakes the camera
The simulator has no camera. It emits synthetic face events at the same 6 Hz
the device uses, with however many people you set in the sidebar, spread across
the frame and drifting. That is enough to exercise counting and layout, and
nothing like a real room. Test multi-face on a device before you publish.
Storage in the simulator is backed by the browser’s own localStorage with the
same caps enforced, so you hit a cap there before a user hits it for you.
How the simulator runs your skill
Drop your folder on it. It reads the files straight off your machine: nothing is uploaded, nothing is downloaded, and you do not need a server. This is the way to run your own skill.
It provides the SDK, exactly as the app does. hairobo is defined before
your first script runs, so your skill needs no script tag for it. The same
code runs here and on a robot, with nothing in it that exists only for the
simulator.
A reference that is not in your folder is named in the log. A missing
stylesheet used to show up as a page that simply looked wrong; now it says which
file it wanted. The entry comes from your manifest’s entry, so a skill that
does not use index.html still runs.
The path box is for the bundled examples, which sit next to the simulator on its own server. Your own skill will not be there. Use the folder.
Publishing
You don’t host anything. Upload your folder, try it on your own robot, then submit it for review.
1. Build it in the simulator
The simulator runs your skill with the robot stubbed out: revoke permissions, fake a missing dock, watch every call. Get it working there first; it is much faster than a device.
2. Upload it as a draft
Drop your folder on hairobo.com/developers/submit.
manifest.json plus every file it lists.
The checks run immediately, before anything is stored. Blocking errors come
back as checks.errors and nothing is recorded until they are fixed:
- manifest doesn’t parse, or
idisn’t reverse-DNS, orversionisn’t semver entryisn’t a relative.htmlpath inside your folder- a file in
fileswasn’t in the upload - bundle over 5 MB, or over 60 files
- code calls
fetch,XMLHttpRequest,WebSocket,EventSource,sendBeaconorimportScripts: skills have no network access (connect-src 'none'), so these cannot work on a device - a remote
src=: the sandbox blocks it and you get a blank screen. Bundle the file and list it infiles. - you call
hairobo.motor.*without"motor"inpermissions: every call would be refused at runtime
checks.warnings don’t block. The most common one:
permission “camera” is requested but never used
Asking for a permission you don’t use is the single most common reason a submission comes back. Remove it.
3. Try it on your own robot
Open HAi Robo signed in with the account you uploaded from, then Settings → Plugins → Browse the Plugin Gallery. Your draft is at the top, marked DRAFT, above the public skills. Tap Try it.
Nobody else can see it. It is not in the gallery for anyone but you, and it will not be until a reviewer approves it.
You get the real thing: the real sandbox, the real permission prompt, the real CSP, the real motors. The only difference from a published skill is that nobody has reviewed it.
4. Iterate
Re-upload the same version as often as you like. A draft is mutable: fixing one line does not cost you a version number. Tap Reinstall in the gallery to pull the new bytes.
Freezing happens when you submit, not when you upload.
5. Submit for review
One button on the portal. From that moment the bytes are fixed: what a reviewer reads and what eventually ships are exactly what you submitted, and further uploads to that version are refused.
The checks run again here, against what is stored rather than what you last uploaded, so the report a reviewer sees describes the code a reviewer reads.
6. Review
A person reads the source, INCLUDING any system prompt you pass to
assistant.ask. A prompt is behaviour, and changing one is a new version, not
a tweak. See CONTENT-POLICY.md. They run it, and check the permissions match what it
does. Not a code review: a check that the skill is what it says it is and isn’t
hostile. Expect a few days.
If it comes back, review_note says why. Fix it, upload a new version, and
submit again. A rejected version stays rejected.
Approved skills appear in the gallery for everyone.
Updating
Upload a new version and submit it. Each version is reviewed on its own, and
your live version keeps serving throughout. A rejected update never takes down
the version already approved.
Adding a permission means the user is asked again, and the new permission stays ungranted until they agree; the rest keep working meanwhile. Removing one needs nothing.
Version numbers
- A draft can be re-uploaded any number of times.
- Once submitted, that version is closed. To change anything, bump the version.
- A rejected version can never be resubmitted. Bump and upload again.
What gets removed
Set out fully in CONTENT-POLICY.md. In short: skills that request permissions they do not use, misrepresent what they do, target or harass anyone, or try to get around the sandbox rather than work within it.
Users can report a skill from the gallery. Reports are read by a person.
A skill can be pulled two ways, and both are used together when something is actually wrong: suspending it hides it from the gallery, which stops new installs; the blocklist stops the copies already on people’s devices.
Being useful, not just working
The skills people keep are the ones that respect where they are: a device on a desk that is mostly idle. Quiet by default, obvious at a glance, and correct about whether a dock is even connected.
info.get() tells you what you actually have. Use it before offering a robot
feature you cannot deliver.
Skill content policy
Skills run on a device in someone’s home, with access to a voice, a camera and motors. That is the standard everything here is measured against.
Prompts are behaviour, and they are reviewed as behaviour
A skill holding the assistant permission ships a system prompt. That string
decides what the robot says on someone’s behalf, so it is part of what your
skill does. It is not configuration, and not a detail of how it is built.
Three consequences, stated because a reviewer cannot infer them from code:
- A prompt is reviewed line by line, like source. It is prose, so it reads as harmless at a glance; it is also the one place where a skill’s behaviour can be changed completely without touching a line of JavaScript.
- Changing a prompt is a new version. Shipping a different
systemstring under the same version is misrepresenting what the skill does, which is already forbidden below. - What the assistant says is what your skill said. You do not get to disown a reply because a model produced it. Every rule in this document about what the robot may say applies to text you caused it to say.
Your prompt must be readable in your source
Write the system string as a literal in your source: one string, or
adjacent strings joined at the point of use. Do not assemble it at runtime from
fragments, decode it, or fetch it from your own storage.
This is a review requirement, not a style preference. A prompt that only exists once the skill is running cannot be read by the person deciding whether to publish it, and “trust me, it says what I claim” is not a review. Submissions whose prompt cannot be read in the source are rejected, and the remedy is always the same: write it out.
Interpolating the user’s own words into a prompt you otherwise wrote as a literal is fine. That is the skill doing its job. Building the instruction itself out of pieces is not.
Not allowed in a prompt
- Impersonation. Instructing the assistant to speak as a named person, as a company or brand, or as HAi Robo itself in a way that implies official standing. A tutor with a character name is fine; a prompt claiming to be someone is not.
- Elicitation. Steering the conversation toward personal information:
address, workplace, school, financial details, who else is in the house, when
the house is empty. You have no network and nowhere legitimate to put such
answers, and once
microphoneis granted there is no second consent prompt per question. A skill that asks is out, whatever it intends to do with it. - Laundering. Using the assistant to produce what this policy already
forbids the robot saying aloud. The prompt is a route around every rule
written about skill output, and it is closed: reaching a banned outcome
through the model is the same violation as writing it into
voice.speak. - Instructing the assistant to break its own rules. Telling it to ignore its instructions, to reveal them, or to behave as though it has none. The host wraps your prompt and states plainly that it cannot override safety rules; attempting it anyway is grounds for removal, not a failed attempt.
What a good prompt looks like
Say what role the assistant takes, what it should do with what it hears, and what shape the reply takes. Nothing about the user beyond the task.
Not allowed
- Requesting permissions you do not use. The clearest signal of bad intent, and the easiest to check.
- Misrepresenting what the skill does, in the name, summary or at runtime.
- Impersonating a person, brand or organisation you are not.
- Targeting or harassing anyone, including through what the robot is made to say.
- Sexual content, graphic violence, or content aimed at children that adults would not want a robot saying aloud.
- Working around the sandbox rather than within it: attempts to reach the network, escape the skill folder, or navigate away.
- Collecting personal information. You have no network; a skill that asks for personal details has nowhere legitimate to put them.
- Using the camera for anything but what you told the user. You receive positions, not images, and the permission is still granted on the basis of your description.
Expected
- The summary describes what the skill actually does.
- Permissions match the features.
- It degrades honestly with no dock and with permissions declined.
- It stops what it started (lights, gaze, camera) when it closes.
Reports and removal
Users can report a skill from the gallery. Reports are read by a person.
A skill found in breach is removed from the gallery and can be blocked remotely on devices that already have it. Blocking is used for skills that are harmful or deceptive, not for ones that are merely poor.
If yours is removed and you think that is wrong, reply to the submission address and say so. There is no automated appeal because there is no automated removal.