# Testing and debugging

> Run a plugin in the web editor, read its console and events, test it on the phone, and write automatic tests with the command-line runner.

URL: https://wmkeyboard.pages.dev/plugins/guides/testing/
Section: Plugins

There are three places to run a plugin while you build it: the web editor,
the editor on the phone, and a command-line runner for automatic tests. All
three run the same interpreter the keyboard uses.

## The web editor

The [web editor](https://wmkeyboard.pages.dev/addons/new?type=plugin) runs your plugin in the browser,
next to the code. With **Run as you type** on, it runs again each time you
stop typing. The tabs under the code show what is going on:

| Tab | Shows |
|---|---|
| **Console** | What `print()` writes, errors with their line numbers, and a line for each thing the plugin did that a browser cannot really do |
| **Problems** | Mistakes the checker finds before you run: a misspelt name, a name the `wm` package does not have, Python that plugins do not support |
| **Outline** | Your functions, to jump between them |
| **Events** | Every button, switch, box and tab in the panel, with a button to send each one an event. **Any event** sends one you write yourself, such as a `keyword` event |
| **Storage** | What the plugin saved, which you can edit or delete |
| **Try** | Calls one of your providers or selection actions the way the keyboard would, and shows the reply |
| **Details** | The name, permissions and hosts, providers, selection actions and the settings page, the whole `plugin.json`, and the file's size |

### What the editor knows about your plugin

The editor reads your `plugin.json` as well as your code, so its suggestions
and checks are about this plugin and this keyboard, not Python in general.

- **Suggestions.** `@` at the start of a line offers the decorators
  (`@plugin.action`, `@plugin.provider`, `@plugin.keyword` and the rest).
  Inside them, the strings come from your manifest: the provider lists, your
  provider and action ids, your keyword and your setting keys. A widget or
  `http` call offers the arguments it takes and, where it takes only a few
  values, those values (`style=`, `color=`, `kind=`, icon names). `on_click=`
  offers your own functions. `http.get("` offers your hosts. Inside a
  provider, `request["` offers the keys that list sends.
- **Values the package returns.** After `r = http.get(...)`, `r.` offers
  `status_code`, `json()` and the rest; after `state = plugin.state(count=0)`,
  `state.` offers `count`. `plugin.settings.` offers your settings.
- **Help while you type.** Inside a call, the strip under the code shows its
  parameters with the one you are typing marked, and what that one takes.
  Elsewhere it explains the name at the caret. On the web, resting the mouse
  on a name shows the same.
- **Checks.** Beyond Python mistakes, the editor flags a module used without
  its permission, an address on a host `plugin.json` does not list, a provider
  or action the manifest does not declare (or one it declares that nothing
  answers), a keyword or setting that does not match, and an argument or value
  a widget does not take.
- **Fixes.** Most of those checks come with a fix. Put the caret on the
  problem, or open **Problems**, and press the button: it adds the
  permission, host, provider, action or setting to `plugin.json`, corrects a
  typo, adds the missing import, or writes a stub for a provider that has
  none.

### Trying a provider or an action

**Try** lists the providers and selection actions your `plugin.json`
declares. Pick one and press **Try it**. A provider gets the request its list
sends, filled in with an example you can edit; an action gets a line of
text. The pane shows the reply, what `print()` wrote, how long the call took,
and, for a provider, whether the list can read the reply and how many results
it finds in it. See [Answering for a provider list](https://wmkeyboard.pages.dev/plugins/guides/providers/).

### What the preview can and cannot do

The preview treats every permission your manifest asks for as allowed. For the
rest, it does what a browser can:

- `http` sends real requests to your `hosts`, if the website lets other pages
  read its answers. Many APIs do.
- `field` writes to a pretend text field, and the console shows what it holds.
- `ai`, `location`, `files`, `sensors` and `scan` get pretend answers of the
  right shape. [Phone features](https://wmkeyboard.pages.dev/plugins/guides/phone-features/#in-the-web-editor)
  lists each one.
- Translations do not apply: the preview shows your English.

## On the phone

The editor on the phone works the same way, with a preview beside the code,
the same suggestions, checks, fixes and **Try** tab, and the strip under the
code in place of a mouse hover. It runs inside the real keyboard, so it is the
place to check how the panel looks in your theme and how the field calls
behave. Its preview and **Try** reach your hosts and the AI as an installed
plugin would. Providers, selection actions and the settings page are in
**Details** in its menu.

Tools / Plugins / Write a plugin

**Install the plugin** puts the current version in the keyboard at once. Each
installed plugin also keeps a log of what it printed, on its page:

Tools / Plugins / Manage plugins

## Automatic tests

`wm-python-cli` runs a plugin with no phone and no browser. It is for tests
you run again and again, and for checking plugins in an addon repository's CI.

1. Install [Rust](https://rustup.rs/).

2. Get the WM Keyboard source: `git clone https://github.com/wasi-master/wmkeyboard`

3. Build the runner: `cd wmkeyboard/native/wm-python-cli && cargo build --release`

4. The program is `target/release/wm-python-cli`.

### Run once

```bash
wm-python-cli run my-plugin/ --mock mock.json
```

This loads the plugin in `my-plugin/` and prints the first draw as JSON. The
mock file answers host calls by name:

```json title="mock.json"
{
  "http.request": {"status": 200, "text": "{\"temp\": 21}", "headers": {}},
  "ai.ask": {"text": "A pretend reply"}
}
```

A call with no answer in the file raises `NotImplementedError`. An answer that
is only an error raises that error, so you can test the unhappy paths:

```json
{"http.request": {"error": {"kind": "TimeoutError", "message": "no answer in 15 s"}}}
```

### Write tests

Put a `test.py` beside `main.py` and run `wm-python-cli test my-plugin/`. A
test fails by raising, so plain `assert` works:

```python title="test.py"

mock = {"http.request": {"status": 200, "text": json.dumps({"temp": 21}), "headers": {}}}

s = plugin_test.session(mock=mock, settings={"units": "metric"})
r = s["load"]()
assert r["ok"], r

r = s["event"]({"type": "click", "id": "button:Refresh"})
assert "21" in json.dumps(r["ui"]), r["ui"]
```

`plugin_test.session(mock=, settings=)` gives you the plugin as the keyboard
runs it:

| | Does |
|---|---|
| `load()` | runs the file and draws the panel |
| `render()` | draws it again |
| `event(e)` | sends one event, then draws |
| `transform(text, id)` | runs a selection action |
| `storage()` | what the plugin saved, as a dict |
| `settle()` | answers every waiting `then=` call from the mock, then any calls those answers started, and gives back the last result, or the first one that failed |
| `calls()` | every host call so far, as `{"op", "args"}` dicts, to check what the plugin sent |

Each gives back a dict: `ok`, the drawn `ui`, any `error`, and the `host`
block (timers, header buttons, how many `then=` answers are waiting).

Events look like this. A widget's id is its type and text, such as
`"button:Look up"` or `"input:Word"`, unless you passed `key=`:

```python
{"type": "click", "id": "button:Look up"}
{"type": "change", "id": "input:Word", "value": "rain"}
{"type": "submit", "id": "input:Word", "value": "rain"}
{"type": "keyword", "id": "define", "value": "rain"}
{"type": "command", "id": "clear_history"}
```

A call made with `then=` waits until you answer it with a `result` event. Its
token is the module's name and a number that counts every such call (and
every `plugin.after`) in the session: the first look-up is `"http:1"`, and an
`ai.ask` after it would be `"ai:2"`.

```python
s["event"]({"type": "click", "id": "button:Look up"})
r = s["event"]({"type": "result", "token": "http:1", "value": mock["http.request"]})
```

`settle()` saves you counting tokens. It answers each waiting call the way the
mock would answer it, so a click that starts a look-up, whose answer starts a
second request, is one `event` and one `settle`:

```python
s["event"]({"type": "click", "id": "button:Look up"})
r = s["settle"]()
```

`ui.toast` and `ui.dialog` are calls to the keyboard too. A plugin that shows a
toast from a callback needs an answer for it in the mock, such as
`"ui.toast": None`, or the callback stops with `NotImplementedError`.

### A different answer for each address

A plugin that talks to one service often calls several of its addresses. Give
an op a list of routes instead of one answer. The first route whose `when`
fits the call answers it:

```python
mock = {
    "http.request": [
        {"when": {"method": "GET", "url": "https://example.com/api/stock"},
         "reply": {"status": 200, "text": "[]"}},
        {"when": {"method": "POST", "url": "https://example.com/api/*"},
         "reply": {"status": 200, "text": "{}"}},
    ],
    "secrets.get": [{"when": {"name": "api_key"}, "reply": "test-key"}],
}
```

`when` is checked against the call's arguments. Text must be equal, or start
the same when the pattern ends in `*`. A dict such as `"params": {"q": "milk"}`
fits when each of its keys does. A route with no `when` fits every call, and a
call that no route fits raises `NotImplementedError` naming the address.

To check what was sent, look through `calls()`. For a request, `args` holds
`method`, `url`, `params`, `headers` and the `body` as text:

```python
posts = [c["args"] for c in s["calls"]() if c["op"] == "http.request" and c["args"]["method"] == "POST"]
assert json.loads(posts[0]["body"]) == {"amount": 2}
```

`plugin_test.read(name)` reads a file from the plugin's folder, which is
handy for a long mock reply kept in its own file. Files under `tests/` stay
out of the plugin's archive, so that is a good place for them.

An API 5 plugin's other `.py` files are modules `test.py` can import, to
test them on their own without a session:

```python

assert fares.total([2.9, 1.0]) == 3.9
```

## Reading an error

When your code raises an error nothing catches, the run stops and the console
or the panel shows the error and its line. The [Help](https://wmkeyboard.pages.dev/plugins/help/#error-messages)
page explains the common ones.

Two kinds of stop come from the keyboard, not from Python:

- **Too much work.** Each part of a run has a budget of steps: 9 million to
  load, 6 million for each event, 1.2 million to draw. A step is about one
  line or one turn of a loop. A plugin that runs out is stopped. Look for a
  loop that never ends, or work that belongs in a smaller piece.
- **Too slow.** Loading has 3 seconds, an event 2, a draw half a second. Time
  spent waiting for `http` or `ai` does not count.

A plugin that is stopped twice is switched off until the user turns it on
again, so fix these before you share it.
