Live view and control
Beside the tiles, which ask for a picture about once a second, qKVM offers one machine live: one connection per viewer that carries the machine's picture down as it comes and a keyboard and a mouse up, key by key. A program that speaks it (the page's enlarged view; the qKVM desktop program) opens it in two steps: it asks POST /api/v1/devices/{id}/live for a pass, a call like every other, signed in and with the X-QKVM: 1 header, and gets {"v": 2, "ticket": "..."}, a pass good for half a minute and for one line; then it opens a WebSocket at the same address, which carries no cookie, and shows the pass in its first message. GET /api/v1/wall says "live": 2 of a qKVM that has this; a program that speaks another form of it asks for none. A line whose first message is not a pass this qKVM gave for that machine is closed with a word (no_pass), and so is one whose pass was used before or has run out.
In the page. Enlarging a tile opens the machine's line, where it has one: the stills give way once the first live picture is drawn, and come back when the line ends. Control (or the key C) takes its keyboard and mouse; the right Ctrl by itself gives them back. Where somebody else has them, the button says who (Controlled by olga); to an admin it offers Take over from olga, which asks first, naming them. Whoever loses them is told why at the edge of the page: admin took over this machine's keyboard and mouse, or Given back after 15 minutes without input.
In the enlarged view, Control is a toggle: highlighted while you hold the keyboard and mouse. It sits beside Actions in a header that matches the home page height. Turning Control on opens a compact input row beneath the header, with keyboard, shortcuts and relative mouse. Click Control again, or press Right Ctrl by itself, to release control and close that row. The video stays in place. A thin bottom status strip always shows stream information and control state. Keyboard opens a floating panel near the bottom. Drag its title bar to move it, or drag its bottom-right handle to resize it. Both handles also accept arrow keys when focused. Reset position returns it to the bottom; closing it leaves Control active. The panel stays inside the browser when its size changes. It provides letters, numbers, function and navigation keys for mouse or touch input. Shift, Ctrl, Alt and Win apply to the next key only; they are not held on the remote machine between clicks. Type text briefly releases this page's live control after confirmation, waits for acknowledgement, types through the normal action channel, then requests control again. A missing acknowledgement or another session holding control blocks the text. Closing the keyboard, losing page focus or releasing control clears those modifiers. The labels use US key positions; the remote machine's keyboard layout determines the resulting text. Details in the status strip opens device information, stream statistics, history graphs and permitted recent activity together.
Who may. Whoever may see the machine may watch it: a viewer, a wall screen opened with its display link, a token for scripts. Taking its keyboard and mouse takes an operator or an admin, in person or by a token with that role. One connection holds a machine's keyboard and mouse at a time: a second one that asks is refused (taken, with by, the name of whoever has them), and so is an action from the page while somebody holds them (409 busy, which says who as well), and the other way round.
Taking over. An admin, in person, may take a machine's keyboard and mouse from whoever holds them: another admin, an operator, a script's token. Never an operator, never a token (no token is an admin), never a wall screen (which only looks). Whoever held them loses them as a line loses them for any reason: everything they had down is let go of, and nothing more of theirs goes out (what was still waiting on its way goes nowhere, and a word already sent is let go of again before the admin's first word), and they are told who took them (control, on: false, taken, with by). Their line goes on showing the machine. An admin's action from the wall (a power button, keys, text) does not take over by itself: it is refused with the holder's name, and the answer says that an admin can take them over in the live view, where it is done on purpose, after a question.
Given back by themselves. Keyboard and mouse are given back after 15 minutes without input from whoever holds them (no key, no button, no turn of the wheel, no move of the pointer): what they had down is let go of, they are told why (idle), and the line goes on showing the machine. Control takes them again.
One hand at a time. Whoever gets keyboard and mouse next (after a take-over, a giving back, or a line that lost them) sends nothing to the machine until whatever the one before had down has been let go of, and whatever of theirs was still on its way is through (what they gave back goes, while they may still act; what was taken from them goes nowhere) and what it put down is let go of again, for 16 seconds at the most: a key of the new holder's never lands beside a Ctrl the one before still had down. An action from the page, and switching a PC's Intel AMT KVM, wait for the same; one whose tile is changed or taken off the wall meanwhile sends nothing, and says so (409 conflict). A tile changed or taken off the wall waits for that letting go before its connection is closed.
A line ends when the sign-in it was opened from ends (signing out, being signed out everywhere, an account removed, suspended or limited to other machines, a token taken back) and when the machine is edited, taken off the wall or the wall locks itself; what the line had down on the machine is let go of then, as it is when a line is cut or given back. What is still waiting on its way at that moment (the reports queued for a JetKVM, a key waiting up to ten seconds for a PC's next AMT session or for a Proxmox console) goes nowhere, a word already sent is let go of again as soon as it lands, and nothing more of that line's goes to the machine. A line closed or given back by whoever held it lets what it said before go on, for as long as they may still act on the machine. qKVM asks a line every five seconds whether it is there (a WebSocket ping, which a browser and every library answer by themselves); one that has answered none of three is gone, and so is one that takes nothing off the line for ten seconds, or is not reading what it is told, or sends pings by the dozen a second, more than two thousand other frames a second, or a message in more than sixteen pieces: what it had down is let go of either way. One account (a token, a wall screen) has 32 lines open at the most, the passes it was given and has not used counted with them. A pass is redeemed from the address it was given to. While a line holds a machine's keyboard and mouse, an action from the page for that machine is refused (409 busy), for whoever holds them too: text is typed over the line, or after the keyboard is given back.
What is logged. The activity log says who opened a machine's live view, and when it ended and after how long (live_opened, live_closed with seconds and why: closed where its viewer ended it, else the reason, such as gone); who took a machine's keyboard and mouse, and when they were given back or taken away, and why: a take-over as two lines, the holder's (why "taken", by the admin) and the admin's (from the holder); a giving back for want of input with why "idle" and minutes. Never a key, never where the pointer was. An account's (a token's, a wall screen's) live views write 120 entries an hour into the log at the most: the next one after that says how many were left out (unsaid), so that a script opening lines in a loop cannot push out of the log what was done before (see Activity log).
What each kind of machine gives:
| Machine | Picture | Keyboard and mouse |
|---|---|---|
A KVM with the kvmd interface that has its H.264 line (/api/media/ws: stock PiKVM, and a GL.iNet Comet on a recent GLKVM firmware, tried on a real one) | its H.264 stream, passed on as it comes, one connection to the KVM shared by every viewer that can show it; every key frame carries the stream's parameter sets (a KVM that sent them once only has them put back before each key frame); a stream whose parameter sets say a picture larger than qKVM shows any (40 million pixels, 16384 a side) or cannot be read is not passed on, and its viewers get the pictures one by one instead | yes, on its line for events |
A KVM with the kvmd interface without that line, and a viewer that shows JPEG alone | JPEG pictures asked one after the other, as fast as the fastest viewer asks and never more than the KVM gives (30 a second at most), only those that changed | yes; the pointer by steps where the KVM's mouse is set to relative (see below) |
| JetKVM | JPEG made of the frames qKVM decodes anyway; its H.264 is not passed on (see Limits) | yes |
| A server whose controller is AMI MegaRAC (see Servers) | JPEG of its screen, each time the screen changes | yes, as the console's own USB keyboard and mouse reports; the pointer goes as an absolute place where the server's mouse is absolute (what these boards are by default), the back and forward buttons have no bit there |
| A server whose controller runs OpenBMC (see Servers) | JPEG of its screen, each time the screen changes | no: qKVM only looks at an OpenBMC console |
| A PC with Intel AMT whose KVM qKVM may use (see Intel AMT) | JPEG of its screen, each time the screen changes, at the pace the viewer asks for | yes, as the KVM's own key and pointer events; the back and forward buttons have none there |
| A virtual machine on Proxmox VE, while it runs (see Proxmox and virtual machines) | JPEG of its screen, each time the screen changes, at the pace the viewer asks for; the tile's console and the line's are one | yes, as RFB's key and pointer events through its console; the back and forward buttons have none there |
| A VNC server | none in this version (live is not in its can): the tile's pictures | no |
The pointer. A KVM with the kvmd interface moves its computer's pointer the way its mouse is set on the KVM (its own settings): to a place (absolute, stock kvmd's default) or by steps (relative), and drops a move of the other kind; a real Comet set to relative never moved its pointer to a place the page sent. The line says which (pointer in its state message), and the page follows it: where the mouse moves by steps only, the bar says so and a click on the picture hands the pointer to the machine (the browser's pointer lock; Esc gives it back), as the KVM's own page does it, and that click is no click on the machine; where it takes places only, Relative mouse is not offered. A JetKVM, a PC with Intel AMT and a virtual machine take both. With the pointer handed over, the computer moves it by its own settings (Windows speeds a fast move up), so it goes where the computer puts it, which the picture shows.
The messages of the line, for whoever writes a program that speaks it. Text messages are JSON with a field t. From the program: hello (v: 2, ticket, video: the codecs it can show, of h264 and jpeg, max_w, max_fps), shape (max_w, max_fps), key_frame (a frame was lost: send a key frame; of JPEG the last picture is sent again, once in half a second at the most), control (on; and to take them over from whoever holds them, as an admin in person may, take: true, with from: the opaque from reference of the refusal that this live line received, after confirmation naming its by display name. The reference identifies one grant of control, not an account name: another session or a release and reacquisition invalidates it, even for the same account. A stale or missing confirmation is refused taken with the current by and from; ask the person again, never retry takeover automatically. Older clients may send the observed by name only while that same live line's observed grant remains unchanged. These checks do not grant permission: only a currently authorized administrator may take over, and permission and the grant are checked again after the device connection opens. Where nobody holds control it is an ordinary asking; sixty takings and givings back a minute of one account, on all its lines together, and a line that asks more ends with bye flooding), k (u: a key by its USB HID usage on page 7, 4 to 0xE7, but 0x66, the keyboard's Power key, which is dropped unsent: a machine's power is under Power, asked first; d: down), m (x, y: the pointer, 0 to 32767 over the picture), mr (dx, dy), mb (b: a button 1 to 5, left, right, middle, back, forward; d), mw (dx, dy: turns of the wheel, up and right positive, 16 at the most in one message), up (let go of everything). From the server: video (codec, w, h: what follows; sent before the first picture and whenever it changes, and a key frame follows it), state (as in /api/v1/status, when it changes; with pointer, how the machine's pointer is moved, where that is known: absolute, to a place with m; relative, by steps with mr; both), control (on; and when refused code and message, with by, the name of who has them, where somebody has: one answer to each asking, in order, within a minute (the way in to a machine can take that long: a JetKVM connected anew some forty-seven seconds; not made by then, timeout), so wait a little longer than that before giving up on it, and on: false with a code by itself when keyboard and mouse were taken away: taken with by (an admin took them over), idle (15 minutes without input), not_allowed, dropped, in_use), bye (code, message: why the line is about to close). Pictures are binary messages: 16 bytes of header (byte 0 the kind, 1 for one H.264 access unit in Annex B form, 2 for one JPEG; byte 1 the flags, bit 0 a key frame; bytes 2 and 3 zero; bytes 4 to 7 a counter that goes up by one for each picture the machine gave, little-endian; bytes 8 to 15 when qKVM got it, in microseconds of a clock that only goes forward), then the picture. A viewer that falls behind is not waited for: it is brought forward to the next key frame, and sees that as a jump in the counter. Keys and the mouse are taken only from the connection that holds them, and only after the server has said so.