Categories
Hosting, Mining Infrastructure

Remote hands is the hosting service clients use every week and the one least often defined in the contract. What it covers, how to write a request that gets acted on, and the seven questions to ask before signing.

Every hosting agreement describes power, space, and cooling. Far fewer describe the thing clients actually consume day to day, which is a person on site who will walk to a specific machine and do something to it on your behalf.

In colocation this is called remote hands, and in mining it is the difference between a dead unit that is fixed this afternoon and a dead unit that is fixed whenever somebody happens to notice. It is rarely defined properly in a hosting contract, which means expectations on both sides are set by habit rather than agreement. That works until the relationship is under strain, and then it does not.

This is what the service actually covers, how to ask for it in a way that gets acted on, and which parts of it you should pin down before you sign.

What remote hands means when the hardware is miners

In a traditional data center, remote hands covers tasks a customer would do themselves if they were standing in the rack: press a button, swap a cable, read what is on a screen, replace a part the customer shipped in.

Mining inherits that definition and adds its own vocabulary. The routine requests at a mining site are a narrow and predictable set. Power cycle a unit that has stopped hashing. Reseat a hashboard or a data cable. Check whether a fan is actually spinning. Read an error code from the controller. Confirm that a machine which vanished from the dashboard is physically present and powered. Swap a power supply from your own spares. Move a unit to a different position because one part of a room runs hotter than another.

What distinguishes mining from classic colocation is volume and repetition. One rack of servers might generate a handful of hands requests a year. A few hundred miners generate them continuously, because the failure modes are thermal and mechanical and they recur. Any arrangement that treats each request as an individual exception will collapse under that volume.

The three classes, and why the distinction is the whole negotiation

Almost every disagreement about remote hands comes from both sides using the phrase for three different things.

Routine intervention is the work a site does as part of operating the facility. Noticing that a machine has dropped off, power cycling it, reseating a cable, cleaning filters on a schedule. Most managed hosting includes this, and a site that charges per reboot is usually describing a bare metal arrangement rather than a managed one. The distinction between colocation contract models is exactly where this is decided.

Client-directed work is anything the site would not have done on its own initiative. Reconfiguring a pool, flashing firmware across a batch, relocating specific units, pulling a sample for inspection, fitting parts you supplied. This is normally billable at an hourly rate, and that rate belongs in the agreement rather than in an invoice you see for the first time afterwards.

Project work is anything that needs planning, several technicians, or a window: a fleet-wide firmware rollout, a large relocation, a full physical audit. It is quoted, not ticketed.

Get those three named in your agreement with an hourly rate attached to the second and third. It is a five minute conversation at signing and it removes the single most common billing argument in hosting.

How to write a request that gets acted on

A technician standing in a loud room with hearing protection on cannot act on “a few of my miners look down.” The request has to survive being read by somebody who cannot see your dashboard.

A request worth sending contains the serial or the site’s own asset tag, the physical location if you know it, the symptom as observed rather than diagnosed, what you have already tried remotely, and what outcome you want. “Unit SN-xxxxx, rack 14 position 6, offline in the dashboard since 02:10, no response to remote reboot, please confirm whether it is powered and reseat the data cable” is actionable. It is also auditable later, which matters when you are reconciling what was done against what was billed.

Identify machines by serial, not by worker name. Worker names get reused, especially after a replacement, and a worker name points at a slot in your pool configuration rather than at an object in a rack. This is another case where a maintained serial-level asset register pays for itself: it lets you name the exact machine, and it lets you verify afterwards that the machine named is the machine touched.

Response targets, and what they really promise

Response time and resolution time are different commitments and they are routinely conflated. A response target says somebody will acknowledge the ticket. A resolution target says the work will be done. Most hosting agreements that mention timing at all are committing to the first.

Ask how the clock behaves in practice. Does it run overnight and at weekends, or only during staffed hours? A four hour response is a very different product if the site has a technician on shift around the clock than if it has one on call who drives in. Ask how many technicians are on site during each shift, because that number, more than any written target, determines what happens when forty units drop at once during a heat event.

Ask what gets prioritized when requests exceed capacity. A sensible site will tell you honestly: anything that threatens the facility first, then anything affecting many machines, then individual units. That ordering is correct, and a client who understands it in advance is a great deal calmer when their single dead miner waits.

Parts, and who is expected to hold them

Hands work stops at the point where a part is needed. Decide in advance who supplies spares, because the answer determines how long a dead unit stays dead.

Three arrangements are common. You ship a buffer of spare power supplies, fans, and hashboards to the site to be held in your name. The site holds its own pooled stock and bills you for consumables. Or nothing is held and every failure waits on a shipment. The third is the cheapest arrangement and by a wide margin the most expensive outcome, because downtime on a modern unit costs more per week than the part does.

If you are leaving spares on site, that stock is inventory you own sitting in somebody else’s building. Count it the same way you count the machines, and make sure it appears in whatever reconciliation happens when the agreement eventually ends.

What you should get back

The output of hands work is not only the fix. It is the record. For each request you want to know what was found, what was done, which parts were consumed, how long it took, and whether the unit returned to service.

That record is what lets you distinguish a fleet with a hundred unrelated faults from a fleet with one underlying problem. Thirty reseats in a month across random positions is maintenance. Thirty reseats in one aisle is an environmental issue in that aisle, and no amount of further reseating will fix it. Without per-request records that pattern is invisible, and the usual response is to blame the hardware.

Those same records are the basis for any conversation about whether the environment is meeting the terms you are paying for, which is ultimately a question about temperature, airflow, and power quality rather than about technicians.

The questions worth asking before you sign

How many technicians are on site per shift, and is the site staffed overnight and at weekends? What is included as routine intervention and what is billable? What is the hourly rate for client-directed work, and is there a minimum increment? What is the response target, and does the clock run outside business hours? How are requests prioritized during an incident affecting many machines? Who holds spare parts, and if I ship spares, how are they tracked? What record do I receive for each request?

Those seven questions take ten minutes and they describe the service you will actually use every week. The power price gets all the attention during negotiation, and it should, but the power price is what you pay. Remote hands is what you receive.

Explore Rax Mining

Categories