feat(bot): rpc_pool (role/latency/height-aware pool, WS newHeads) + executor_abi (WP 7914, 7912) #15

Merged
buddysan merged 2 commits from perf/rpc-pool-executor-abi into main 2026-09-16 14:25:46 +00:00
Owner

Summary

Phase 4 lane, first PR (WP 7914 + the executor_abi implementation for 7912).

bot/rpc_pool.py — role-based endpoint pool + newHeads subscriber, pure stdlib.

  • File schema via rpc_probe.parse_pool / load_pool(require_measured=True) (no duplicated schema; unmeasured placeholders refused). Pool semantics on top: read/subscribe/submit roles, local/public class, batch caps (public 8 / local unbounded, §4.5), local-class endpoints can never carry submit, and two submit endpoints that are the same host (hostname or pinned resolve IP) are refused at startup (learnings §4.3.1). Shared resolved IPs are only warned about: every *.testnet.arc.io mirror resolves to the same Cloudflare anycast IP (verified 104.18.20.97 on LXC 140), so IP identity behind a CDN proves nothing — the CEO brief's "same resolved IP" rule would have refused the merged devops pool.
  • resolve = "<ip>" honoured with SNI/Host preserved (curl --resolve semantics) — LXC 139 works from LXC 140 through it (ws and https).
  • Keep-alive HTTP/1.1 per (endpoint, thread), TCP_NODELAY, User-Agent: arc-mev-bot/0.1.0, EWMA latency ranking, consecutive-failure cooldown; -32005 is a degraded-endpoint state (immediate cooldown + failover), never an exception that reaches the main loop.
  • Height-aware selection: read/subscribe candidates more than max_lag_blocks (2) behind the best observed height are demoted; heights come from every WS head, poll, eth_blockNumber answer and a periodic probe → arcmev_block_height{endpoint}; arcmev_rpc_latency_seconds / arcmev_rpc_errors_total fed; events via bot.log.
  • Minimal RFC 6455 client (handshake/accept check, masking, fragmentation, ping/pong, close) + HeadSubscriber: fan-in over every subscribe endpoint, first-arrival-wins, 250 ms poller only while WS is silent.

Measured on LXC 140, 301 consecutive blocks (0 missed), box at load 9–13 / 4 vCPU: arc-public WS first 298/301; local-node WS +118 ms p50 / +229 p90; LXC 139 WSS +98 ms p50; 250 ms polling +265 ms p50 / +428 p90. (python3 -m bot.rpc_pool --measure-heads 300; full write-up lands in docs/execution.md with the submit PR.)

bot/executor_abi.pyexecute(Hop[],amountIn,minProfit) per architecture §3 (0x7328892b), strict decoder, uint256 profit decoder, hops_from_path integer slippage. Round-trips byte-for-byte against cast calldata / cast abi-encode fixtures from Foundry 1.8.1 on LXC 140 (0–4 hops, uint128/uint256 extremes). contracts/README.md on main is still the stub; PR #9's ABI is unchanged from §3 as far as its README states — will re-check before the submit PR.

Test plan

  • make lint (secret scan clean) and make test green locally (Python 3.13) and on LXC 140 (Python 3.12.3)
  • fake JSON-RPC HTTP node + fake RFC 6455 server, all offline: UA header, keep-alive, failover/cooldown, rate-limit degrade, batch caps/chunking, semantic errors not failed over, same-host refusal, unmeasured refusal, lag demotion + height probe, WS handshake/fragmentation/ping, poll fallback and hand-back
  • head-arrival measurement above run against the real local node + public WSS + LXC 139 WSS

🤖 Generated with Claude Code

## Summary Phase 4 lane, first PR (WP 7914 + the `executor_abi` implementation for 7912). **`bot/rpc_pool.py`** — role-based endpoint pool + `newHeads` subscriber, pure stdlib. - File schema via `rpc_probe.parse_pool` / `load_pool(require_measured=True)` (no duplicated schema; unmeasured placeholders refused). Pool semantics on top: `read`/`subscribe`/`submit` roles, local/public class, batch caps (public 8 / local unbounded, §4.5), **local-class endpoints can never carry `submit`**, and **two `submit` endpoints that are the same host (hostname or pinned `resolve` IP) are refused at startup** (learnings §4.3.1). Shared *resolved* IPs are only warned about: every `*.testnet.arc.io` mirror resolves to the same Cloudflare anycast IP (verified 104.18.20.97 on LXC 140), so IP identity behind a CDN proves nothing — the CEO brief's "same resolved IP" rule would have refused the merged devops pool. - `resolve = "<ip>"` honoured with SNI/Host preserved (curl `--resolve` semantics) — LXC 139 works from LXC 140 through it (ws and https). - Keep-alive HTTP/1.1 per (endpoint, thread), TCP_NODELAY, `User-Agent: arc-mev-bot/0.1.0`, EWMA latency ranking, consecutive-failure cooldown; **`-32005` is a degraded-endpoint state** (immediate cooldown + failover), never an exception that reaches the main loop. - **Height-aware selection**: read/subscribe candidates more than `max_lag_blocks` (2) behind the best observed height are demoted; heights come from every WS head, poll, `eth_blockNumber` answer and a periodic probe → `arcmev_block_height{endpoint}`; `arcmev_rpc_latency_seconds` / `arcmev_rpc_errors_total` fed; events via `bot.log`. - Minimal RFC 6455 client (handshake/accept check, masking, fragmentation, ping/pong, close) + `HeadSubscriber`: fan-in over every subscribe endpoint, first-arrival-wins, 250 ms poller only while WS is silent. **Measured on LXC 140, 301 consecutive blocks (0 missed), box at load 9–13 / 4 vCPU:** `arc-public` WS first **298/301**; `local-node` WS **+118 ms p50 / +229 p90**; LXC 139 WSS +98 ms p50; 250 ms polling **+265 ms p50 / +428 p90**. (`python3 -m bot.rpc_pool --measure-heads 300`; full write-up lands in `docs/execution.md` with the submit PR.) **`bot/executor_abi.py`** — `execute(Hop[],amountIn,minProfit)` per architecture §3 (`0x7328892b`), strict decoder, `uint256 profit` decoder, `hops_from_path` integer slippage. Round-trips **byte-for-byte against `cast calldata` / `cast abi-encode` fixtures from Foundry 1.8.1 on LXC 140** (0–4 hops, uint128/uint256 extremes). `contracts/README.md` on main is still the stub; PR #9's ABI is unchanged from §3 as far as its README states — will re-check before the submit PR. ## Test plan - [x] `make lint` (secret scan clean) and `make test` green locally (Python 3.13) and on LXC 140 (Python 3.12.3) - [x] fake JSON-RPC HTTP node + fake RFC 6455 server, all offline: UA header, keep-alive, failover/cooldown, rate-limit degrade, batch caps/chunking, semantic errors not failed over, same-host refusal, unmeasured refusal, lag demotion + height probe, WS handshake/fragmentation/ping, poll fallback and hand-back - [x] head-arrival measurement above run against the real local node + public WSS + LXC 139 WSS 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(bot): rpc_pool (role/latency/height-aware endpoint pool, WS newHeads) + executor_abi (WP 7914, 7912)
All checks were successful
ci / forge-tests (pull_request) Successful in 1s
ci / python-tests (pull_request) Successful in 31s
87244af3fa
bot/rpc_pool.py
- parses rpc-pool.toml through rpc_probe.parse_pool/load_pool (shared schema, refuses the
  measured_latency_ms=0 placeholder), adds pool semantics: roles read/subscribe/submit,
  local/public class, per-class batch caps (public 8, local unbounded — §4.5), local-class
  endpoints can never carry `submit`, and two submit endpoints resolving to the same host are
  refused at startup (the arc-arb forwarder-aliasing incident, learnings §4.3.1).
- `resolve = "<ip>"` dials a pinned IP while the URL host stays TLS SNI / HTTP Host (curl
  --resolve) — required for LXC 139 arc-node.falcon-python.ts.net (no MagicDNS on LXC 140).
- persistent keep-alive HTTP/1.1 connections per (endpoint, thread), TCP_NODELAY,
  User-Agent arc-mev-bot/<ver>, EWMA latency, consecutive-failure cooldown; -32005 is an
  EndpointDegraded health state (immediate cooldown + failover), never an exception that
  reaches the main loop.
- height-aware selection: read/subscribe candidates more than max_lag_blocks (2) behind the
  best observed height are demoted (the local follow node fell 175-275 blocks behind under
  CPU load on the 4-vCPU box); heights come from every WS head, poll, eth_blockNumber and a
  periodic probe, exported as arcmev_block_height{endpoint}.
- minimal RFC 6455 WebSocket client (socket/ssl only) and HeadSubscriber: newHeads fan-in
  over every subscribe endpoint, first-arrival-wins dedupe, 250 ms eth_blockNumber poller
  that takes over only while WS is silent, reconnect with backoff.
- `python3 -m bot.rpc_pool --measure-heads N` measures WS-vs-poll head arrival. Measured on
  LXC 140 over 301 consecutive blocks (0 missed): arc-public WS first 298/301; local-node WS
  +118 ms p50 / +229 ms p90; LXC 139 WSS +98 ms p50; 250 ms polling +265 ms p50 / +428 ms p90.

bot/executor_abi.py
- encodes execute((address,uint8,address,address,uint256)[],uint256,uint256) exactly per
  architecture §3 (selector 0x7328892b), strict decoder, uint256 profit return decoder,
  hops_from_path with integer minOut slippage; refuses floats, reserved adapter kinds,
  minProfit=0, discontiguous paths, dirty padding.
- tests round-trip byte-for-byte against `cast calldata` / `cast abi-encode` fixtures
  generated with Foundry 1.8.1 on LXC 140 (0-4 hops, uint128/uint256 extremes).

Tests: bot/tests/test_rpc_pool.py (fake JSON-RPC HTTP node + fake RFC 6455 server, offline),
bot/tests/test_executor_abi.py. `make lint test` green.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Signed-off-by: BuddySan <[email protected]>
Merge branch 'main' into perf/rpc-pool-executor-abi
All checks were successful
ci / forge-tests (pull_request) Successful in 3s
ci / python-tests (pull_request) Successful in 31s
ci / forge-tests (push) Successful in 5s
ci / python-tests (push) Successful in 36s
5db522ddd5
fortiblox-dev left a comment

automerger v2 auto-approved: all 7 gates pass. Approver: fortiblox-dev (opposite-identity rule). Gates: mergeable=true,label=auto-merge,author=buddysan,identity=ok,status_checks=not_required,signatures=upstream_BP_only,opt_in=repo_marker.

automerger v2 auto-approved: all 7 gates pass. Approver: fortiblox-dev (opposite-identity rule). Gates: mergeable=true,label=auto-merge,author=buddysan,identity=ok,status_checks=not_required,signatures=upstream_BP_only,opt_in=repo_marker.
buddysan deleted branch perf/rpc-pool-executor-abi 2026-09-16 14:25:46 +00:00
Author
Owner

Auto-merged by forgejo-automerger at 2026-09-16 14:25:46 UTC. Approver: fortiblox-dev (opposite-identity rule, Option C upgrade 2026-05-13). Merger: buddysan via fast-forward-only (NOT admin force_merge). Criteria passed: mergeable=true,label=auto-merge,author=buddysan,identity=ok,status_checks=not_required,signatures=upstream_BP_only,opt_in=repo_marker. Branch 5db522ddd5 merged onto main and feature branch deleted. See project_forgejo_automerger memory for rationale.

Auto-merged by **forgejo-automerger** at 2026-09-16 14:25:46 UTC. **Approver:** `fortiblox-dev` (opposite-identity rule, Option C upgrade 2026-05-13). **Merger:** `buddysan` via `fast-forward-only` (NOT admin `force_merge`). **Criteria passed:** `mergeable=true,label=auto-merge,author=buddysan,identity=ok,status_checks=not_required,signatures=upstream_BP_only,opt_in=repo_marker`. Branch `5db522ddd5` merged onto `main` and feature branch deleted. See `project_forgejo_automerger` memory for rationale.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
fortiblox/arc-mev-bot!15
No description provided.