Deskpilot: Building an Autonomous Operator for Your Personal Multi-Broker Trading Desk

☕ The 40-Minute Pre-Market Grind

If you run swing trades or options-income strategies across multiple brokers—like Longbridge, Tiger Brokers, or MooMoo—your trading day always starts with the exact same manual routine before you ever make a single decision:

1. Reconcile positions across separate broker accounts and mobile apps.
2. Check option expiries coming due within the next 7 days.
3. Re-read technical indicators (SMA 20/50/200, Wilder RSI(14), ATR%, 52-week position) on every ticker in your book.
4. Size options-income setups off the implied expected move.
5. Try to recall yesterday's intent: Why did I enter this trade? Was I looking to roll, take profit, or hold?

It is 30 to 45 minutes of repetitive, error-prone context gathering every single morning—and the most critical part, "what did I intend yesterday?", lives only in the trader's head. That is precisely the kind of event-driven, multi-step routine an autonomous AI agent should own.

⚡ The Taskmaster Solution: One Prioritized Daily Desk Plan

I built Deskpilot for the Google All Things Agentic Hackathon under the Taskmaster track—a category built for event-driven workflows with autonomous routing that execute a multi-step routine start-to-finish without step-by-step human intervention.

Deskpilot turns a synced multi-broker portfolio into a single, prioritized daily plan. Triggered on a schedule (or on demand), it acts as an autonomous desk operator: recalling yesterday's intent, reviewing live book risk, checking technicals, sizing options setups, and writing a single prioritized plan back to persistent memory—completely unattended.

🛡️ Read-Only Decision Support: Deskpilot never places, modifies, or cancels an order, and gives no personalized buy/sell advice. It surfaces setup opportunities, risk warnings, and technical reasoning; you decide and execute.

🧩 Multi-Agent Orchestration: Why AgentTool Beats Sub-Agent Transfers

Deskpilot is built on the Google Agent Development Kit (ADK). Rather than shoving all instructions into one giant mega-prompt, Deskpilot uses an Orchestrator–Specialist graph running on Gemini (≥ 3.5):

💡 Key Architectural Breakthrough: AgentTool vs. Sub-Agent Transfer

During early development, I tested ADK's native sub_agents and transfer_to_agent transfer pattern. However, a transfer hands control away from the orchestrator and never returns. For a multi-step routine like Deskpilot's—where the system must review risk, read technicals for three names, size options, and then synthesize a single plan—a plain transfer stalled after the first specialist hop.

The solution was re-modeling the specialists as AgentTools (ADK AgentTool). When the Orchestrator invokes RiskOfficer or MarketAnalyst, it invokes them as tools, receiving their structured analysis back while staying in control of the execution loop to complete the routine and persist the final daily plan.

📊 Deterministic Numbers, Agentic Judgment

One of the biggest traps in financial AI is asking LLMs to perform arithmetic (like FIFO P/L, SGD currency conversion, or RSI calculations). Large language models excel at prioritization and natural-language synthesis, but stumble on precise math.

Deskpilot enforces a strict boundary: all numbers come from deterministic Python code; Gemini handles only prioritization, risk reasoning, and strategy synthesis.

• Portfolio Normalization: tools/portfolio.py parses position snapshots, calculates FIFO P/L, converts foreign exchange rates to SGD, and aggregates cash reserves deterministically.
• Public Technicals & Expected Move: tools/market.py uses yfinance to calculate SMA trends, Wilder RSI(14), and the ATM-straddle option-implied expected move cleanly in Python.

📸 Zero-Credential Published Google Sheet Integration: Deskpilot reads your live book directly from a published CSV link, requiring no API keys or broker logins.

To make the data layer zero-friction, Deskpilot reads your portfolio positions directly from a published Google Sheet CSV link (DESKPILOT_PORTFOLIO_CSV_URL). No broker credentials or OAuth tokens are needed, allowing anyone to point Deskpilot at their own portfolio in seconds.

🧠 Persistent Memory as a First-Class Tool

What turns a basic chatbot into a true desk operator is memory over time. Without memory, an AI evaluates your portfolio as if it has never seen your holdings before.

Deskpilot incorporates a Cloud Firestore memory bank (deskpilot/memory/store.py), equipped with four explicit memory tools registered to the Orchestrator:

• get_last_plan: Recalls yesterday's daily plan and active trade theses before starting today's review.
• remember / recall: Stores specific ticker notes, rolling intentions, and target price levels across sessions.
• save_daily_plan: Writes today's prioritized plan into Firestore, creating an auditable timeline of trading decisions.

📸 ADK Dev UI on Cloud Run: Deskpilot executing the daily desk plan, streaming specialist tool calls, and persisting the final plan to Firestore.

🛠️ Hard-Fought Engineering Lessons & Bug Fixes

Building Deskpilot taught several critical lessons about deploying ADK multi-agent systems to production:

1. Tool Outputs Are Part of the Model Request (The NaN Bug)

During live market testing, a run suddenly crashed with an API INVALID_ARGUMENT (400) error. The root cause was subtle: yfinance returned a market holiday row containing NaN values. When Python dictionary tools serialized NaN or inf into the JSON payload sent to Gemini, the model API rejected it because NaN is not valid JSON syntax. I fixed this by coercing NaN/inf to null at the tool boundary and adding a regression unit test in tests/test_tools.py.

2. PowerShell Shell Mangling on Cloud Run Deployments

Deploying to Google Cloud Run via PowerShell brought a shell gotcha: unquoted flags like --set-env-vars A=1,B=2 get split on commas by PowerShell, collapsing environment variables and causing runtime model configuration errors. Wrapping the entire variable string in quotes ("--set-env-vars=A=1,B=2") resolved the deployment issue.

3. Tool-Surface Safety Guardrails

Rather than relying on system prompt instructions like "Do not place orders", safety is hard-coded into the tool surface. No order placement or execution tools exist in the codebase—making Deskpilot read-only by construction.

🚀 Key Takeaways & What's Next

Deskpilot proves that complex, multi-broker trading routines can be automated reliably by combining deterministic Python calculations with Gemini's multi-agent reasoning on Google ADK and Cloud Run.