Spark History MCP — Investigate Jobs via Tools

Main

• Use shs for quick, deterministic inspection; use MCP when you want an agent to run multi-step investigations across apps and stages.

• Keep config explicit: README uses shs setup config > config.yaml and expects you to set your History Server URL there.

• Choose transport by deployment: streamable HTTP is convenient for remote clients; stdio is simple for local setups (README).

• Use comparisons to avoid guesswork: README links a real-world example of comparing two benchmark runs and highlights failure investigation commands.

Source-backed notes

• README says the project provides two interfaces: an MCP server (spark-mcp) and a standalone CLI (shs).
• README shows running the MCP server directly via uvx --from mcp-apache-spark-history-server spark-mcp and mentions PyPI publishing.
• README config shows an MCP port default of 18888 and transport options streamable-http or stdio.

FAQ

• Do I need MCP if I only want scripts?: No — use shs CLI directly; MCP is for agent-driven investigations (README positioning).
• Where do I set the Spark History Server URL?: In config.yaml; README generates it via shs setup config > config.yaml.
• What port does the MCP server use?: README defaults to port 18888 and supports transport configuration.