Contributor reference. This is the provider-implementation taxonomy — how connectors are triaged and built. If you just want to use providers, see Providers & features and Configuration.

Flowcat provider protocol-family map

Mirrors the per-provider feature matrix + shared-file pattern used elsewhere; covers the STT / TTS / LLM provider breadth (see ROADMAP.md).

Purpose. This map covers the ~70 remaining providers, each landing in an isolated worktree. The merge hazard is contention on the two shared files — flowcat-services/Cargo.toml ([features]/[dependencies]) and each category's src/{stt,tts,llm}/mod.rs (the mod-decl + pub use list). Pre-creating every provider's module home + feature + dep + a compiling stub means each provider fills one body — never adds a mod decl or a Cargo line → conflict-free merge. It also classifies every provider as (D) distinct client or (W) thin wrapper so a protocol family's real client is implemented once and the wrappers reuse it.

0. The two-letter triage

The discipline: implement each family's (D) client once; every (W) in that family is ~30 lines of config + delegation. A category agent should own a whole family (the (D) client + all its (W)s) so the wrapper contract is set by the same hand.

1. LLM (26 providers; openai✓ done, anthropic = stub-home here)

The headline finding (confirmed against pipecat): the LLM list is overwhelmingly OpenAI-compatible. In pipecat 15 of the LLM services literally class XLLMService(OpenAILLMService) — they are a base_url + default-model change. They are all (W) over the existing OpenAiLlm (which already has .base_url() + .model() builders and was written for exactly this — its doc already names OpenRouter). Only 3 are (D).

¹ mistral: the brief flags it as arguably (D), but pipecat's MistralLLMService(OpenAILLMService) is OpenAI-compatible → (W). (Mistral STT/TTS are (D) — see below; only the LLM is a wrapper.) ² azure: OpenAI-compatible wire but a different auth (api-key header) + URL shape (?api-version=). It needs OpenAiLlm to accept an api-key-style header — a tiny (D-ish) seam on the OpenAI client (an auth-mode toggle), then the rest is (W). Grouped with the (W) cohort but flagged for the auth seam.

LLM (D) count: 5 (openai✓, openai_responses, anthropic, google/gemini, aws_bedrock). LLM (W) count: 18. → One agent implements the 4 new (D) clients; one agent fans out all 18 (W)s (each a base_url+model struct over OpenAiLlm). The anthropic stub-home is set up here because its llm-anthropic feature already exists from earlier but the impl was never landed (the earlier "anthropic done" is the feature, not the module) — this map fills it.

2. STT (~21 remaining; deepgram✓ done)

Protocol families (cross-checked against pipecat base classes):

• Streaming-WebSocket (each its own JSON schema) — WebsocketSTTService subclasses. Mostly (D) (distinct message schemas), except where they ride a sibling's protocol.
• Whisper-HTTP segmented — BaseWhisperSTTService / SegmentedSTTService: POST a finished audio segment to an OpenAI-/audio/transcriptions-shaped endpoint. One (D) Whisper-HTTP client, the rest (W) over it.
• gRPC — Google + NVIDIA Riva use tonic. (D) each (distinct protos).
• Local — whisper.cpp via whisper-rs. (D), C-toolchain (see §5).

³ xai STT: pipecat's is WebsocketSTTService; xAI exposes an OpenAI-compatible STT — treat as (W) over the Whisper-HTTP/OpenAI-STT family unless a live test shows a distinct WS schema (flag for the implementer). Conservative grouping: (W).

STT (D) count: ~16 (deepgram✓ + assemblyai, gladia, soniox, speechmatics, cartesia, aws_transcribe, azure, elevenlabs, openai-Whisper-base, sarvam, mistral, google-gRPC, nvidia/riva-gRPC, gradium, whisper_local). STT (W) count: ~4 (groq, fal, speaches, xai — all Whisper-HTTP over the openai client). The brief also names nvidia, riva, speaches — riva is nvidia's STT (one provider, one feature stt-nvidia); speaches is the (W).

3. TTS (~30 remaining; cartesia✓ done)

Two big families + a long tail of HTTP-POST-audio providers:

• Streaming-WebSocket TTS — WebsocketTTSService subclasses: connect a WS, send a synthesis request, read base64/binary PCM chunks. The Cartesia ref impl is the template. Each has its own request/response schema → (D).
• HTTP-POST-audio (one-shot / segmented) — TTSService subclasses that POST text and read an audio body (mp3/pcm/opus). The wire shape is similar (POST → audio bytes) but the request schemas + audio containers differ enough that there is no single reusable client; most are (D), a few are (W) where they share an OpenAI-TTS shape.

TTS (D) count: ~28 (cartesia✓ + 27). TTS (W) count: ~2 (groq, xai — OpenAI-TTS-HTTP). TTS is the least wrapper-friendly category — almost every vendor has a bespoke request body. The leverage here is two shared client helpers (a WsTtsClient connect/send/read-base64 skeleton mirroring Cartesia, and an HttpTtsClient POST-and-read-audio skeleton) that the (D) impls reuse for the transport plumbing while each supplies its own request encode + container decode. (Those helpers are an implementation convenience for the implementations, not part of this map.)

4. Recommended fan-out grouping (which agent owns what)

To keep each protocol family's (D) client authored once and its (W)s consistent, group by family, not by alphabetical slice:

8 groups, family-coherent, no two touching the same provider. The new auth/SigV4 paths (aws_bedrock, aws_transcribe, aws_polly, the azure api-key seam) are the security-sensitive ones.

5. Toolchain / dependency notes for the fan-out

This crate declares every dep optional + dep:-gated so a default build pulls nothing and --features <provider> compiles the stub. Three families need a toolchain/dep beyond the existing reqwest/tokio-tungstenite/tokio/base64:

opentelemetry/gRPC/whisper-rs are the only non-pure-Rust-network deps. Everything else is reqwest (rustls) or tokio-tungstenite (rustls) — the rustls-only + zero-default-cost discipline holds. This adds hmac + sha2 (optional) for the AWS SigV4 family; no other new dep.

6. Acceptance

• cargo build -p flowcat-services (default) → pulls nothing, compiles the trait re-exports + stubs only.
• cargo build -p flowcat-services --features stt-all,tts-all,llm-all → compiles every stub (modulo stt-whisper-local's C build, which needs cmake — see §5).
• cargo test -p flowcat-services (existing tests) → still green.
• Every stub: Apache SPDX header + //! WS-n: <provider> — TODO + a frozen-trait impl whose bodies return FlowcatError::Other("<provider>: not yet wired (WS-n)") (or todo!() for the &str name() accessor) so the crate compiles. #[allow(dead_code)] on the held config fields is the sanctioned clippy escape on a stub.

7. What this map deliberately does NOT do

• It does not implement any provider body (that is the fan-out).
• It does not add the WS/HTTP TTS helper clients (WsTtsClient/HttpTtsClient) or the gRPC build.rs / .protos — those are owned by the impls.
• It does not touch flowcat-core, the Deepgram/Cartesia/OpenAI reference impls' logic, other crates, or the embedder.
• It does not create a per-(W) stub where the (W) is literally the (D) client + a base_url (the LLM wrappers) — those get a thin module stub anyway (so the mod/feature exists for the fan-out to fill), but their stub body just notes "wrapper over OpenAiLlm — set base_url".