Production AI: Integrating OpenAI & Perplexity in NestJS

The internet is flooded with tutorials on how to build a ChatGPT wrapper in a weekend. They all follow the same pattern: receive an HTTP request, await openai.chat.completions.create(), and return the response.

This works perfectly until you deploy it to production.

In the real world—like when I was architecting the backend for a complex AI platform—AI generation isn't a 200ms operation. Complex LLM chains, especially those involving live web search (like Perplexity AI) or parsing large documents, can take anywhere from 10 to 45 seconds.

If you tie a 45-second operation to an HTTP request lifecycle, you are begging for dropped connections, gateway timeouts, and terrible UX. Here is how I architected a production-grade AI system in NestJS that actually scales.

1. AI is Data Engineering, Not Web Routing

In this platform, the backend utilizes both OpenAI (for complex reasoning and text generation) and Perplexity AI (for up-to-date, cited web research).

To prevent the API from locking up, the AI orchestration is completely decoupled from the HTTP controllers using BullMQ and Redis.

Client  →  API Controller  →  Redis Queue  →  AI Worker  →  LLM (OpenAI/Perplexity)
  │                                                              │
  │   ←──  202 Accepted { jobId }                                 │
  │                                                              │
  │                                            saves result  ←──┘
  │                                                to DB
  │
  └──────  GET /jobs/:id  →  200 OK { status: 'completed', data }

The key insight: the client gets an immediate 202 Accepted with a job ID, while the actual LLM work happens asynchronously in a worker process. The client polls (or listens via WebSocket) for the result.

2. Structuring AI Modules in NestJS

NestJS’s dependency injection is perfect for building modular AI pipelines. Instead of leaking API keys and prompt logic into random controllers, I structure them as isolated providers.

// agent-ai.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bullmq';
import { Queue } from 'bullmq';

@Injectable()
export class AgentAiService {
  private readonly logger = new Logger(AgentAiService.name);

  constructor(
    @InjectQueue('ai-processing') private aiQueue: Queue,
  ) {}

  async requestAnalysis(userId: string, contextData: any) {
    // 1. Save pending state to DB
    const jobRecord = await this.database.createPendingJob(userId);

    // 2. Enqueue the work
    await this.aiQueue.add('analyze-profile', {
      jobId: jobRecord.id,
      context: contextData,
    }, {
      attempts: 3, // LLM APIs fail randomly. Always retry.
      backoff: { type: 'exponential', delay: 2000 }
    });

    // 3. Return immediately
    return { status: 'processing', jobId: jobRecord.id };
  }
}

3. The Multi-Model Advantage

Not all LLMs are created equal. OpenAI is fantastic at structuring JSON and strictly following complex system prompts. However, if you need real-time data or citations, it hallucinates.

For this project, I built an orchestration layer that delegates tasks to the right model. We use Perplexity AI (@perplexity-ai/perplexity_ai) to scrape and summarize live data, and then feed that context into OpenAI to format and finalize the output based on strict schemas.

By isolating these calls inside asynchronous workers (like ai-processing.worker.ts), if Perplexity takes 15 seconds and OpenAI takes 20 seconds, the user's mobile app doesn't freeze. The app simply shows a skeleton loader, polls the job status (or listens via WebSockets), and renders the result when the queue finishes.

The Takeaway

When you move AI into production, you have to treat it like heavy infrastructure. Abstract the providers behind clean interfaces, decouple the execution from the request lifecycle using queues, and always build in retries. That’s how you build AI products that survive contact with real users.