Improvements to Stirling Engine to prepare for SaaS release (#6603)

# Description of Changes
- Use pool for postgres connections
- Add ability to require user ID to be set on API calls to the engine
- Add process-wide concurrency cap on AI access (in addition to existing
user caps)
- Allow number of workers (threads) to be specified for stirling engine
- Update env var names to reflect that the DB is not just for RAG

engine/pyproject.toml

@@ -6,7 +6,7 @@ requires-python = ">=3.13"
 dependencies = [
     "fastapi>=0.116.0",
     "pgvector>=0.3.6",
-    "psycopg[binary]>=3.2",
+    "psycopg[binary,pool]>=3.2",
     "pydantic>=2.0.0",
     "pydantic-ai>=1.67.0",
     "pydantic-ai-slim[voyageai]>=1.67.0",