tanghoongAug 28月2日
Sync or async for an AI support chatbot
If customer A is waiting for the LLM, does customer B have to wait too? The answer is an architecture decision, and it matters long before you have a thousand customers.
Tag標籤
22 posts篇文章
If customer A is waiting for the LLM, does customer B have to wait too? The answer is an architecture decision, and it matters long before you have a thousand customers.
Producing a convincing answer is no longer the hard part. The hard part is the agent still working tomorrow — and that turns out to be a state problem, not a memory problem.