Short write‑ups of problems we hit running a real commerce operation and what we changed in the code because of them. No product announcements; just the reasoning.
We built reply suggestions for the customer service chat inbox, set a goal of 60% close to what staff really sent, and spent ten days between 10 and 20%. Better writers, a critic, 23,000 past replies as examples and a coverage gate did not move the live number. We stopped. For a drafter that gets one shot, the technology at a cost that makes sense is not there yet, so the next attempt puts each staff member's own AI assistant on the conversation instead.
The next thing we are building — an MCP server for customers, so a trade buyer or a dropshipper can connect their own AI assistant to their account and ask, browse and eventually order without an integration project. What it will do first, what it will not do until someone accountable says so, and the groundwork already in place.
We did not embed the business into a vector store and hope the right chunk came back. We gave the model forty‑five tools — the same actions the staff app runs, under the same permissions — and a guide on how to use them. The result is a report that used to take three people a day, or a ticket to engineering, arriving in thirty seconds with real figures. The reactions, in order: disbelief, checking whether the numbers were random, and then a very quiet "oh".
We log every call an assistant makes to our MCP server — who, which tool, how long, did it fail and why — and put it on a sysadmin dashboard. Two weeks of data found a user whose assistant had made 454 calls with a 100% error rate, and fixed the server's instructions instead of the user.
When the application moved to long‑lived workers, the commercial APM agent we had run for years stopped telling the truth — and on the test runner it stopped the tests. We replaced it with NightOwl, an open‑source telemetry stack for Laravel: an agent on each box, a local buffer that survives outages, and every request, job, query, exception and log line landing as rows in a PostgreSQL database we own, on a server we already pay for, with retention measured in years. The best part arrived by accident — an AI assistant connected straight to that database with read‑only SQL.