PromptQL | Tanvi's Portfolio

Context

Current gap

Most business teams wait on an analyst to write the query, run the SQL, pull the report. It's slow, and the people closest to decisions are always waiting on someone else.

How PromptQL fixes it

PromptQL is an AI that learns your business — your data, your metrics, your terminology, and answers questions about it in plain English. It builds a knowledge layer from your sources so it can generate reports, charts, and dashboards on demand. Users can steer responses, teach the AI over time, and automate the queries that matter most.

That's PromptQL. Ask it anything about your business data, it runs the query and it gives you a real answer in plain language.

My role

My job was designing the part that makes people trust it:

what happens when the AI is wrong?
how can admins configure their data without hand-holding/ experts from PromptQL to help setup?
whether the product handles those moments well enough to keep users coming back

#1

How do you trust an AI answer when business decisions depend on it?
That's what I set out to design in my first week — an evaluation framework for PromptQL.

PROBLEM

Enterprise customers won't move business-critical workflows onto PromptQL without a way to verify accuracy between builds. Hallucination isn't theoretical. Every LLM does it, and without evals there's no way to know if a build got better or catch when it got worse. Evals are the trust infrastructure.

WHAT WE WERE SOLVING FOR

Before evals existed in the product, an FDE(Forward Deployed Engineer) setting up PromptQL for an enterprise account had no formal way to verify the agent's answers before go-live. They'd run a set of test queries, check the outputs by hand, and track results in a spreadsheet or a doc. Every deployment was a manual audit with no repeatability and nothing to show the customer. That process and the friction in it — became the spec.

EVAL-SET

Making AI quality legible to everyone

Each eval item carries three things: a description, the criteria being tested, and a business impact tag. Tying it to a business outcome is what makes the list legible to anyone beyond the person who wrote it. I wanted the eval-set to work as a shared artifact, something a non-technical user could also read and understand without context. Each eval item/question has a deliberate binary rating (pass/fail) to keep the signal clean.

EVAL RUN

Spotting what needs attention at a glance

The design question was: how do you let someone run 20 evals and still know where to focus without reading every result? Each run row shows a pass/fail bar — the ratio of passing to failing items at a glance, so you can tell which runs need attention before opening anything. Clicking into a run ID brings up the full item breakdown. Overview first, detail on demand.

EVAL HEALTH

Giving teams a clearer picture of system health

I wanted to add an insights layer that sat above the raw results — something that answered whether the system was actually getting better over time. The charts surface accuracy trend, pass/fail ratio, and last run snapshot in one view without drilling. That way an evaluator can walk into a customer conversation and show progress, not just point at a table of rows.

FROM PROTOTYPE TO PRODUCTION

The first prototype shipped in week one. That speed came from ramping on the design system fast enough to not problem-solve the scaffolding every time. But week one wasn't the right version. The iteration that mattered came after: the FDEs used it in real deployments and their friction reshaped the flow. Every Figma frame got annotated with UX interactions & references; daily syncs from spec to staging closed the design-to-live gap without the usual back-and-forth.

OUTCOME

134 unique users since launch (in span of 5 months)
44% average monthly growth
But the number I keep coming back to is 8.96 pageviews per user — that's not someone clicking in once out of curiosity. That's teams running eval workflows repeatedly, building it into how they ship. McDonald's, Swiggy, and McKinsey are using it in production as part of how they verify PromptQL before a build goes live.

#2

What does the human-AI loop look and feel like when the AI is wrong, mid-thought, or nudging you toward a better answer? That's what I was designing.

THE SURFACE

A PromptQL conversation isn't a form or a dashboard — it's a live exchange between a user and an agent that streams answers, reasons through data, and sometimes gets it wrong. Every interaction inside that thread is a design decision. What does the agent say when it's uncertain? How does it ask for help? Does it tell you when a thread has gotten too long to reason about cleanly, or does it just quietly degrade?

When the agent is "confidently" wrong

I designed the error states to surface uncertainty early and visibly. The agent names what it couldn't do and leaves a clear path forward. Confidence without accuracy destroys trust faster than a visible error ever could.

Add "knowledge" to PromptQL's wiki for better answers
The agent doesn't just answer — it notices what's missing. When context is thin, PromptQL nudges the user to add something that would help it reason better. It then adds the knowledge to the wiki, and improves its capabilities. I designed this as a visible but non-blocking prompt — it improves the loop without interrupting it.

Need help? Ask PromptQL using @autograph
When a user asks for human support, the agent doesn't just drop them. It acknowledges the request, confirms it's queued, and keeps the conversation grounded. The user can then view the all of the requests on the "Requests" page, if they need to.

What I learned from the "startup" experience, and what I'd bring

I ship fast, then rebuild around real use.

The first eval prototype took a week. It wasn't the version that mattered until FDEs used it and broke it.

Ownership goes past the screen.

IA groupings, the plain-English copy, merged frontend PRs, the shared design system — all of that is design work to me, not just the visual layer on top of it.

Talk to the people who use it.

Sitting in on customer & sales calls changed how I write briefs. Hearing a customer say "project" instead of "workspace" unprompted told me more about their mental model than a research session would have.

Trust is a design decision.

Every choice about what to show, hide, or confirm shapes whether someone believes the product. That's as much design work as the interface itself.

Trade-offs are the job.

Speed, polish, and safety rarely line up. Knowing which one to protect or cut, & when, is the actual skill.

Work with AI, but judgement decides.

AI gets a first version out fast, and that's real value. But its first instinct is often structurally reasonable and still wrong for the actual people using the product. The job isn't handing the draft off. It's catching where AI got it wrong and knowing why.