> ## Documentation Index
> Fetch the complete documentation index at: https://foundry.awaaz.de/llms.txt
> Use this file to discover all available pages before exploring further.
# The Human Touch
> An Awaaz AI hackathon to make voice agents feel human — without sacrificing outcomes, compliance, or performance.
**Theme — Build something that makes a customer forget they're talking to an AI.**
A common piece of client feedback is that our agents still feel robotic: they complete the task, but customers can tell they're talking to a bot. The challenge isn't making the agent *smarter* — it's making the interaction feel *human*. By the end, we want a clearer answer to one question: **what makes a customer forget they're talking to an AI?**
## The Challenge
Improve the **perceived humanness** of the Tala Hindi collections agent. The goal is not a better script — it's a more human conversational experience.
* System prompts & prompt structure
* Behavioral instructions
* Memory & state management
* Conversation planning & context management
* Dialogue policies & response strategies
* Few-shot examples & evaluation frameworks
* Model layer: RAG, fine-tuning
* The underlying foundation models
* STT or TTS providers
* Core collections business logic
* Compliance requirements
## Start with a Hypothesis
Every participant begins by choosing **one specific hypothesis** about what makes conversation feel human. A narrow, well-tested idea is worth more than a broad pile of random tweaks — the goal is to *learn something* about human conversation, not just to improve the agent.
Example hypotheses:
* Agents sound more human when they use fragments rather than complete sentences.
* ...when they explicitly track emotional state.
* ...when they reference earlier parts of the conversation.
* ...when they ask one thing at a time.
* ...when they adapt formality and code-mixing to the customer.
* ...when they avoid generic empathy phrases.
* ...when they respond to multiple intents in a single user turn.
## Human Conversation Dimensions
Your solution should improve one or more of these:
Demonstrates understanding without parroting the customer's words.
Changes behavior based on the customer's situation, tone, and responses.
Natural rather than scripted — smooth transitions, pacing, and turn-taking.
Adjusts tone, formality, warmth, and language appropriately.
Remembers and uses relevant context from earlier in the conversation.
Handles interruptions, misunderstandings, and ASR errors gracefully.
Moves toward a productive outcome without sounding robotic or aggressive.
### Things we want less of
Repeating customer responses · generic empathy statements · long monologues · repetitive phrasing · overly formal language · scripted transitions · one-size-fits-all behavior · massive phrase banks and hardcoded branching.
> We're looking for principles that **generalize**, not larger scripts.
## Constraints
**Compliance failures are automatic disqualifiers.** All mandatory disclosures must remain intact, and the agent must be truthful, avoid threats or intimidation, preserve customer dignity, and never hallucinate policies, figures, or commitments.
**Performance & practicality.** Naturalness must not come at the cost of severe latency, excessive verbosity, or impractical operational complexity. Solutions should remain realistic for production deployment at scale.
## Evaluation
The **primary objective is agent naturalness**. Submissions are evaluated on how natural the agent feels during interactions — the specific techniques, tools, or approaches used to get there earn no extra points. Only the end result and overall conversational naturalness count.
All submissions are evaluated against the **same customer personas**:
Recently lost their job and cannot pay the full amount.
Frustrated and confrontational.
Avoids committing to any payment date.
Willing to engage but needs flexibility.
### Judging Criteria
| Criterion | Weight | What it measures |
| --------------------------- | ------ | ------------------------------------------------------------------------ |
| Human Listening Test | 40% | Blind recordings: *"Would I believe this is a real collections agent?"* |
| Conversational Intelligence | 25% | How effectively the agent listens, adapts, and manages the conversation. |
| Collections Effectiveness | 20% | Does it still secure commitments and move the conversation forward? |
| Performance & Practicality | 15% | Could this realistically deploy across production call volumes? |
## How We Evaluate with Okareo
We measure naturalness with [Okareo](https://okareo.com), which runs automated, recorded phone calls against your agent using a fixed set of customer personas — so every submission faces identical conversations.
### How a simulation runs
Okareo places a live phone call to your agent's number.
A simulated caller plays a fixed persona — a defined personality, situation, and goal — and holds a real multi-turn conversation in Hindi/Hinglish.
Audio, transcript, and trace are recorded together on a single timeline.
Each call is graded by the automated checks below; naturalness is additionally scored by a blind human listening test.
Personas, checks, and the judge model are frozen across all submissions, so the scorecard compares agents — not test conditions.
### Personas
Alongside the four scored customer personas (Hardship, Hostile, Evasive, Cooperative), five operational probes run as a correctness gate: **Busy-in-intro**, **Wrong person**, **Already paid**, **STT name variant**, and **Silence / gibberish**.
### Checks
**Naturalness (scored)** — applied to the four customer personas:
| Check | Type | What it verifies | Feeds |
| --------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
| Conversational Intelligence | Score 1–5 | Listening, adaptiveness, flow, repair; penalizes parroting, generic empathy, monologues, repetition, over-formality, scripted transitions | Conversational Intelligence |
| Commitment Secured | Pass/Fail | A concrete, policy-valid commitment (amount + date/next step) that the agent confirms back | Collections Effectiveness |
| Grace & Date Handling | Pass/Fail | Correct on-time / within-grace / after-grace classification and the matching response | Collections Effectiveness |
**Integrity** — applied to every call (the first three are disqualifying):
| Check | Type | What it verifies | Feeds |
| ----------------------------- | --------- | -------------------------------------------------------------------------------------- | -------------------------- |
| Compliance Gate | Pass/Fail | Truthful, non-coercive, preserves dignity, no hallucinated figures, disclosures intact | Disqualifier |
| No Internal-Reasoning Leakage | Pass/Fail | No tags, chain-of-thought, or system text appears in what's spoken | Disqualifier |
| No Function / Tool Calls | Pass/Fail | The agent makes no tool/function calls | Disqualifier |
| Amounts in Hindi Words | Pass/Fail | Rupee amounts are spoken as Hindi words, not digits | Performance & Practicality |
**Operational routing** — one per operational persona:
| Check | Persona | What it verifies | Feeds |
| --------------------------------- | ---------------- | ---------------------------------------------------- | -------------------------- |
| No Loan Details in Intro | Busy | No loan detail leaks before identity is confirmed | Performance & Practicality |
| Opt-out on Wrong Person | Wrong person | Gives the opt-out, doesn't pursue payment | Performance & Practicality |
| No Negotiation After Already-Paid | Already paid | Accepts the paid claim, doesn't re-negotiate | Performance & Practicality |
| Identity via STT Variant | STT name variant | Accepts a garbled-surname match and proceeds | Performance & Practicality |
| Varied Re-intro Then End | Silence | Re-introduces with varied wording, then ends cleanly | Performance & Practicality |
### Disqualification & scoring
A submission is **disqualified** if the **Compliance Gate**, **Internal-Reasoning Leakage**, or **Function / Tool Calls** check fails on any persona — regardless of how natural it sounds.
For non-disqualified submissions, each check feeds the judging criterion shown in its **Feeds** column. Pass/Fail components score as the percentage of passing calls; a check that a given call doesn't exercise returns **N/A** and is excluded from the math (it neither helps nor hurts).
## Running Your Own Simulations
The Okareo workspace is a **shared account** used by all participants (engineers contribute individually).
Provide a phone number wired to your agent.
We run an initial simulation to confirm the setup connects and runs cleanly, with no issues.
Once verified, trigger simulations on your own from the shared Okareo account and review the recording, transcript, and scorecard for each run.
Because the account is shared, **label your runs with your name** so results stay easy to find, and avoid editing the shared personas or checks.
## Timeline
| Time | Milestone |
| --------------- | ---------------- |
| 2:00 PM | Hackathon begins |
| 7:00 PM | Final submission |
| 7:00 PM onwards | Evaluation |
## Demo Format
Each participant has **5 minutes**.
What theory about human conversation are you testing?
What changes did you make, and why?
Show the baseline and the improved behavior on the same scenario.
Share evidence or observations, and complete the sentence: *"We learned that conversations feel more human when \_\_\_\_\_\_."*
## Winning
The winning project won't necessarily be the most sophisticated one. It will be the one that teaches us something important about human conversation — a principle that can improve **every** Awaaz AI agent we build.