Docs / agents/calendar-seed-playbook.md · mirrored from the repo

Events-Calendar — human-mimicked run playbook & tuning notes

I (an LLM acting as the events-calendar agent) filled the calendar for 2026-06-24 → 2027-06-24 by hand-running the agent loop, so the next agent has realistic, source-grounded test data and so the prompts/orchestration can be tuned from a worked example. This documents how I thought, how I orchestrated tools and web search, how I modelled the data — and the concrete changes I’d make to the agent based on doing it.

Artifacts produced:

• agents/events-calendar/seed/seed-calendar-2026-2027.mjs — the dataset generator (idempotent; reuses the agent’s makeEventId).
• agents/events-calendar/seed/calendar-2026-2027.sql — the generated INSERTs (53 events).
• This file — the reasoning + tuning notes.

1. The loop I actually ran

This mirrors the kernel orchestrator (recall → STM → tool turns → finish), but executed deliberately:

1. Recall / objective. Objective = “comprehensive, grounded Hindu calendar for the next 12 months; as many real events as possible, no duplicates, every date sourced.” Empty calendar ⇒ cold start.
2. Query existing calendar. Confirmed empty (cold start) → research broadly, don’t worry about dedupe-vs-existing yet.
3. Build a coverage plan (the checklist). I did not research festival-by-festival. I tiled the space into a handful of batch objectives:
   • Major festivals Jul–Dec 2026
   • Major festivals Jan–Jun 2027
   • Recurring tithis (Ekadashi / Purnima / Amavasya / Sankranti) across the window
   • Regional festivals & new years (Onam, Pongal, Bihu, Ugadi, Vishu, …)
4. Batch web research (2 searches did ~80% of the work). See §2.
5. Cross-check & ground each date against ≥2 authoritative panchang sources; assign tiered confidence. See §3.
6. Compose proposal-ready events with canonical names, Hindi names, region, type, significance, sources.
7. Dedup + window-filter + upsert. One row per (canonical-name + date + region); everything inside the window; status approved (seed/trusted).
8. Report + memory. This doc is the “artifact”; §6 lists the lessons the LTM should have stored.

Note for tuning: steps 3 (coverage plan) and 5 (cross-check) are where quality is made. They’re implicit in the current prompts; making them explicit, ordered tool-steps would help a weaker model.

2. Web-research strategy (what the research subagent should do)

Two batched searches covered the whole window — this is the batch-by-window behaviour the research prompt asks for, and it works:

• "major Hindu festivals dates list July…December 2026" → hit drikpanchang, calendarlabs, bankbazaar, calendarhindu in one shot.
• "Hindu festivals 2027 dates Makar Sankranti … Baisakhi" → calendarlabs 2027 + infoplease (multi-year table) for cross-checking the majors.

Observations that should shape the agent:

• Batch queries by month-range + “list/dates” retrieve a whole table per call — far more efficient than one festival per query (the local 7B tended to do single-festival queries and burn turns).
• Source tiering matters. drikpanchang.com, calendarlabs.com, bankbazaar.com agreed closely and are reliable. pujalane.com and (partly) hinduamerican.org gave systematically wrong dates (e.g. Diwali = Oct-26, Janmashtami = Aug-15). A single source is not enough for majors.
• The search answer field was null; the value was in the per-result content tables. The agent should parse result content, not rely on a synthesized answer.

3. Grounding & confidence (tiered)

I applied exactly the tiered rule the prompts describe:

Conflict resolution that the agent needs an explicit rule for: when sources disagree on the same festival (Diwali Oct-26 vs Nov-08), take the majority of authoritative sources, not the first hit. The current agent has no tie-break rule — a weak model picked a hallucinated/odd date.

4. Data-modelling decisions (schema usage)

• id = makeEventId(name,date,firstRegion) — canonical name is what prevents duplicates. I used one canonical English name (Diwali, not Deepavali) and folded variants into significance.
• type ∈ festival | devotional | national | auspicious | regional. I used festival for celebrations, devotional for Ekadashi/Jayanti vrats, auspicious for Amavasya/Sankranti/Akshaya Tritiya, national for civil days.
• regions carries pan-India (['all']) vs regional (['kerala'], ['bihar','east']). Co-celebrated festivals (Makar Sankranti = Pongal = Magh Bihu) are one row with regions + a significance note, not duplicates.
• name_hi (Devanagari) populated for every event — the global strategy wants Hindi-first; this is cheap to add and high value.
• status = approved because this is trusted seed data. A real agent run lands events as proposed (manual mode) or approved (auto mode).
• sources holds the URLs I grounded against — this is the audit trail that makes “aligned with research” real.

5. What’s in the dataset (53 events)

• Jul–Dec 2026: Rath Yatra, Guru Purnima, Hariyali Teej, Nag Panchami, Onam, Raksha Bandhan, Janmashtami, Ganesh Chaturthi, Vishwakarma Puja, Anant Chaturdashi, Navratri→Dussehra (Saptami/Ashtami/Navami), Sharad Purnima, Karva Chauth, Dhanteras→Diwali→Govardhan→Bhai Dooj, Chhath, Kartik Purnima, Geeta Jayanti, Vaikunta Ekadashi + named Ekadashis/Amavasya/Sankranti.
• Jan–Jun 2027: Makar Sankranti/Pongal, Thaipusam, Vasant Panchami, Maha Shivratri, Holika Dahan→Holi, Ugadi/Gudi Padwa, Baisakhi/Vishu/Puthandu, Ram Navami, Hanuman Jayanti, Akshaya Tritiya, Vat Savitri.
• National: Independence Day, Gandhi Jayanti, Republic Day.

Deliberately not exhaustively enumerated: every Pradosh/Sankashti/Masik-Shivratri (≈ a dozen/month). A real run can add them via a dedicated “enumerate all tithis for month X” research objective — they come cleanly from a single panchang page (calendarhindu lists them all).

6. Tuning notes — concrete changes I’d make to the agent

Ranked by expected impact:

1. Enforce research-before-propose. The weak model proposed events (with fabricated sources) before calling research. Add a hard rule/state: propose_events should be rejected (or warned) until at least one research result is in the STM. Cheap host-side guard, big quality win.
2. Give the model an explicit conflict/tie-break rule (majority of authoritative sources; prefer drikpanchang/calendarlabs/timeanddate; distrust SEO/blog sources). Encode the source allow-list in the research prompt.
3. Batch research by month-range, return a structured table. Already in the prompt — but reinforce “one query per quarter, parse the result table, return one line per event.” This alone cut my work to 2 searches.
4. Add a tithi-enumeration objective. “List every Ekadashi/Purnima/Amavasya/Sankranti for ” is a high-yield single call; the model shouldn’t research these one by one.
5. Memory should store source-reliability lessons, not dates. The two lessons worth persisting from this run (and that LTM should have written):
   • procedural: “Batch festival research by quarter and parse the result table; ~1 query covers a quarter.”
   • semantic: “drikpanchang.com / calendarlabs.com / bankbazaar.com agree and are reliable for Hindu dates; pujalane.com and generic blog SEO pages had wrong dates — require ≥2 authoritative sources for major festivals.”
6. Keep the window filter (already added) — the weak model emitted past dates (Makar Sankranti Jan-2026, in the past); the host-side window filter is essential, not optional.
7. Canonical-name discipline prevents the Diwali-on-two-dates duplicate we saw — reinforce “use one canonical English name; fold regional names into name_regional/significance.”

A stronger model (e.g. GPT-4-class via the OpenAI provider) should satisfy 1–4 from the prompt alone; 2 and 5 are worth encoding regardless of model.

7. Reproduce / reset

# regenerate SQL (edit the .mjs to change events), then load into the agent's local D1:
node agents/events-calendar/seed/seed-calendar-2026-2027.mjs > agents/events-calendar/seed/calendar-2026-2027.sql
docker compose exec -T events-calendar sh -c "cd /app && npm --workspace=events-calendar exec -- \
  wrangler d1 execute DB --env local --local --persist-to /app/.wrangler-state \
  --file=/app/agents/events-calendar/seed/calendar-2026-2027.sql"

# verify
curl -s 'http://localhost:8787/api/agents/events-calendar/events?days=400' \
  -H 'x-mock-user-email: dash.ayushman.99@gmail.com' | node -e 'console.log(JSON.parse(require("fs").readFileSync(0)).length,"events")'

Dates are grounded as of 2026-06-24; for a different window, re-run with fresh research (lunar dates shift year to year — never copy-forward).