Generative AI Projects: 10 Portfolio Ideas with Architecture

Two freshers apply for the same GenAI role in Indore. The first lists "Chatbot using LangChain" and "PDF Q&A app". The second writes: "Hindi–English help-desk bot over 140 college circulars. Recall@5 of 0.91 on a 120-question test set, p95 latency 2.3 seconds, cost ₹0.04 per answer."

The interviewer asks the first candidate what went wrong while building. He is not sure. The second candidate talks for ten minutes about chunk sizes, a judge model that disagreed with her and a cache that halved her bill.

Projects are not about the topic. They are about evidence that you can build, measure and improve.

What makes a good Generative AI project?

A strong Generative AI portfolio project is a working application that solves one clearly defined problem on realistic data, documents its architecture, and reports measured quality, speed and cost, along with what you tried and what failed.

Recruiters skim. They look for four signals:

1. A real problem and real data, not the same demo PDF everyone uses.
2. An architecture diagram showing the pieces and how data flows.
3. Numbers: an evaluation set, a metric, latency and cost.
4. Engineering habits: tests, a clean repo, a demo, honest limitations.

The GenAI career roadmap explains which skills to learn in which order and describes three starter projects. The ten below go further, and many reuse ideas from the LLMOps lessons.

Pick at least one project from each layer. Together they tell a complete story.

LLM application projects

1. Structured data extractor for invoices and bills

Small shops and CA offices receive hundreds of GST bills as photos and PDFs. Build a tool that turns each into validated JSON: vendor, GSTIN, date, line items, tax and total.

upload -> OCR text -> LLM with JSON schema -> Pydantic validation
       -> if invalid: retry with error message -> save to SQLite -> CSV export

Measure: field-level accuracy on 50 hand-labelled bills, and how often validation needed a retry. Stretch: flag bills where line items do not add up to the total. Schema prompting is covered in structured output and JSON.

2. Text-to-SQL analytics assistant with guardrails

Let a shop owner ask "which product sold most in Diwali week?" in plain English. The model writes SQL, but code decides whether it may run.

question -> schema + examples in prompt -> LLM writes SQL
         -> guard: single SELECT, allowed tables, LIMIT added -> run read-only
         -> result table + chart + the SQL shown to the user

The guard is the part interviewers will ask about. Here is a minimal version you can extend.

import re, sqlite3

db = sqlite3.connect(":memory:")
db.executescript("""CREATE TABLE sales(product TEXT, qty INT, day TEXT);
INSERT INTO sales VALUES ('kaju katli',40,'2026-11-08'),('soan papdi',65,'2026-11-08'),
('kaju katli',55,'2026-11-09'),('besan laddoo',30,'2026-11-09');""")
ALLOWED = {"sales"}

def guard(sql):
    s = sql.strip().rstrip(";")
    if ";" in s or not re.match(r"(?is)^\s*select\b", s):
        return None, "only one SELECT statement allowed"
    if re.search(r"(?i)\b(insert|update|delete|drop|alter|attach|pragma)\b", s):
        return None, "write or admin keyword found"
    tables = set(t.lower() for t in re.findall(r"(?i)\b(?:from|join)\s+(\w+)", s))
    if not tables <= ALLOWED:
        return None, f"table not allowed: {tables - ALLOWED}"
    return (s if re.search(r"(?i)\blimit\b", s) else s + " LIMIT 100"), "ok"

for sql in ["SELECT product, SUM(qty) q FROM sales GROUP BY product ORDER BY q DESC",
            "DROP TABLE sales", "SELECT * FROM users"]:
    safe, why = guard(sql)
    print(why, "->", db.execute(safe).fetchall() if safe else "blocked")

Measure: execution accuracy on 40 questions with known answers. Stretch: run on a real database with a read-only user, the stronger guard.

3. Lecture notes summariser with a Gradio demo

Turn a two-hour recorded lecture transcript into chapter-wise notes and ten revision questions.

transcript -> split by time/topic -> summarise each chunk (map)
           -> merge summaries (reduce) -> notes + quiz -> Gradio app on Spaces

Measure: a rubric (coverage of key points, no invented facts) scored on five lectures by you and a classmate. Stretch: stream output and cite timestamps. See Gradio and Spaces.

Retrieval and data projects

4. Bilingual help-desk RAG bot with an evaluation report

Answer students' questions from a college's circulars and rules, in Hindi or English, with citations.

PDFs -> clean + chunk -> embeddings -> vector DB
query -> hybrid search (BM25 + vectors) -> re-rank top 20 -> LLM answer with [source]
offline: 120-question golden set -> recall@5, faithfulness, answer correctness

Measure: retrieval recall@5 and faithfulness, as in RAG evaluation. Stretch: compare two chunking strategies in a table.

5. Semantic job-search engine with a re-ranker

Search 5,000 job postings by meaning ("data role, fresher, Indore, no night shifts"), not keywords.

postings -> embeddings -> vector index + metadata filters (city, experience)
query -> retrieve 50 -> cross-encoder re-rank -> top 10 with "why this matched"

Measure: precision@10 on 30 queries you labelled. Stretch: add a hybrid keyword score and show which queries it fixes. Background in hybrid search and re-ranking.

Model and fine-tuning projects

6. Support-ticket router: prompting vs LoRA fine-tuning

Classify 3,000 synthetic or anonymised support tickets into eight queues. Compare a prompted model with a small model fine-tuned with LoRA.

tickets -> clean + label -> train/val/test split
A: few-shot prompt on a hosted model     B: LoRA on a small open model
both -> macro F1, latency, cost per 1,000 tickets -> comparison table

Measure: macro F1 and cost per 1,000 tickets for both. Stretch: error analysis of the five most confused queue pairs. See fine-tuning with LoRA.

LLMOps and serving projects

7. Evaluation and regression harness in CI

Build the test suite for any chatbot: a golden set, code rubrics, an LLM judge, and a GitHub Actions job that blocks merges when scores drop.

golden.jsonl -> run app -> exact match / F1 / rubric / judge (JSON verdicts)
judge calibration: 60 human labels -> Cohen's kappa
CI: compare with main branch -> fail if quality or safety drops

Measure: judge kappa against your labels, and one regression the harness caught. Uses LLM evaluation and LLM-as-a-judge.

8. Red-team report for an open-source chatbot

Pick an open-source chatbot or your own project 4, write a threat model, attack it, fix it, and publish the before/after attack success rates.

threat model -> seeds per category x strategies -> run attacks
-> detectors + judge -> ASR by category -> fixes in code -> rerun

Measure: attack success rate per category before and after. Stretch: add the attacks to project 7's CI. Method in red teaming LLM apps.

9. Self-hosted model on vLLM with a load-test report

Serve a small open model with vLLM on a rented GPU for a few hours, load-test it, and write up capacity and cost.

vllm serve <model> -> vllm bench serve at 1, 4, 8, 16 req/s
-> TTFT and TPOT p50/p95 -> requests per GPU-hour -> cost per 1,000 replies
compare: 16-bit vs quantised, prefix caching on vs off

Measure: the request rate at which p95 TTFT crosses one second. Guide: serving LLMs with vLLM.

10. Observability dashboard for an LLM app

Instrument project 4 with traces and build a dashboard: latency percentiles, cost per chat, thumbs-down rate by prompt version.

app -> OpenTelemetry spans (retrieve, chat, guardrails) -> mask PII
-> trace store -> dashboard: p50/p95, tokens, cost, feedback -> alert rules

Measure: find and fix one real slow-tail cause. See LLM observability.

How to combine projects into a capstone

Projects 4, 7, 9 and 10 fit together into one production-style system: a RAG bot, served on vLLM, tested in CI and traced in production. That single capstone shows the full LLMOps loop.

One system, four portfolio entries, and a story you can explain end to end in an interview.

How to present projects on GitHub

1. Title and one-line problem statement at the top of the README.
2. Architecture diagram and a short "how it works" list.
3. Results table: metric, baseline, your system, and the test-set size.
4. Demo: a hosted link or a 60-second screen recording.
5. What failed and what you changed. This section starts the best interview conversations.
6. How to run it, with a sample .env and no real keys committed.

Common portfolio mistakes

• Tutorial clones. If your project matches a popular video step by step, it shows nothing new.
• No numbers. "Works well" is not a result. Report a metric on a test set.
• Committed API keys. Recruiters notice, and so do bots that scan GitHub.
• Ten shallow projects. Three deep ones with evaluation win.
• Naming hype instead of problems. Lead with the user and the outcome, not the framework.

Key takeaways

• Good projects solve a specific problem on real data and report quality, latency and cost.
• Cover four layers: applications, retrieval, models and LLMOps.
• Keep architecture outlines simple and put a diagram in every README.
• Combine RAG, evaluation, serving and observability into one capstone.
• Next, prepare to defend your work with Generative AI interview questions.