The assignment was one line: build a news classifier and improve its consistency using few-shot prompting.

I read that as two tasks. The first — classify an article as Sports, Politics, Business or Technology — is an afternoon's work. The second one is the one I got stuck on, because of a single word.

Improve.

Improve compared to what? You can't put "improved" in a README and call it done. Improvement is a comparative claim, and a comparative claim needs a baseline, a metric, and a number. The moment I took that seriously, the shape of the whole project changed: it stopped being a classifier that happens to use few-shot examples, and became a measurement rig that ships a classifier as its default configuration.

This post is the whole thing, start to finish — what zero-shot and few-shot prompting actually are, how I built both, how I compared them, the experiment that failed, and what the numbers said once I fixed it.

Table of contents

1. The brief and the first decision

2. What "consistency" actually means

3. Zero-shot prompting, explained

4. Few-shot prompting, explained

5. Making bad output structurally impossible

6. One class, two switches

7. The first experiment failed

8. Fixing the experiment

9. The results

10. Shipping it: three surfaces and a lab

11. Five things that bit me

12. Testing an LLM app without an API key

13. What this does not prove

14. What I'd take to the next project

The brief and the first decision

The classifier takes a short news article and returns exactly one of four categories:

• Sports

• Politics

• Business

• Technology

Plus a confidence score and a one-line rationale, so a human can see why it chose what it chose.

The first real decision was architectural, and I made it before writing any prompt: the ability to turn few-shot examples off had to be a first-class feature, not a debugging hack. If I couldn't run the exact same classifier with the examples disabled, I could never produce a before/after — and the before/after is the assignment.

So NewsClassifier was born with two independent switches:

class NewsClassifier:
    def __init__(
        self,
        client: openai.OpenAI | None = None,
        model: str = MODEL,
        use_few_shot: bool = True,   # replay the worked examples?
        use_guide: bool = True,      # include the category boundary rules?
    ) -> None:
        ...

Two booleans give a 2×2. That grid turned out to be the single most useful thing in the project, and I'll come back to why.

The stack, for anyone who wants to replicate it:

What "consistency" actually means

Before I could improve consistency, I had to define it in a way a script could compute. I landed on two numbers that must always be reported together:

Agreement — classify the same article N times and see what fraction of the runs land on the most common ("modal") label.

5 runs → Sports, Sports, Sports, Technology, Sports
modal label = Sports (4 of 5)
agreement   = 0.80

An agreement of 1.0 means the classifier never wavers on that article. This is the consistency metric.

Accuracy — is that modal label the one we expected?

Here's the crucial part, and it took me a failed experiment to internalise it: agreement on its own is not a quality metric. A prompt that confidently returns "Business" for every article ever written scores 100% agreement. It is perfectly consistent and perfectly useless. Consistency is only meaningful next to correctness, so nothing in this project — not the CLI, not the API response, not the UI — reports one without the other.

The other thing baked in from the start: a single run cannot measure consistency. One run gives you one label. Consistency is a property of repeated runs, so every measurement path in the codebase takes a runs parameter, defaults it to 3, and caps it at 10.

Zero-shot prompting, explained

Zero-shot prompting means you describe the task to the model in instructions and ask it to perform the task cold — with no worked examples of the input/output pair. Zero examples. Hence, zero-shot.

It's what most people write first, and it's a genuinely strong baseline with modern models. My zero-shot prompt has three parts.

Part 1 — the header, which fixes the task

_HEADER = (
    "You classify short news articles into exactly one of four categories: "
    "Sports, Politics, Business, Technology.\n"
)

Part 2 — the category guide, which fixes the boundaries

This is where most of my prompt-writing effort went. It's easy to write "Sports — athletic competition, teams, players." That's fine until an article about a football club's takeover arrives and the model has to decide between Sports and Business.

So every definition states what it excludes as well as what it includes:

CATEGORY_GUIDE = """\
Sports — athletic competition, teams, players, matches, transfers, tournaments,
  scores, injuries, coaching. A club's finances or broadcast-rights deal is
  Sports only when the article is framed around the sport; if it is framed as a
  corporate transaction, it is Business.
Politics — government, elections, legislation, courts, diplomacy, policy debate,
  political parties, protests, public officials acting in office. Regulation of
  companies is Politics when the article is about the policy or the lawmakers,
  and Business when it is about the effect on the company.
Business — companies, markets, earnings, mergers and acquisitions, funding
  rounds, jobs, trade, commodities, the economy, consumer prices. A tech
  company's earnings or IPO is Business, not Technology.
Technology — products, software, hardware, AI systems, research breakthroughs,
  cybersecurity, space and scientific engineering. Choose Technology when the
  article is about what the technology does or how it works, rather than about
  the money around it.
"""

Every one of those cross-references is a boundary I watched the model get wrong before I wrote it down. Overlapping categories are where classifiers become inconsistent, because the model re-decides the tie fresh on every call unless you tell it how the tie breaks.

Part 3 — the decision rules

_RULES = """\
Rules:
- Choose the single category the article is *about*, not every topic it mentions.
  Decide by the framing of the lede, not by keywords appearing later.
- Every article gets one of the four categories. There is no "Other" -- pick the
  closest fit and lower the confidence instead.
- Confidence reflects how cleanly the article sits inside one category: use 0.95+
  for unambiguous articles, 0.7-0.9 when it straddles a boundary, below 0.7 when
  the article could reasonably be filed either way.
- Keep the rationale to one sentence naming the decisive signal.
"""

Two of those rules exist purely to kill inconsistency:

• "Decide by the framing of the lede, not by keywords appearing later" is the single rule the whole classifier turns on. An article can mention a chip fabrication process nine times and still be a story about a government export ban.

• "There is no Other" removes an escape hatch. Without it, a model faced with an ambiguous article invents a fifth category or hedges — and it does so unpredictably.

Assembling them is one function, and the use_guide switch is right there in the signature:

def build_system_prompt(use_guide: bool = True) -> str:
    parts = [_HEADER]
    if use_guide:
        parts.append("\n" + CATEGORY_GUIDE)
    parts.append("\n" + _RULES)
    return "".join(parts)

SYSTEM_PROMPT = build_system_prompt()

Where zero-shot falls down

Zero-shot is telling. And telling has a specific failure mode: the model agrees with your rule in principle and then applies it differently from one call to the next, because your rule is prose and prose has slack in it.

"Framed around the sport" is clear to you and me. To a model reading an article about a £1.2 billion consortium buying a Premier League club, it's a judgement call — and a judgement call made twice can come out twice differently. That wobble is the inconsistency the assignment asked me to fix.

Few-shot prompting, explained

Few-shot prompting means you include a handful of worked examples — actual input/output pairs — in the prompt, so the model infers the task from demonstrations instead of (only) from description. A few examples. Hence, few-shot.

The difference is telling versus showing. Instead of describing how to resolve a boundary case, you show the boundary case already resolved, with the reasoning attached.

Design decision 1: examples as conversation turns, not pasted text

The naive approach is to paste examples into the system prompt as a wall of text. I replayed them as alternating user/assistant turns instead:

def _few_shot_turns() -> list[dict[str, str]]:
    turns: list[dict[str, str]] = []
    for article, result in FEW_SHOT_EXAMPLES:
        turns.append({"role": "user", "content": article})
        turns.append({"role": "assistant", "content": result.model_dump_json()})
    return turns

So the request the model actually receives looks like this:

system    : header + category guide + rules
user      : "Manchester City edged past Arsenal 2-1 at the Etihad..."
assistant : {"rationale":"A match report centred on the result...","category":"Sports","confidence":0.99}
user      : "The Senate voted 58-42 on Thursday to advance..."
assistant : {"rationale":"Legislative process in the Senate...","category":"Politics","confidence":0.99}
...  (10 pairs total)
user      : <the article you actually want classified>

This matters for three reasons:

1. The model sees the exact input format it will receive.

2. It sees the exact output shape it should produce, and where it should put it.

3. It sees a conversation it is already participating in, which is a much stronger frame than "here are some examples, now do a different thing."

Design decision 2: four easy examples, six hard ones

I ship 10 examples. The split is deliberate.

Four unambiguous ones — a match report, a Senate vote, a share price drop, a battery research result. These aren't there to teach anything subtle. They fix the label vocabulary and the output shape, one per category, so the classifier never returns "Tech" or "sports" or a two-sentence rationale.

Six that sit exactly on the boundaries — these are the ones doing real work:

Notice the pairs. Nvidia-earnings and OpenAI-capability are the same type of company pointed in opposite directions. That contrast teaches "framing, not keywords" far more sharply than either example alone. The same is true of the club takeover (Sports keywords → Business) against the coach resignation (Politics keywords → Sports).

Here's one in full so that you can see the shape:

(
    "Nvidia reported quarterly revenue of $60.9 billion, up 22% from a year "
    "earlier and ahead of analyst estimates. The chipmaker's stock rose 8% "
    "in after-hours trading and its market capitalisation now exceeds $3 trillion.",
    Classification(
        rationale="Framed around earnings, estimates and market cap rather than the technology itself.",
        category=Category.BUSINESS,
        confidence=0.92,
    ),
),

Two details in there are intentional:

• The rationale is written in the style I want back — one sentence, naming the decisive signal. Few-shot examples teach tone and format as much as they teach labels.

• The confidence is 0.92, not 0.99. If every example is maximally confident, the model learns to always be maximally confident, and the confidence score stops carrying information. The boundary examples deliberately sit between 0.82 and 0.95 so the score stays calibrated to how genuinely ambiguous the article is.

Design decision 3: keep the examples balanced

A skewed example set biases the classifier toward the over-represented category. That's a quiet failure — everything looks fine until your Sports recall is suspiciously high. So there's a test that fails the build if the set gets lopsided, and another asserting every category is still demonstrated at all.

Design decision 4: the prefix must be byte-identical

The system prompt and the 10 example turns are identical on every single call. That's ~1.3K tokens of prefix that never changes.

This is not just tidiness — it's what makes OpenAI's automatic prompt caching kick in. There's no cache_control to set; the API caches long identical prefixes on its side. But a prefix that varies between calls gets no discount at all. So the prompt is built once in __init__ and reused:

self._system = build_system_prompt(use_guide)
self._prefix = _few_shot_turns() if use_few_shot else []

Build one NewsClassifier and reuse it. Rebuilding it per request throws the cache away. There's a test that asserts the prefix is byte-stable across calls, because that property is load-bearing and would rot silently.

Making bad output structurally impossible

Before comparing prompts, I closed off an entire category of inconsistency that has nothing to do with prompting.

If the model replies in free text, you have to parse it, and it can return Tech, sports, Business/Technology, "I'd say Politics, though..." or a paragraph of prose. Every one of those is an inconsistency, and none of them is a prompting problem — they're a schema problem.

So the output is a Pydantic model, and the category is an enum:

class Category(str, Enum):
    SPORTS = "Sports"
    POLITICS = "Politics"
    BUSINESS = "Business"
    TECHNOLOGY = "Technology"


class Classification(BaseModel):
    rationale: str = Field(
        description="One short sentence citing the decisive signal in the article."
    )
    category: Category = Field(description="The single best-fitting category.")
    confidence: float = Field(
        ge=0.0, le=1.0, description="Confidence in the chosen category, 0.0-1.0."
    )

Passed to the API as text_format, this becomes a JSON schema where category is an enum of exactly four strings. The model cannot return anything else — it's constrained at the API boundary, not cleaned up afterwards.

The field order is a design decision

Look at the order: rationale, then category, then confidence.

That's not alphabetical or aesthetic. Structured outputs are generated in field order, so putting rationale first means the model has to name the decisive signal before it commits to a label. Put category first and the rationale becomes a post-hoc justification for a label it already picked — which is a different, and worse, computation.

It's a one-line change with a real effect on the reasoning, and it's the kind of thing that's invisible unless you're thinking about the generation order.

One class, two switches

Here's the classify path, with the ablation switches doing their work:

def classify(self, article: str) -> Classification:
    article = article.strip()
    if not article:
        raise ValueError("Article text is empty.")

    request = {
        "model": self.model,
        "max_output_tokens": MAX_OUTPUT_TOKENS,
        "input": [
            {"role": "system", "content": self._system},   # guide on/off
            *self._prefix,                                  # examples on/off
            {"role": "user", "content": article},
        ],
        "text_format": Classification,
    }
    ...

The four configurations live in one file as data, not as scattered if statements:

CONFIGS: list[PromptConfig] = [
    PromptConfig("zero_shot", "Zero-shot", True, False,
        "The full system prompt, with no worked examples. The baseline.", "zero-shot"),
    PromptConfig("full", "Few-shot", True, True,
        "The same prompt plus 10 worked examples. What this project ships.", "few-shot"),
    PromptConfig("minimal", "Zero-shot, bare prompt", False, False,
        "Category names only -- no boundary rules, no examples.", "zero-shot"),
    PromptConfig("examples_only", "Few-shot, bare prompt", False, True,
        "No boundary rules; the examples alone have to teach them.", "few-shot"),
]

The CLI, the JSON API and the browser UI all read this same list. Adding a fifth configuration means adding a tuple — not touching three call sites. This is the highest-leverage 20 lines in the repo.

The first experiment failed

I wrote six held-out boundary articles — deliberately not reused from the few-shot set, so the examples would have to generalise rather than be recalled — ran the comparison, and got this:

Perfect scores everywhere. Which looks like a triumph and is actually a broken experiment.

If both configurations score 100%, the measurement has no headroom. It can't tell me the examples helped; it can't tell me they didn't. It tells me nothing at all, and reporting "few-shot is 100%!" would have been a lie of omission.

Two causes, and it's worth separating them:

1. My eval articles were too easy. They straddled categories in theory, but the framing was clear enough that any competent prompt got them right.

2. The category guide was already doing the examples' job. This is the subtle one. CATEGORY_GUIDE explicitly states the boundaries — "a tech company's earnings is Business, not Technology" — which is exactly what the Nvidia example teaches. My baseline wasn't a naive prompt. It was a carefully engineered zero-shot prompt that had already absorbed the lessons. I'd optimised my control group into the ground.

That second point is a genuinely useful lesson about ablation studies: if your baseline already contains the treatment, you will measure nothing.

Fixing the experiment

Two changes.

1. A harder evaluation set — and it lives in a file, not in code

I wrote 12 articles engineered to remove the ceiling, built around three specific traps:

• Lede/body conflicts — the story opens in one category and the details live in another.

• Vocabulary borrowed from the wrong category — dense semiconductor jargon in an article that is fundamentally about a government rule-making action.

• Genuinely overlapping events — a broadcast-rights auction is honestly both a sports story and a business story; only the framing decides.

Each case carries a hard_because field, so a miss reads as "fell for the semiconductor vocabulary" rather than just a red cell:

{
  "text": "The Commerce Department added four more chip-design firms to its export blacklist on Wednesday, barring them from selling advanced lithography components abroad without a licence. ... Industry groups warned the measure would cost domestic suppliers an estimated $4 billion in annual revenue.",
  "expected": "Politics",
  "hard_because": "Dense semiconductor vocabulary and a revenue figure, but the article is a government rule-making action.",
  "source": null
}

Critically, the eval set is data, not code. It loads from JSON or CSV, resolved in this order: an explicit path → the NEWS_CLASSIFIER_CASES environment variable → the bundled default. Anyone can point it at real articles:

NEWS_CLASSIFIER_CASES=my_articles.csv uv run news-classifier-eval --hard

And there's one loader decision I'm particularly happy with. A broken user file must not stop the app from starting — but it must not silently fall back to synthetic data either, or you'd believe you had measured on real articles when you hadn't:

try:
    return load_cases(), None
except CaseSetError as exc:
    # fall back to the bundled set AND keep the error
    return load_cases(DEFAULT_PATH), str(exc)

That error then surfaces in the CLI, in /api/eval/config, and as a banner in the UI. The app runs; it just refuses to let you think you measured something you didn't.

2. Ablate the guide, not just the examples

Since the guide was masking the effect, I made it a variable too. That's the second switch, and it turns a single before/after into a 2×2:

Now I could ask two different questions:

• zero_shot vs full → do examples help a prompt that already explains the boundaries?

• minimal vs examples_only → can examples teach the boundaries on their own?

The second question is the one the first experiment couldn't reach.

The results

12 articles × 3 runs, per configuration:

Three findings.

Finding 1 — examples genuinely do teach the boundaries

Compare the bottom two rows. With no rules in the prompt at all, adding the examples moves accuracy from 67% → 75%. Nobody told the model that a chipmaker's earnings report is Business; it inferred that boundary from six worked demonstrations.

That's the few-shot contribution in isolation, and it's the cleanest result in the set.

Finding 2 — on top of a good prompt, examples buy consistency, not accuracy

Compare the top two rows. Accuracy is 83% either way — the examples didn't make it more correct. But agreement goes 89% → 94%: the same article gets the same label more often across repeated runs.

That's not a disappointment. That is precisely the thing the assignment asked me to improve. The guide had already taught the model what the right answer is; the examples made it stop wavering on the way there. Rules tell you where the boundary is. Demonstrations make you land on the same side of it twice in a row.

Finding 3 — the row that justifies the whole methodology

Look at "Neither": the highest agreement in the table (97%) and the lowest accuracy (67%).

The bare prompt is consistently wrong. It has settled into a confident, stable, incorrect habit. If I had only measured consistency — the literal ask — the winning configuration would have been the worst classifier in the experiment.

This is the entire argument for reporting both numbers everywhere, and it's why the failed first experiment was worth having. I'd have shipped a number that flattered me and meant nothing.

Caveat, stated up front: 12 articles × 3 runs is a small sample and a few points sit inside the noise. I'd treat the 67% → 75% accuracy gap as solid and the 89% → 94% agreement gain as suggestive — worth re-running at -n 10 before quoting as settled. Publishing the caveat alongside the result costs nothing and is the difference between a measurement and a marketing number.

Shipping it: three surfaces and a lab

The measurement is the interesting part, but it still had to be usable.

The CLI

uv run news-classifier --demo                        # four built-in samples
uv run news-classifier "Arsenal beat Spurs 2-0..."   # one article
uv run news-classifier -f article.txt                # from a file
cat article.txt | uv run news-classifier             # from stdin
uv run news-classifier --demo --json                 # JSON Lines

Sports       (confidence 0.99)
  why: A match report centred on the result and the scorer.

And the evaluation runner:

uv run news-classifier-eval -n 5             # shipped config
uv run news-classifier-eval -n 5 --compare   # few-shot vs zero-shot
uv run news-classifier-eval --hard           # hard set
uv run news-classifier-eval --ablate         # all four configurations

The JSON API

FastAPI, with Pydantic models on both request and response, and generated docs at /docs.

curl -X POST http://127.0.0.1:8000/api/classify \
  -H 'Content-Type: application/json' \
  -d '{"article": "The Senate voted 58-42 to advance the bill."}'

{"category":"Politics","confidence":0.99,
 "rationale":"A floor vote on a bill.","elapsed_ms":812}

Errors are real HTTP status codes, not a 200 with an error body: 400 empty article, 401 bad key, 422 failed validation, 429 rate limited, 502 upstream error, 503 no credentials.

The web UI, and the Consistency Lab

The main page is what you'd expect — paste an article, get a card with the category, a confidence meter and the rationale.

The button I actually care about is Compare prompts. It takes your article, runs it N times with the examples and N times without, everything else held identical, and leads with the delta:

Few-shot was more consistent: +25 pts agreement

           Configuration   Label     Agreement                Confidence
ZERO-SHOT  Zero-shot       Sports    50%  Sports×2, Tech×2     0.88
FEW-SHOT   Few-shot        Sports    75%  Sports×3, Tech×1     0.81

That's a real run on the camera-tracking article from the hard set. The zero-shot prompt was a coin flip on it. The few-shot prompt wasn't.

Two details I'd defend:

• When the runs disagree, the UI shows the dissenting label and its reasoning too. Showing only the winner hides the fact that the model wavered, which is the entire finding.

• When the delta is zero, it says so plainly. Unambiguous articles score 100% under both prompts. A tool that only reports wins isn't a measurement tool.

Then there's /eval — the Consistency Lab — which runs the full ablation in the browser as a matrix: one row per article, one column per configuration. Green for correct, red for wrong, and a striped cell when a configuration disagreed with itself between runs. Hovering shows the full label spread. Each row carries its hard_because.

Two implementation notes there. The browser drives the loop one article per request, so progress renders live and no single request approaches a serverless timeout. And both tables render before you run anything, with cells reading not run — so you can see exactly what's about to be compared, and the estimated API call count, before spending a rupee.

Five things that bit me

1. The reasoning parameter isn't universal

Reasoning models accept reasoning={"effort": "low"}. The gpt-4o family rejects it with a 400. The obvious fix is a hardcoded list of which models support it — which goes stale every release.

So instead: send it optimistically, catch that one specific rejection, drop it, and remember on the instance.

if self._supports_reasoning is False:
    response = self.client.responses.parse(**request)
else:
    try:
        response = self.client.responses.parse(
            **request, reasoning={"effort": REASONING_EFFORT})
        self._supports_reasoning = True
    except openai.BadRequestError as exc:
        if not _is_unsupported_reasoning(exc):
            raise                       # never swallow a real 400
        self._supports_reasoning = False
        response = self.client.responses.parse(**request)

Cost: one extra request on the first call, nothing after. The raise is the important line — it would be very easy to write a bare except BadRequestError here and silently swallow genuine errors like a malformed schema. There's a test that specifically asserts other 400s still propagate.

2. Reasoning tokens eat your output budget

I set max_output_tokens=256 at first. It's a classification — how many tokens could it need?

The answer is that reasoning tokens count against max_output_tokens. The model burned the budget thinking and got truncated before emitting any JSON. And the failure is quiet: response.output_parsed comes back as None rather than raising.

Fix: a 2048 ceiling, and an explicit error rather than a silent None:

parsed = response.output_parsed
if parsed is None:
    raise RuntimeError(
        f"Model returned no parsed result (status={response.status!r})."
    )

3. Serverless has a hard clock

Vercel caps a request at 60 seconds. One classification is 1–3s, which is fine. A batch of 25 run sequentially is ~40s and can time out. So batches fan out over a bounded thread pool:

workers = min(BATCH_CONCURRENCY, len(request.articles))
with ThreadPoolExecutor(max_workers=workers) as pool:
    results = list(pool.map(lambda a: _classify_one(classifier, a), request.articles))

pool.map preserves input order — which matters enormously when you're returning results the caller has to line up with their inputs, and which has its own test. Concurrency defaults to 5 to stay inside rate limits, overridable via BATCH_CONCURRENCY. A 25-article batch drops from ~25s to ~5s.

4. Error messages should say what to do

The common failures in an LLM app aren't bugs. They're an unset key, an empty billing account, or a rate limit — and each has a completely different fix. Generic error handling makes all three look identical.

if isinstance(exc, openai.RateLimitError):
    # OpenAI returns 429 for both throttling and an exhausted quota.
    message = getattr(exc, "message", str(exc))
    if "quota" in message.lower() or "billing" in message.lower():
        return ("The OpenAI account is out of quota. Add credit at "
                "https://platform.openai.com/settings/organization/billing")
    return "Rate limited by the OpenAI API. Wait a moment and retry."

That 429 disambiguation cost me twenty minutes of confusion the first time I hit it, so it's now permanently encoded.

Related: a missing API key does not crash the server. It's caught at startup, reported through /api/health, and returned as a 503 on classify — so the UI can say something useful instead of failing blank.

5. The src/ layout vs. the serverless cold start

The project uses a src/ layout, so news_classifier is only importable once the project itself is installed. If a build installs only dependencies, you get ModuleNotFoundError at cold start — in production, on a Friday.

_SRC = Path(__file__).parent / "src"
if _SRC.is_dir() and str(_SRC) not in sys.path:
    sys.path.insert(0, str(_SRC))

from news_classifier.api import app

Three defensive lines, and a test that imports app.py the way Vercel would.

One more Vercel gotcha worth flagging: don't copy the excludeFiles example from Vercel's docs verbatim. It excludes static/**, which would strip the entire UI out of your bundle.

Testing an LLM app without an API key

65 tests, 4.4 seconds, no network, no key. A stub classifier stands in for the model everywhere.

class StubClassifier:
    """Records what it was asked and returns a fixed answer."""

The tests I'd point at in a code review aren't the "does it return a value" ones — they're the ones pinning down properties the design depends on:

• the few-shot prefix is byte-stable across calls — the premise of the caching claim

• every example's assistant turn still parses back into the current schema (so changing the schema can't silently invalidate the examples)

• no hard case is reused from the few-shot set — so the eval measures generalisation, not recall

• concurrent batching preserves input order

• the reasoning capability is probed exactly once per instance, and other 400s aren't swallowed

• the rationale displayed next to a label comes from a run that actually produced that label (with N runs and disagreement, it's easy to show a mismatched pair)

• app.py imports without the package installed — the Vercel path, tested locally

The unifying idea: test the invariants your design leans on, not just the happy path. Every one of those would rot silently otherwise.

What this does not prove

Every article in the repo is synthetic. The 10 few-shot examples, the 12 hard cases, the 6 first-pass eval articles, the 5 UI samples — all written for this project. None is real journalism; names, figures and events are invented.

That was deliberate: an article that reliably puts its lede in one category and its body in another isn't something you find on demand, and the hard set needs exactly that to discriminate. But it costs something, and the cost should be named:

• The same person wrote the examples and the eval set, so they're stylistically related. A test forbids literal reuse, but stylistic correlation isn't something a test can rule out. The examples may be unfairly well-matched to these cases.

• So the measured gain is a lower-confidence result than the same measurement on independently sourced articles. It should be reported as "measured on a synthetic benchmark", not as a general claim about news classification.

Which is exactly why the eval set is data and not code, and why the synthetic: true flag travels with the file, through the API, and into a banner above the results in the UI. The weakness is carried in the payload rather than buried in a README, because the person reading the numbers is the person who needs to know.

Swapping in real articles is one environment variable. The plumbing to do it honestly is already built.

What I'd take to the next project

"Improve X" is a request for two numbers, not one. The baseline is half the deliverable. If you can't switch your improvement off, you can't claim it works.

Ablate more than one variable. My first experiment measured nothing because my baseline had already absorbed the treatment. The 2×2 rescued it — and the extra cell answered a question I hadn't thought to ask.

A metric alone can be gamed by accident. 97% agreement at 67% accuracy is a perfectly consistent, perfectly wrong classifier. Consistency needed correctness next to it to mean anything.

Constrain the output before you tune the prompt. An enum-typed schema deleted a whole class of inconsistency that no amount of prompt-wording would have fully fixed.

Field order in a structured output is a reasoning decision. rationale before category makes the model think before it commits. One line; real effect.

A failed experiment that you diagnose is worth more than a passing one you don't understand. The 100%-everywhere run was the most useful thing that happened in this build.

The classifier ships as guide + few-shot — the configuration that scored 94% agreement and 83% accuracy. But the switch to turn the examples off is still there, still tested, still exposed in the CLI, the API and the UI. Anyone who doubts the claim can re-run the measurement on their own articles, and that's rather the point.

This post walks through an Advanced RAG Course Assistant — an app that answers questions about a video course using its .vtt transcripts, and cites the exact module, lesson, and timestamp for every claim. It routes across three data sources, transforms queries with HyDE, reranks with an LLM, and — the part I care about most — verifies its own citations with a check that doesn't trust the model at all.

The stack: Next.js (App Router) + TypeScript, OpenAI for embeddings and chat, Qdrant as the vector store, and an embedded SQLite DB standing in for an auth service.

Let's get into it.

The mental model: one scoped LLM call per stage

The single most important design decision in this project is that every stage of the pipeline is its own small, focused LLM call — not one giant prompt trying to do everything.

Here's the pipeline a single query flows through:

1. Input guardrail — is this on-topic and safe?

2. Query router — which data source(s) does this need?

3. HyDE — generate a hypothetical answer to search with

4. Rerank — score retrieved chunks against the real question

5. Text-to-SQL — (if needed) turn account questions into read-only SQL

6. Generate answer — final response with forced citation format

7. Output guardrail — verify every citation is real (no LLM call)

Here's the whole flow, including the corrective-retry loop and the branches:

Each of stages 1–6 is a separate prompt in its own file under lib/prompts/. Why not one mega-prompt? Because:

• Every call pays for every instruction. A guardrail classification shouldn't carry SQL rules and citation formats it will never use.

• Instructions bleed into each other. The model doing reranking gets distracted by citation rules; the guardrail starts trying to answer.

• Different calls want different output shapes — {"allowed": bool}, {"sources": [...]}, {"scores": [...]}, raw SQL. One prompt can't cleanly demand all of them.

• You can tune one stage without risking the others.

The tradeoff is more round-trips (5–7 sequential calls per query), which adds latency. For a RAG pipeline where correctness and citation integrity matter more than shaving 500ms, that's the right bet.

Key insight: the LLM never sees your file structure. Splitting prompts into files is code organization; the benefit lands at the call level — each call gets only the instructions it needs.

The three data sources

A query router decides which source(s) a question needs, so branches that aren't relevant never run:

• vector-store (Qdrant) — real transcript chunks. The default for almost every question.

• auth-db (SQLite) — student account data (enrollment, lesson progress). The LLM translates the question into SQL (text-to-SQL), which runs read-only.

• s3 (mock) — lesson attachments / downloadable resources, backed by a static manifest.

A single question can hit more than one source — "have I finished the lesson with the OAuth starter code, and can I get that file?" routes to both auth-db and s3.

The router prompt is deliberately opinionated about defaults:

// lib/prompts/queryRouter.ts
const system = `You are a query router for a course assistant. Decide which data
source(s) are needed to answer the student's question:

- "vector": the question is about course CONTENT — concepts, code, how something
  works. This is the default for almost all questions.
- "auth-db": the question is about the STUDENT'S OWN ACCOUNT — their progress...
- "s3": the question is asking to DOWNLOAD a FILE — attachments, starter code...

Respond ONLY with strict JSON, no markdown:
{"sources": ["vector" | "auth-db" | "s3", ...]}`;

Notice a design pattern that recurs throughout: fail-safe defaults chosen per stage. The router fails open — if parsing breaks, it defaults to ["vector"] (still try to answer). The input guardrail, as we'll see, fails closed.

HyDE: search with a fake answer, not the question

Here's a problem that quietly wrecks retrieval quality. A user's question and the transcript that answers it are written completely differently:

• Question: "How do I save data between app launches?"

• Transcript: "So we import AsyncStorage, call setItem with a key and a value, and that persists it to disk so when you reopen the app it's still there…"

Embedding the bare question and matching it against answer-style transcript vectors is an asymmetric search. Similarity is often weak.

HyDE (Hypothetical Document Embeddings) fixes this: instead of embedding the question, we first ask an LLM to write a plausible answer — in the voice of the course — and embed that. Now both sides of the search look like answers.

// lib/prompts/queryTransform.ts
const system = `You are an expert instructor for a course on: ${courseTopic}.
Given a student's question, write a short, plausible answer (3-5 sentences)
as if it came directly from the course transcript. Use natural spoken,
instructional language similar to how an instructor explains things out loud.
Do not hedge or say "I don't know" — just write your best hypothetical
explanation. This is used only to improve search retrieval, not shown to the user.`;

Two implementation details make this safe:

1. The HyDE output is used only for embedding — never for scoring. The vector branch passes the hypothetical answer as the search input, but the original question goes to the reranker:

// lib/rag/pipeline.ts
const hydePrompt = buildHydePrompt(query, COURSE_TOPIC);
searchInput = await callLLM(hydePrompt.system, hydePrompt.user);
relevantChunks = await retrieveAndRerank(searchInput, query);
//                                         ↑ HyDE       ↑ real question

So a hallucinated HyDE answer can only affect which candidates get pulled — not which ones survive. The reranker is the safety net.

2. Corrective retry deliberately drops HyDE. If the HyDE-driven search returns nothing that clears the relevance bar, the loop retries with the raw question:

if (attempt === 0) {
  searchInput = await callLLM(hydePrompt.system, hydePrompt.user); // HyDE
} else {
  searchInput = query; // corrective retry: fall back to the literal question
}

The prompt tells the model to write in spoken, instructional language and to never hedge — because the goal isn't a correct answer, it's a transcript-shaped one. Vocabulary matters more than accuracy; it's competing on cosine similarity against an actual spoken transcript.

Reranking: similarity is a rough filter

Vector search gives you the top-K by cosine similarity, but similarity ≠ relevance. So we retrieve the top 10, then have an LLM score each candidate 0–10 against the original question and keep only the top 4 scoring ≥ 5:

// lib/prompts/rerank.ts
const system = `You score how relevant each transcript excerpt is to a student's
question, on a scale of 0-10. 10 = directly and completely answers the question.
0 = completely unrelated.

Respond ONLY with strict JSON:
{"scores": [{"index": number, "score": number}, ...]}`;

If nothing clears the threshold, that's what triggers the corrective retry mentioned above. This is "corrective RAG" in its simplest, most useful form.

Text-to-SQL, where the prompt is not the security boundary

For account questions, the LLM translates natural language into a SQL query against the SQLite schema:

// lib/prompts/textToSql.ts
const system = `You translate a student's question into a single READ-ONLY SQL
query (SQLite dialect) against this schema:

${AUTH_DB_SCHEMA}

Rules:
- Only ever generate a SELECT statement. Never INSERT/UPDATE/DELETE/DROP/ALTER.
- Always scope results to the current student: student_id = ${studentId}...`;

Critical point: that prompt is not the security boundary. It just asks nicely. The real enforcement is in code — runReadOnlyQuery() hard-blocks anything that isn't a SELECT, and studentId is injected server-side so one student can never query another's data. Prompts request behaviour; code enforces it. Never confuse the two.

The output guardrail that doesn't trust the model

This is my favourite part of the whole design.

The generation prompt forces a rigid citation format:

(Module: <moduleName>, Lesson: <lessonName>, Timestamp: <startTimestamp>)

Because the format is rigid, we can verify citations with pure regex + a set lookup — no LLM call. The output guardrail pulls every citation out of the answer and checks each against the chunks that were actually retrieved. If the model cited a timestamp that wasn't in the context, it's flagged and a single repair pass is triggered.

// lib/prompts/outputGuardrail.ts
const validSet = new Set(
  usedChunks.map((c) => `${c.moduleName}|${c.lessonName}|${c.startTimestamp}`)
);
// ...for each citation found in the answer, check it's in validSet

This is the single most important anti-hallucination check in the pipeline, and it costs nothing. It catches the worst failure mode — an invented timestamp — deterministically.

A real bug this surfaced

While building the UI, I noticed the model was citing ranges (00:10:10-00:10:53) and sometimes grouping multiple citations in one parenthetical separated by ;. The original guardrail regex was anchored on a single timestamp immediately followed by ):

// BEFORE — misses ranges and grouped citations
/\(Module:\s*(.+?),\s*Lesson:\s*(.+?),\s*Timestamp:\s*(\d{2}:\d{2}:\d{2})\)/g

That regex silently matched zero citations on range-formatted answers — meaning the guardrail wasn't verifying anything on exactly the answers it was supposed to protect. The fix matches each citation entry independently (not anchored on )) and tolerates an optional range, capturing the start timestamp for validation:

// AFTER — matches single OR range timestamps, and grouped entries
/Module:\s*(.+?),\s*Lesson:\s*(.+?),\s*Timestamp:\s*(\d{2}:\d{2}:\d{2})(?:\s*[-–]\s*\d{2}:\d{2}:\d{2})?/g

Lesson: a guardrail that silently matches nothing is worse than no guardrail, because it looks like it's working. Always test the negative case — I verified a fabricated 09:99:99 still gets flagged.

Bringing ingestion into the browser

Ingestion started as a CLI script: parse every .vtt, embed in batches of 50, upsert to Qdrant. That's fine for a one-time setup, but I wanted to trigger it from the frontend.

The trick is realising the parser splits cleanly into two halves:

• Content parsing (parseVTT + chunkCues) works on a raw string — no filesystem needed.

• Metadata (moduleName, lessonName) comes from the folder structure — the file's parent folder is the lesson, the grandparent is the module.

So I extracted a shared helper both the CLI and the API route use:

// lib/parser.ts
export function chunksFromVtt(
  raw: string, moduleName: string, lessonName: string, sourceFile: string
): TranscriptChunk[] {
  const cues = parseVTT(raw);
  if (cues.length === 0) return [];
  return chunkCues(cues).map((chunk) => ({ moduleName, lessonName, ...chunk, sourceFile }));
}

Then two upload modes in the UI, hitting one POST /api/ingest:

• Folder / multi-file — pick whole course folders. Module & lesson are derived automatically from each file's webkitRelativePath.

• Single file — upload one .vtt and tag it manually (a lone file has no folder context).

One gotcha worth knowing: browsers drop folder paths when files hit FormData — the server only sees the basename. So the client sends each file's relative path alongside it, and the server derives module/lesson from that:

for (const f of folderFiles) {
  form.append("files", f);
  form.append("paths", relPath(f)); // webkitRelativePath, preserved explicitly
}

And because webkitdirectory only lets you pick one folder per dialog, I made selections accumulate across picks (deduped by relative path) so you can add multiple courses before ingesting.

Nicer citations in the UI

Raw model output dumps citations inline as noisy parentheticals:

Expo is recommended as the best way to start your mobile journey (Module: module 1, Lesson: 02_react-native-vs-expo, Timestamp: 00:10:10-00:10:53; Module: module 1, ...).

The UI now strips those out of the prose and turns them into numbered superscript chips (¹ ²) that map to a de-duplicated source list rendered as cards below the answer — module badge, lesson name, and a monospace timestamp pill. The prose stays readable; the evidence is one glance away.

When (and when not) to reach for Zustand

The app grew enough state that it's worth asking: do we need a state-management library?

My honest answer for most of the build was no. Zustand solves shared state across distant components and prop-drilling. If your state is local and shallow — useState in the component that owns it — a store is just indirection and a dependency you don't need. Classic YAGNI.

But one real need did emerge: when an ingest was running and you closed the ingest panel, the component unmounted and the running status was lost. That's a genuine state-sharing problem. The fix is either "lift state up" or a small store.

I went with a Zustand store because the app is set to grow (auth, more pages, persisted history), and putting the async actions in the store means an in-progress ingest survives the panel unmounting:

// lib/store.ts
export const useAppStore = create<AppState>((set, get) => ({
  studentId: 1,
  messages: [],
  chatLoading: false,
  sendQuery: async (query) => { /* fetch + set(...) */ },

  folderFiles: [],
  ingestStatus: { kind: "idle" },
  ingestFolders: async () => { /* build FormData, POST, set(...) */ },
}));

Now closing the panel mid-ingest keeps it running; reopening shows live status. Reach for a store when state is genuinely shared — not before.

Gotchas from the trenches

A couple of things that cost me time, in case they save you some:

• better-sqlite3 + bleeding-edge Node. On Node 25 (a non-LTS "Current" release) there were no prebuilt binaries, so it fell back to compiling from source and failed with node-gyp: command not found. Fix: npm rebuild better-sqlite3 (npm bundles its own node-gyp) once you have the compiler toolchain. Or just use Node 22 LTS and get prebuilds.

• One COURSE_TOPIC constant drives three prompts. The input guardrail, HyDE, and router all reference it. If it's wrong, the guardrail blocks legitimate questions and HyDE generates the wrong domain's vocabulary — silently tanking retrieval. It's the highest-leverage line in the codebase.

• Ingestion upserts, it doesn't replace. Re-ingesting the same transcripts creates new point IDs (UUIDs) rather than overwriting — so you get duplicates. Clear the collection first or dedup on a stable ID if that matters.

Takeaways

The through-line of this whole design:

• One scoped LLM call per stage beats one mega-prompt — cheaper, sharper, independently tunable.

• Fail-safe defaults, chosen per stage — the guardrail fails closed, the router fails open.

• HyDE improves recall but never gets to judge — the reranker uses the real question.

• The things that actually protect you are enforced in code, not prompts — read-only SQL and deterministic citation verification.

• Add libraries (like Zustand) when a real need appears — not preemptively.

RAG isn't "embed and pray." It's a pipeline of small, verifiable steps — and the most important step is the one that assumes the model is lying and checks its work.

Built with Next.js, TypeScript, OpenAI, and Qdrant. If you found this useful, drop a comment about which stage you'd tune first.

This article walks through what RAG is, how it works, where it shines, and — more importantly — where and why it breaks down in practice.

1. The Problem: LLMs Without External Knowledge

A large language model is trained on a fixed snapshot of data, up to a certain cutoff date. Once training is done, that knowledge is frozen. This creates two obvious problems:

• Outdated information: The model has no idea about anything that happened after its training cutoff — new product launches, recent news, updated pricing, or company-specific data.

• No private/internal knowledge: An LLM has never seen your company's internal wiki, your product documentation, or your customer database. If you ask it something specific to your business, it will either say "I don't know" or — worse — confidently make something up. This second failure mode is what we call hallucination: the model generates a plausible-sounding but incorrect or fabricated answer, because it's trained to produce fluent text, not to verify facts.

So the question becomes: how do we give an LLM access to information it was never trained on, without retraining the entire model every time something changes?

That's where RAG comes in.

2. What RAG Is and Why It Was Introduced

Retrieval-Augmented Generation (RAG) is a technique where, instead of relying purely on what the model "remembers" from training, we fetch relevant information from an external knowledge source at the moment a question is asked, and hand that information to the model as context before it generates a response.

Think of it like an open-book exam versus a closed-book exam. A model without RAG is answering from memory alone. A model with RAG gets to look up the relevant page in the textbook first, and then answer using both its own reasoning and the material it just read.

RAG was introduced to solve two things at once:

1. Freshness — you can update the external knowledge source (documents, database, wiki) without retraining the model.

2. Grounding — answers can be tied back to real source material instead of purely generated from the model's internal weights.

3. How a Basic RAG Pipeline Works

At a high level, a simple RAG pipeline looks like this:

User Query → Retrieval → LLM → Response

Breaking that down into steps:

Step 1: Chunking and Indexing (done ahead of time)

Your documents (PDFs, wiki pages, support tickets, etc.) are split into smaller pieces called chunks. Each chunk is converted into a vector (a numerical representation of its meaning) using an embedding model, and stored in a vector database.

Step 2: Query Embedding

When a user asks a question, that question is also converted into a vector using the same embedding model.

Step 3: Retrieval

The system searches the vector database for chunks whose vectors are most "similar" to the query vector — essentially, chunks that are semantically close to what the user asked.

Step 4: Augmentation

The retrieved chunks are inserted into the prompt sent to the LLM, usually with instructions like "answer the question using only the following context."

Step 5: Generation

The LLM reads the query plus the retrieved context and generates a response, ideally grounded in that context rather than pure guesswork.

Diagram idea: A simple left-to-right flow diagram showing User Query → Retrieval (Vector DB) → LLM → Response, with a side box showing the offline "Chunking + Indexing" step feeding into the vector DB.

4. Common Scenarios Where RAG Works Well

RAG tends to shine when:

• You have a large, relatively stable knowledge base — internal documentation, product manuals, legal policies, onboarding guides.

• The answer exists explicitly in your documents — e.g., "What is our refund policy?" where the policy is written down word-for-word somewhere.

• You need source attribution — customer support bots that need to say "here's what our docs say" rather than inventing an answer.

• The domain is narrow and well-defined — a RAG system built only on your company's HR policies will perform far better than a generic one built on the entire internet. A simple real-world example: imagine a customer support bot for a software product. Without RAG, if a user asks "How do I reset my API key?", the model might hallucinate a plausible-but-wrong set of steps. With RAG, the system retrieves the actual support doc describing the reset flow, and the model just needs to summarize or rephrase it — a much easier and more reliable task.

This is where RAG's value is obvious: retrieval turns "guessing an answer" into "summarizing a known answer."

5. Why RAG Sometimes Gives Incorrect Answers

Here's the important nuance beginners often miss: RAG improves the odds of a correct answer, but it does not guarantee correctness. It's a pipeline of multiple steps, and each step can introduce errors that compound by the time you reach the final response.

Let's go through the major failure points.

5.1 Poor Retrieval and Missing Context

The entire pipeline depends on retrieving the right chunks. If retrieval fails, everything downstream fails too — the LLM can only work with what it's given.

Retrieval can go wrong when:

• The user's query is phrased very differently from how the answer is written in the source documents (a vocabulary mismatch).

• The embedding model doesn't capture the true semantic meaning of niche or technical terms.

• The similarity search returns chunks that are topically related but don't actually contain the specific answer. Example: A user asks "Why was my payment declined?" but the actual document talks about "transaction failure reasons." If the retrieval system doesn't recognize these as related, it might pull the wrong section entirely — and the model will either answer incorrectly or say it doesn't know, even though the answer exists in the knowledge base.

Diagram idea: Side-by-side comparison — "Good Retrieval" showing the query vector landing close to the correct chunk, versus "Poor Retrieval" showing the query vector landing near unrelated chunks.

5.2 Poor Chunking and Its Impact on Responses

How you split documents into chunks matters enormously.

• Chunks that are too small may cut a sentence or idea in half, losing important context. A step in a process might get separated from the condition that triggers it.

• Chunks that are too large may dilute the relevant information with a lot of irrelevant surrounding text, making it harder for the retrieval step to match it accurately, and harder for the LLM to focus on what matters.

• Bad chunk boundaries can also separate a heading from its related content, or split a table in the middle — leading to answers that are technically retrieved but practically useless. Example: A troubleshooting guide might have a numbered list of steps. If chunking splits step 3 from step 4, the retrieved chunk might describe the problem and the first fix, but miss the crucial follow-up step — leading to an incomplete or misleading answer.

Diagram idea: A document shown as one long block of text, with one version chunked cleanly along logical sections, and another version chunked arbitrarily by character count, cutting through the middle of ideas.

5.3 Context Window Limitations

Every LLM has a maximum context window — a limit on how much text (query + retrieved chunks + system instructions) it can process at once.

Problems arise when:

• Too many chunks are retrieved, and they don't all fit within the context window, so some get truncated or dropped entirely.

• Important information ends up "buried in the middle" of a long context, a well-documented phenomenon where models pay more attention to the beginning and end of their input than the middle.

• Long contexts increase the chance of the model blending or confusing details from multiple retrieved chunks, especially when several chunks discuss similar topics. This means simply retrieving "more" context isn't always better — a large context window filled with tangentially related material can dilute the model's focus.

Diagram idea: A horizontal bar representing the context window, with segments for "System Prompt," "Retrieved Chunks," and "User Query" — showing how the retrieved chunks can overflow or get cut off if too many are stuffed in.

5.4 Hallucinations Even With RAG

A common misconception is that RAG eliminates hallucinations entirely. It doesn't.

Even with correct, relevant context provided, an LLM can still:

• Blend facts from multiple chunks incorrectly, creating a plausible-sounding but wrong combination of details.

• Add details that weren't in the source at all, especially when asked to "elaborate" or "explain further."

• Ignore the provided context and fall back on its training data if the retrieved information seems incomplete or contradicts what it "learned" during training. RAG reduces hallucination by giving the model something concrete to ground its answer in — but the model is still a text generator at its core, not a fact-checker. It can misread, misinterpret, or over-extend the context it's given.

5.5 Keeping Knowledge Bases Up to Date

RAG solves the "frozen training data" problem only if the external knowledge base itself is kept current. In practice, this is an ongoing operational challenge:

• Documents get updated, but the vector index doesn't automatically refresh — someone has to re-run the embedding and indexing pipeline.

• Outdated or contradictory documents may still be sitting in the knowledge base (e.g., an old pricing page next to a new one), and retrieval might pull the stale one.

• As your knowledge base grows, re-indexing becomes more expensive and needs a proper pipeline, not a manual one-off process. If your retrieval source is stale, RAG doesn't just fail to help — it can actively mislead, because the model will confidently answer using outdated "facts" it was explicitly told to trust.

6. When RAG Is Not the Right Solution

RAG isn't the answer to every problem. It tends to be a poor fit when:

• The task requires reasoning or computation, not lookup — e.g., mathematical problems, logic puzzles, or multi-step planning. Retrieval doesn't help if there's no document containing the "answer."

• The knowledge base is very small — if your entire knowledge fits comfortably in a prompt, you may not need retrieval at all; just include it directly.

• You need guaranteed, deterministic answers — RAG still involves a generative model at the final step, so exact, ruleset-driven tasks (like calculating tax slabs or validating a form) are better handled with plain code or rules engines, not an LLM.

• Your data changes so quickly that indexing can't keep up — like live stock prices or real-time system status, where a direct API call is far more reliable than a retrieval step.

• The question requires combining and reasoning over many documents at once — basic RAG retrieves a handful of chunks; it isn't designed for tasks like "summarize everything we know across 500 documents," which usually needs a different architecture (e.g., multi-step retrieval, agents, or map-reduce style summarization).

7. Summary: When RAG Helps, and What to Watch For

RAG is a powerful pattern for connecting LLMs to external, evolving knowledge without retraining the model. It's especially useful for narrow, document-grounded question answering where the answer already exists somewhere in your data.

But it's not a silver bullet. Its reliability depends on a chain of steps — chunking, embedding, retrieval, context assembly, and generation — and a weakness at any one of these stages can lead to an incorrect or incomplete answer. Understanding these failure points isn't just academic: it's what separates a RAG system that "sort of works in the demo" from one that's actually trustworthy in production.

Key takeaway: RAG improves the odds of a correct, grounded answer. It does not guarantee correctness. Treat it as one tool in the toolbox — not a replacement for good data hygiene, careful chunking strategy, and realistic expectations about what retrieval can and can't do.

A beginner-friendly deep dive into LLMs, tokenization, and Transformers — GenAI with JS 2026

You type a question into ChatGPT, hit enter, and within seconds you get a thoughtful, human-like answer. Ever wondered what actually happens between your keystroke and that response? Spoiler: there's no tiny human typing back, and the answer isn't copy-pasted from Google. It's a fascinating pipeline of math, probability, and pattern recognition.

Let's break it down, step by step.

1. What is an LLM?

LLM stands for Large Language Model.

At its core, an LLM is a program trained on massive amounts of text — books, articles, code, websites — to learn the patterns, structure, and relationships between words in human language. It doesn't "know" facts the way a database does; it learns to predict what word (or piece of a word) is most likely to come next, given everything that came before it.

What problems do LLMs solve?

• Understanding unstructured text — humans write messy, ambiguous sentences. LLMs learn to make sense of that.

• Generating human-like text — writing emails, code, summaries, explanations.

• Bridging the gap between natural language and machines — instead of learning a rigid command syntax, you can just talk to the system.

• Scaling knowledge work — tasks like summarising, translating, or explaining that used to need a human expert can now be assisted by a model.

Popular examples of LLMs

• GPT (OpenAI) — powers ChatGPT

• Claude (Anthropic)

• Gemini (Google)

• LLaMA (Meta)

• Mistral

Common applications in daily life

• Chatbots and virtual assistants (ChatGPT, Claude, Siri-like tools)

• Code autocompletion (GitHub Copilot)

• Email and document summarisation

• Language translation

• Content writing and brainstorming

• Customer support automation

2. What Happens When You Send a Message to ChatGPT?

Let's trace the journey of a single prompt.

flowchart LR
    A[You type a prompt] --> B[Message is processed]
    B --> C[Model generates a response]
    C --> D[Response streamed back to you]

Step 1: Typing a prompt

You write something like "Explain gravity to a 10-year-old." This is plain text — nothing special yet.

Step 2: Processing your message

Your text is sent to the model, converted into a numerical format the model can actually work with (more on this in the next section), and combined with the ongoing conversation history — this is called context.

Step 3: Generating a response

The model doesn't "look up" an answer. It predicts the response one token at a time, each new token chosen based on everything before it (your prompt + the tokens it has generated so far), continuing until the answer is complete.

Step 4: Why responses aren't copied from the internet

This is one of the most misunderstood parts of LLMs. The model isn't searching the web or pasting from a stored article. During training, it adjusted billions of internal parameters (weights) based on patterns it saw in text. When generating a response, it's essentially doing a very sophisticated version of "what word statistically makes sense next," shaped by everything it learned — not retrieving a saved document. That's also why LLMs can occasionally make mistakes or "hallucinate" — they're generating plausible text, not fetching verified facts.

3. Why Computers Don't Understand Human Language

Here's the uncomfortable truth: computers don't understand words at all.

Text vs numbers

Computers are fundamentally number-crunching machines. Every operation — addition, comparison, storage — happens on numbers (well, technically, binary). The word "hello" means nothing to a CPU; it's just a sequence of characters until it's converted into something numeric.

Why computers need everything converted into numbers

To make language usable for a neural network, every word, symbol, or piece of text has to be turned into numbers the model's math can operate on — this includes multiplying, comparing, and adjusting values across billions of parameters. You can't run matrix multiplication on the word "banana." You can run it on [0.23, -0.91, 1.42, ...].

Introduction to tokens

This is where tokens come in — the bridge between human language and machine-readable numbers. Instead of converting whole words directly, models break text into smaller chunks called tokens, and each token gets mapped to a number (and eventually a rich numerical representation called an embedding).

4. Tokenization

What tokens are

A token is a small chunk of text — it could be a whole word, part of a word, a single character, or even punctuation. Tokenization is the process of splitting text into these chunks.

Why tokenization is needed

Human language has an almost infinite number of possible words, especially once you count variations, misspellings, and made-up terms. If a model tried to treat every unique word as a single unit, its vocabulary would be enormous and inefficient. By breaking words into smaller, reusable sub-word pieces, the model can:

• Handle rare or unseen words by breaking them into familiar pieces

• Keep the vocabulary size manageable

• Represent any language, code, or symbol using a shared set of building blocks

Words vs tokens

A common assumption is "1 word = 1 token" — but that's often wrong. On average, 1 token ≈ 4 characters or roughly ¾ of a word in English.

flowchart LR
    T["'Tokenization is powerful'"] --> S["Token, ization, is, power, ful"]

Simple examples

This is also why LLMs have a context window — a maximum number of tokens (prompt + conversation history + response) they can process at once.

flowchart TB
    subgraph Context Window
    direction LR
    P[Your Prompt] --> H[Conversation History] --> R[Model's Response]
    end

If a conversation grows too long and exceeds this token limit, older parts of the conversation get dropped or summarized — which is why very long chats can sometimes cause a model to "forget" earlier details.

5. Transformers

What a Transformer is

The Transformer is a neural network architecture introduced in the 2017 paper "Attention Is All You Need." It's the engine underneath nearly every modern LLM, including GPT (the "T" in GPT literally stands for Transformer).

Why it changed AI

Before Transformers, models processed text mostly in sequence — one word after another — which made it hard to capture relationships between words that were far apart in a sentence, and slow to train at scale. Transformers introduced a mechanism called self-attention, which lets the model look at all the words in a sentence at once and figure out how much each word should "pay attention to" every other word.

How it helps understand language

Consider: "The trophy didn't fit in the suitcase because it was too big."

What does "it" refer to — the trophy or the suitcase? Self-attention allows the model to weigh the relationship between "it" and both candidate words, and lean toward the correct one based on patterns learned from massive amounts of text. This is how Transformers capture context, ambiguity, and long-range relationships in language — something earlier architectures struggled with.

Why almost every modern LLM uses Transformers

• Parallelization — unlike older sequential models, Transformers can process entire sequences at once, making training on huge datasets dramatically faster.

• Better long-range understanding — self-attention captures relationships regardless of how far apart words are.

• Scalability — Transformers scale remarkably well with more data and compute, which is a big reason today's LLMs keep getting more capable as they grow.

Bonus: Temperature — Controlling Creativity

One more concept worth knowing: temperature controls how "safe" or "creative" a model's word choices are.

Low temperature is great for factual tasks (coding, math); high temperature is great for creative writing (poems, brainstorming).

Putting It All Together: The High-Level Workflow

flowchart TB
    A[You type a prompt] --> B[Text is broken into Tokens]
    B --> C[Tokens converted to numerical Embeddings]
    C --> D[Transformer processes Embeddings using Self-Attention]
    D --> E[Model predicts the next Token, one at a time]
    E --> F[Tokens are converted back into readable text]
    F --> G[Response streamed back to you]

Wrapping Up

When you ask ChatGPT a question, you're not talking to a search engine or a database — you're interacting with a neural network that:

1. Breaks your words into tokens

2. Converts those tokens into numbers

3. Runs them through a Transformer that uses self-attention to understand context

4. Predicts a response one token at a time based on learned patterns

It's less "magic" and more elegant math — but understanding these fundamentals is the first real step toward building with LLMs instead of just using them.

You don't just write navigation code. You make decisions that every screen, every deep link, and every new team member will live with.

Ask ten React Native developers which navigation library to use, and you'll get ten passionate opinions. Some swear by React Navigation. Others won't start a new project without Expo Router. A few will confidently tell you they're "basically the same thing" — and they're half right.

This article isn't a fanboy piece for either side. It's a practical breakdown — from beginner to production scale — of when each approach wins, where each approach hurts, and how to make the right call for your specific situation in 2026.

Let's start at the beginning.

1. What Routing Means in Mobile Applications

Routing Is Not Just "Going to Another Screen"

In web development, routing is straightforward: the URL changes, the browser renders a different page. The browser handles history, the back button, deep links — all of it.

In mobile, you own all of that.

There's no address bar. No browser history stack. No automatic deep link handling. When a user taps a notification and expects to land on a specific chat room inside your app, you write the code that makes that happen. When they press the Android back button, you decide what happens. When they switch apps and come back, you manage the state they return to.

Routing in React Native means:

• Managing a stack of screens (which screens are "open" in memory)
• Handling transitions between screens (how screens appear and disappear)
• Parsing deep links (a URL that opens a specific screen)
• Protecting routes (making sure unauthenticated users can't reach the dashboard)
• Preserving state (the feed scroll position survives a tab switch)

This is what navigation libraries exist to solve. And the two most important ones in the React Native ecosystem — React Navigation and Expo Router — solve it very differently.

2. Why Navigation Is Important in React Native Apps

Navigation Is Architecture

Most developers think of navigation as a feature — something you "add" to an app. In reality, navigation is the skeleton of your app. Every screen, every user flow, every deep link depends on it. The choices you make early in navigation setup ripple through your entire codebase.

Get it wrong and you'll face:

• Deep link nightmares — adding deep links to an existing app built without them in mind can take days.
• Auth guard spaghetti — protecting routes across a manually configured navigator becomes a maze of conditionals.
• Navigation prop drilling — passing navigation as a prop through 4 layers of components to make one button work.
• Onboarding new developers slowly — a new team member needs to read a 300-line navigation config file before they understand how the app flows.

Get it right and navigation becomes invisible. Screens just work. Deep links just open. Auth protection just applies. New developers can read the folder structure and understand the app.

3. Brief History of React Navigation

The Standard Bearer Since 2017

React Navigation was created in 2017 to solve a real problem: React Native's built-in navigation (NavigatorIOS, Navigator) was platform-specific, buggy, and being deprecated. The community needed a JavaScript-first, cross-platform navigation library.

React Navigation quickly became the de facto standard. It offered:

• Stack navigators (push/pop screens)
• Tab navigators (bottom tab bar)
• Drawer navigators (side menu)
• Deep link configuration
• A flexible, composable API

By 2020, React Navigation v5 introduced a component-based API that made composing navigators much cleaner. v6 (2021) refined the API further. Today, React Navigation v7 is mature, battle-tested, and powers thousands of production apps — including apps at companies you use daily.

It is still the most widely used navigation library in React Native. That matters.

The Mental Model: Declarative Navigator Trees

React Navigation's model is: you declare navigators as components, nest them, and configure screens as children.

// The React Navigation way
function AppNavigator() {
  return (
    <NavigationContainer>
      <Stack.Navigator>
        <Stack.Screen name="Home" component={HomeScreen} />
        <Stack.Screen name="Profile" component={ProfileScreen} />
        <Stack.Screen name="Settings" component={SettingsScreen} />
      </Stack.Navigator>
    </NavigationContainer>
  );
}

Clear, explicit, and powerful. You see exactly which screens exist and how they're connected — all in one place.

4. Problems Developers Faced With Traditional Navigation Setup

The Boilerplate Accumulation Problem

React Navigation is powerful, but power comes with ceremony. As your app grows, your navigation file grows with it — and it grows fast.

Here's what a moderately complex app's navigation setup looks like:

// This is a SIMPLIFIED version. Real apps are 3x longer.
function AppNavigator() {
  const { isAuthenticated } = useAuth();

  return (
    <NavigationContainer linking={linking}>
      <RootStack.Navigator screenOptions={{ headerShown: false }}>
        {isAuthenticated ? (
          <>
            <RootStack.Screen name="Main" component={MainTabs} />
            <RootStack.Screen name="CreatePost" component={CreatePostModal} />
            <RootStack.Screen name="StoryViewer" component={StoryViewerModal} />
            <RootStack.Screen name="ImageViewer" component={ImageViewerModal} />
          </>
        ) : (
          <>
            <RootStack.Screen name="Login" component={LoginScreen} />
            <RootStack.Screen name="Register" component={RegisterScreen} />
            <RootStack.Screen name="ForgotPassword" component={ForgotPasswordScreen} />
          </>
        )}
      </RootStack.Navigator>
    </NavigationContainer>
  );
}

function MainTabs() {
  return (
    <Tab.Navigator>
      <Tab.Screen name="Feed" component={FeedStack} />
      <Tab.Screen name="Explore" component={ExploreStack} />
      <Tab.Screen name="Notifications" component={NotificationsStack} />
      <Tab.Screen name="Profile" component={ProfileStack} />
    </Tab.Navigator>
  );
}

function FeedStack() {
  return (
    <Stack.Navigator>
      <Stack.Screen name="FeedHome" component={FeedScreen} />
      <Stack.Screen name="PostDetail" component={PostDetailScreen} />
      <Stack.Screen name="Comments" component={CommentsScreen} />
      <Stack.Screen name="UserProfile" component={UserProfileScreen} />
    </Stack.Navigator>
  );
}
// ... ExploreStack, NotificationsStack, ProfileStack all follow the same pattern

This is maintenance overhead that grows with every new screen. Add a new screen? Update the navigator. Add a deep link? Update the linking config separately. Rename a screen? Update the navigator, the linking config, and every navigation.navigate('ScreenName') call across the codebase.

The Five Core Pain Points

1. Screens and routes are decoupled by default. You define a screen in the navigator, then write a separate component file for it, then configure its deep link path in a third place. Three places to update for one new screen.

2. The navigation prop is everywhere. To navigate from a child component, you either pass navigation down as a prop, use useNavigation() hook, or write workarounds. It's not terrible, but it adds friction.

3. Deep link configuration is manual and error-prone.

const linking = {
  prefixes: ['myapp://'],
  config: {
    screens: {
      Main: {
        screens: {
          Feed: {
            screens: {
              PostDetail: 'posts/:postId',       // manually written
              Comments: 'posts/:postId/comments', // easy to get wrong
            }
          }
        }
      }
    }
  }
};

For a large app, this config can be hundreds of lines. And it's maintained separately from the actual screens. When routes change, deep links break silently.

4. TypeScript is powerful but verbose. React Navigation has excellent TypeScript support, but typing your navigator params is a substantial amount of boilerplate that you write and maintain manually.

5. Sharing layout between screens is manual. Want a common header, a banner, or a context provider shared by a group of screens? You wrap the stack in a component, or use screen options. It works, but it's not declarative — you can't look at the file structure and know what's shared.

5. Why Expo Router Was Introduced

The Insight: Your Folder Structure Is Your App

In 2022, Evan Bacon (at Expo) introduced Expo Router with a simple idea: what if your file system defined your routes?

Next.js had already proven this model for web. Every file in pages/ or app/ is a route. No configuration. No separate routing file. You create a file, and the route exists.

Expo Router brought this exact mental model to React Native.

The goals were explicit:

1. Eliminate routing boilerplate. Create a file, get a route.
2. Make deep links automatic. Every route is a URL by default.
3. Enable shared layouts without ceremony. _layout.tsx wraps everything in its folder.
4. Improve discoverability. Read the folder, understand the app.
5. Unify web and native routing. The same Expo Router app can render on web and native with the same route structure.

The Key Insight: Expo Router IS React Navigation

Here's what most articles don't say clearly enough: Expo Router is built on top of React Navigation. It uses the same underlying library. You get the same navigation performance, the same transitions, the same native feel.

What Expo Router adds is a convention layer — a set of rules about how files map to routes, so you don't have to configure them manually.

Traditional Navigation Setup vs File-Based Routing
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

React Navigation:
  navigator.tsx        ← You write this: 200+ lines
      │
      ├── Define Stack
      ├── Define Tabs
      ├── Define Screens (manually)
      ├── Define deep link config (manually)
      └── Define auth guard (manually)

Expo Router:
  app/
  ├── (auth)/login.tsx     ← File exists = route exists
  ├── (tabs)/index.tsx     ← Parentheses = layout group
  └── [postId].tsx         ← Brackets = dynamic route

  _layout.tsx files handle auth, shared providers — once.
  Deep links are automatic. TypeScript types are generated.

6. File-Based Routing Explained Simply

The Core Concept: Files Are Screens

In Expo Router, the app/ directory is special. Every .tsx file inside it becomes a screen. The file path becomes the route path.

app/index.tsx          →  /              (home screen)
app/profile.tsx        →  /profile
app/settings.tsx       →  /settings
app/posts/[id].tsx     →  /posts/123     (dynamic)
app/posts/[id]/comments.tsx  →  /posts/123/comments

That's it. No navigator config. No screen registration. No linking config. Create the file, the route exists.

Expo Router Folder → Screen Mapping

app/
├── _layout.tsx              ← Root layout (wraps everything)
├── index.tsx                → Route: /
│
├── (auth)/                  ← Route GROUP (no URL segment)
│   ├── _layout.tsx          ← Auth layout (no tab bar, simple stack)
│   ├── login.tsx            → Route: /login
│   ├── register.tsx         → Route: /register
│   └── forgot-password.tsx  → Route: /forgot-password
│
├── (tabs)/                  ← Route GROUP (tab bar layout)
│   ├── _layout.tsx          ← Defines the tab navigator
│   ├── index.tsx            → Route: /          (Home tab)
│   ├── explore.tsx          → Route: /explore
│   ├── notifications.tsx    → Route: /notifications
│   └── profile.tsx          → Route: /profile
│
├── posts/
│   ├── [postId].tsx         → Route: /posts/abc123    (dynamic)
│   └── [postId]/
│       └── comments.tsx     → Route: /posts/abc123/comments
│
└── +not-found.tsx           → 404 screen

Special File Conventions

Dynamic Routes in Practice

// app/posts/[postId].tsx
import { useLocalSearchParams } from 'expo-router';

export default function PostDetail() {
  const { postId } = useLocalSearchParams<{ postId: string }>();
  // postId is automatically typed and populated from the URL
  
  return <PostView id={postId} />;
}

No route.params. No prop drilling. No navigator config. The URL segment becomes a typed variable.

7. Nested Layouts and Shared Layouts in Expo Router

The _layout.tsx File: Your Secret Weapon

Every folder in app/ can have a _layout.tsx. This file:

• Wraps all screens in that folder
• Defines which navigator type is used (Stack, Tabs, Drawer)
• Provides context, headers, or shared UI to all child screens
• Is the single source of truth for layout in that section of the app

// app/(tabs)/_layout.tsx
import { Tabs } from 'expo-router';
import { Ionicons } from '@expo/vector-icons';

export default function TabLayout() {
  return (
    <Tabs screenOptions={{ tabBarActiveTintColor: '#007AFF' }}>
      <Tabs.Screen
        name="index"
        options={{
          title: 'Home',
          tabBarIcon: ({ color }) => <Ionicons name="home" color={color} size={24} />,
        }}
      />
      <Tabs.Screen
        name="explore"
        options={{ title: 'Explore' }}
      />
      <Tabs.Screen
        name="profile"
        options={{ title: 'Profile' }}
      />
    </Tabs>
  );
}

This one file configures the entire tab bar for the (tabs) section. Every screen in that folder automatically gets the tab bar. Remove a screen file, it disappears from the tab bar.

Nested Layout Hierarchy

Nested Layout Hierarchy:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

app/_layout.tsx
└── Root Layout
    ├── Loads fonts
    ├── Sets up auth state
    ├── Wraps everything in ThemeProvider
    └── Decides: show (auth) or (tabs)?

    app/(auth)/_layout.tsx
    └── Auth Stack Layout
        ├── Simple Stack navigator
        ├── No tab bar
        └── Minimal header
            ├── login.tsx
            ├── register.tsx
            └── forgot-password.tsx

    app/(tabs)/_layout.tsx
    └── Tab Layout
        ├── Bottom tab bar
        ├── Tab icons and labels
        └── Shared tab bar styling

        app/(tabs)/(feed)/_layout.tsx
        └── Feed Stack Layout
            ├── Stack navigator inside the tab
            ├── Shared header back button behavior
            └── Gesture configuration
                ├── index.tsx        (Feed home)
                ├── [postId].tsx     (Post detail)
                └── [postId]/comments.tsx

Shared Context Without Prop Drilling

One of the most underrated features of _layout.tsx: you can wrap child screens in context providers once, and every screen in that folder gets access.

// app/(tabs)/_layout.tsx
export default function TabLayout() {
  return (
    <UserPreferencesProvider>  {/* All tab screens get this */}
      <Tabs>
        <Tabs.Screen name="index" />
        <Tabs.Screen name="profile" />
      </Tabs>
    </UserPreferencesProvider>
  );
}

In React Navigation, you'd wrap each screen component individually, or create a higher-order wrapper, or use a global context at the root. With Expo Router, you place the provider in the right _layout.tsx and it's scoped precisely where you need it.

8. Protected Routes and Authentication Flows

The React Navigation Approach

In React Navigation, auth protection is typically done by conditionally rendering different navigator trees:

// React Navigation auth pattern
function AppNavigator() {
  const { isAuthenticated } = useAuth();

  return (
    <NavigationContainer>
      {isAuthenticated ? (
        <AuthenticatedStack />
      ) : (
        <UnauthenticatedStack />
      )}
    </NavigationContainer>
  );
}

This works well but has some rough edges:

• The auth check happens inside the navigator, so you can't navigate until auth state is resolved.
• Deep links during the auth-loading phase can cause race conditions.
• If you need different protection levels (admin routes, premium routes), the conditional logic compounds.

The Expo Router Approach

Expo Router centralizes auth protection in the root _layout.tsx using useSegments() and useRouter():

// app/_layout.tsx — Protected Route Auth Flow
export default function RootLayout() {
  const { user, isLoading } = useAuthStore();
  const segments = useSegments();
  const router = useRouter();

  useEffect(() => {
    if (isLoading) return; // Wait for auth check to complete

    const inAuthGroup = segments[0] === '(auth)';

    if (!user && !inAuthGroup) {
      // Not authenticated, trying to access protected screen → redirect to login
      router.replace('/(auth)/login');
    } else if (user && inAuthGroup) {
      // Authenticated, on login screen → redirect to app
      router.replace('/(tabs)');
    }
  }, [user, isLoading, segments]);

  if (isLoading) return <SplashScreen />;

  return (
    <Stack screenOptions={{ headerShown: false }}>
      <Stack.Screen name="(auth)" />
      <Stack.Screen name="(tabs)" />
    </Stack>
  );
}

Protected Route Authentication Flow

App Launches
    │
    ▼
Root _layout.tsx renders
    │
    ├── isLoading = true → Show SplashScreen (block render)
    │
    └── isLoading = false
            │
            ├── Check: is user on an (auth) screen?
            │       AND user is authenticated?
            │       └── YES → router.replace('/(tabs)')
            │
            ├── Check: is user on a protected screen?
            │       AND user is NOT authenticated?
            │       └── YES → router.replace('/(auth)/login')
            │
            └── No redirect needed → render current route
                    │
                    ▼
              App flows normally.
              Auth state changes trigger the useEffect again.
              Logout automatically redirects to /(auth)/login.

Granular Route Protection (Admin Example)

For apps with multiple access levels (free user, premium, admin):

// app/(admin)/_layout.tsx
export default function AdminLayout() {
  const { user } = useAuthStore();
  const router = useRouter();

  useEffect(() => {
    if (user && user.role !== 'admin') {
      router.replace('/(tabs)'); // Kick non-admins out
    }
  }, [user]);

  if (user?.role !== 'admin') return null;

  return <Stack />;
}

Each section of your app can protect itself. No centralized mega-conditional. The (admin) folder's layout handles admin protection. The (premium) folder's layout handles premium protection. Each is a one-time, self-contained concern.

9. Performance Comparison

Bundle Behavior

React Navigation: You import and register every screen upfront. All screen components are part of the initial bundle unless you manually implement lazy loading with React.lazy() and Suspense.

// Every screen is imported at the top — all land in the bundle
import HomeScreen from '../screens/HomeScreen';
import ProfileScreen from '../screens/ProfileScreen';
import { HeavyMapScreen } from '../screens/HeavyMapScreen'; // Loaded even if user never visits

Expo Router: Route files are loaded on-demand by default. The app/ directory is code-split automatically. A screen isn't imported until its route is navigated to.

app/
├── (tabs)/index.tsx       ← Loaded at startup (default tab)
└── heavy-map.tsx          ← Loaded only when user navigates to /heavy-map

Winner for bundle behavior: Expo Router — automatic code splitting with no configuration.

Navigation Transitions

Both libraries use the same underlying navigation primitives (@react-navigation/native, react-native-screens). Transition animations are identical in quality because Expo Router delegates all transition logic to React Navigation.

Winner: Tie — same engine under the hood.

Developer Workflow

React Navigation workflow for a new screen:

1. Create src/screens/NewScreen.tsx
2. Import it in the navigator file
3. Add <Stack.Screen name="NewScreen" component={NewScreen} />
4. Add its deep link config to the linking object
5. Add TypeScript param types to the navigator's param list
6. Navigate to it with navigation.navigate('NewScreen', { id: '123' })

Six steps. Four files touched.

Expo Router workflow for a new screen:

1. Create app/new-screen.tsx

One step. One file. Deep link is automatic. TypeScript types are generated.

Winner for developer workflow: Expo Router — by a significant margin.

10. Developer Experience (DX) Comparison

Side-by-Side DX Comparison

Developer Experience at a Glance
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

                        React Navigation    Expo Router
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Adding a new screen     Manual (4 steps)    Create a file
Deep link setup         Manual config       Automatic
TypeScript types        Write manually      Auto-generated
Understanding app flow  Read navigator.tsx  Read folder tree
Shared layouts          Wrap + HOC          _layout.tsx
Protected routes        Conditional render  useSegments hook
Onboarding new devs     Hours → days        Minutes
Web + Native parity     Extra config        Built-in
Expo Go support         Yes                 Yes (first-class)
Testing routes          Simulate navigate   Actual URL
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

TypeScript Experience

React Navigation TypeScript requires you to define a params map and annotate every navigator:

// Types you write and maintain manually
type RootStackParamList = {
  Home: undefined;
  Profile: { userId: string };
  PostDetail: { postId: string; fromFeed: boolean };
  Settings: undefined;
};

// Then use everywhere
const navigation = useNavigation<NativeStackNavigationProp<RootStackParamList, 'Profile'>>();

This is verbose but gives full type safety. The downside: it's manual. Every new screen needs a new type entry.

Expo Router auto-generates TypeScript types from your file structure when you run npx expo start. You get a typed-routes.d.ts file automatically:

// Expo Router typed navigation — zero manual maintenance
import { Link } from 'expo-router';

// This is fully typed — TypeScript knows /posts/[postId] requires a postId
<Link href="/posts/123" />
<Link href={{ pathname: '/posts/[postId]', params: { postId: '123' } }} />

Winner for TypeScript DX: Expo Router — types reflect reality automatically.

Beginner Perspective

For someone new to React Native:

React Navigation has a steeper learning curve because you must understand:

• What a NavigationContainer is and why it goes at the root
• The difference between Stack, Tab, and Drawer navigators
• How to compose nested navigators
• How navigation.navigate() works and where to get the navigation object
• How to configure deep links separately

All of this before writing a single screen.

Expo Router has a gentler entry:

• Create an app/ folder. Put a file in it. That's a screen.
• The routing works like Next.js — familiar to anyone with web experience.
• Links between screens use the <Link> component — same mental model as HTML.

Winner for beginners: Expo Router — the file-based model requires less upfront knowledge.

11. Scalability Comparison for Large Applications

React Navigation at Scale

React Navigation scales well when:

• Your team establishes conventions early (naming, file structure, param types)
• You split your navigator into multiple files (one per feature)
• You build a typed navigation service that wraps navigate()

The challenges at scale:

• Navigation config is centralized — changes to routes require careful coordination
• Refactoring screen names is a grep-and-replace operation across the whole codebase
• New team members must learn the navigation architecture before contributing

Mitigation pattern used by large teams:

src/navigation/
├── RootNavigator.tsx
├── AuthNavigator.tsx
├── TabNavigator.tsx
├── FeedNavigator.tsx
├── ProfileNavigator.tsx
└── types.ts               ← All param types in one place

Splitting into files helps, but it's still manual maintenance.

Expo Router at Scale

Expo Router scales naturally because:

• The file structure is the navigation structure — no separate mental model
• Adding a feature means adding a folder — no navigator file to update
• Route names can't drift from file names — they're the same thing
• New developers understand the app structure from the folder tree

The challenge at scale:

• Expo Router imposes conventions. If your app needs deeply non-standard navigation patterns, you fight the framework.
• Large Expo Router projects can have deep folder nesting that becomes hard to navigate in the file system.

Scalability Comparison

Scalability Dimensions
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Dimension               React Navigation    Expo Router
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Team onboarding         Slow                Fast
Adding new screens      Manual, error-prone Automatic
Route refactoring       Risky               Safe (file rename)
Feature boundaries      Manual convention   Folder = feature
Deep link maintenance   Manual config       Zero config
Multi-team development  Conflict-prone      Independent
Monorepo support        Yes                 Yes
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Winner for scalability: Expo Router — convention-over-configuration wins at scale when teams are large and changing.

12. Real-World App Folder Structure Examples

React Navigation: Production App Structure

my-app/
├── src/
│   ├── navigation/
│   │   ├── RootNavigator.tsx        ← Main navigator file
│   │   ├── AuthNavigator.tsx        ← Auth screens stack
│   │   ├── TabNavigator.tsx         ← Tab bar config
│   │   ├── FeedNavigator.tsx        ← Feed stack
│   │   ├── MessagesNavigator.tsx    ← Messages stack
│   │   └── types.ts                 ← All RootStackParamList types
│   │
│   ├── screens/
│   │   ├── auth/
│   │   │   ├── LoginScreen.tsx
│   │   │   ├── RegisterScreen.tsx
│   │   │   └── ForgotPasswordScreen.tsx
│   │   ├── feed/
│   │   │   ├── FeedScreen.tsx
│   │   │   └── PostDetailScreen.tsx
│   │   ├── messages/
│   │   │   ├── ConversationsScreen.tsx
│   │   │   └── ChatRoomScreen.tsx
│   │   └── profile/
│   │       └── ProfileScreen.tsx
│   │
│   ├── components/
│   ├── hooks/
│   ├── services/
│   └── store/
│
└── App.tsx                          ← Renders <NavigationContainer>

Count of files to update when adding a new "Explore" screen: ExploreScreen.tsx + TabNavigator.tsx + types.ts = 3 files minimum.

Expo Router: Production App Structure

my-app/
├── app/
│   ├── _layout.tsx                  ← Root layout + auth guard
│   ├── index.tsx                    ← Entry redirect
│   │
│   ├── (auth)/
│   │   ├── _layout.tsx
│   │   ├── login.tsx
│   │   ├── register.tsx
│   │   └── forgot-password.tsx
│   │
│   ├── (tabs)/
│   │   ├── _layout.tsx              ← Tab bar defined here
│   │   ├── index.tsx                ← Home/Feed tab
│   │   ├── explore.tsx              ← Explore tab
│   │   ├── messages.tsx             ← Messages tab
│   │   └── profile.tsx              ← Profile tab
│   │
│   ├── posts/
│   │   └── [postId].tsx
│   │
│   ├── chat/
│   │   └── [chatId].tsx
│   │
│   └── +not-found.tsx
│
├── src/
│   ├── features/                    ← Business logic, unchanged from React Navigation
│   ├── shared/
│   ├── lib/
│   └── store/
│
└── app.json

Count of files to update when adding a new "Explore" screen: app/(tabs)/explore.tsx = 1 file. Add it to _layout.tsx tabs config = optionally 1 more.

Feature-Based Expo Router Structure (Enterprise Scale)

app/
├── _layout.tsx
│
├── (auth)/
│   ├── _layout.tsx
│   ├── login.tsx
│   └── register.tsx
│
├── (tabs)/
│   ├── _layout.tsx
│   │
│   ├── (feed)/                      ← Feed feature — nested stack in tab
│   │   ├── _layout.tsx
│   │   ├── index.tsx                → /feed
│   │   ├── [postId].tsx             → /feed/post123
│   │   └── [postId]/
│   │       └── comments.tsx         → /feed/post123/comments
│   │
│   ├── (explore)/
│   │   ├── _layout.tsx
│   │   ├── index.tsx
│   │   └── search.tsx
│   │
│   ├── (messages)/
│   │   ├── _layout.tsx
│   │   ├── index.tsx                → /messages
│   │   └── [chatId].tsx             → /messages/chat456
│   │
│   └── profile.tsx
│
└── (modals)/
    ├── _layout.tsx
    ├── create-post.tsx
    └── story-viewer.tsx

This structure maps directly to your features. A new engineer can look at the app/ folder and understand the entire navigation structure of the app in under a minute.

13. Which Approach Companies and Teams Prefer

The Reality of Adoption in 2026

React Navigation remains the most widely deployed. The reasons are practical:

• Most existing React Native codebases were started before Expo Router existed.
• It's the first library in every React Native tutorial, book, and course written before 2023.
• Large enterprises are conservative about switching navigation libraries in production apps.
• Many React Native apps are built without the Expo managed workflow — just bare React Native — and React Navigation integrates into any setup without opinion.

Expo Router has rapidly gained adoption among:

• New projects started in 2023 and later.
• Teams already using the Expo ecosystem (EAS Build, EAS Submit, Expo Go).
• Teams who want web + native parity (Expo Router is the only solution that routes identically on both).
• Developer advocates, bootcamps, and educators — the learning curve is genuinely lower.

What Production Teams Say

"We use React Navigation because we migrated from an older codebase" This is the most common answer from established teams. Migrating a large navigation system is expensive — usually not worth the immediate DX gain.

"We use Expo Router for all new projects" This is increasingly common from teams building greenfield apps. The file-based model, automatic deep links, and built-in web support make it the natural default for new Expo projects.

"We use React Navigation but follow Expo Router's folder conventions" Some teams adopt the folder structure pattern from Expo Router (one folder per feature, layouts separated) while keeping manual React Navigation config. Best of both worlds, but requires team discipline.

Rule of Thumb

14. When NOT to Use Expo Router

Expo Router Is Not for Every Situation

Expo Router is excellent, but there are real situations where it's the wrong choice. Use it with open eyes.

1. You're not using the Expo ecosystem Expo Router requires Expo. If your project is bare React Native (created with react-native init) and heavily integrated with native modules via custom native code, adding Expo Router means adding Expo tooling. That's a significant dependency for projects that have intentionally avoided it.

2. You need non-standard navigation patterns Expo Router works beautifully for apps that fit the Stack/Tab/Modal model. But if your app needs:

• Heavily custom transition animations that override the default gesture system
• Conditional tab bars that appear only in specific states
• Complex side-drawer navigation with nested tabs and stacks in unusual combinations

...you'll be fighting Expo Router's conventions. React Navigation gives you lower-level access.

3. You're migrating an existing large app Migrating an existing React Navigation app to Expo Router isn't just a refactor — it's a full restructuring. Your file system has to match your routes. Your navigation calls change. Your deep link config moves. For a large existing app, this is months of work with no user-visible benefit. The ROI is rarely there.

4. Your team doesn't use TypeScript Expo Router's TypeScript auto-generation is one of its best features. Without TypeScript, you lose a significant part of its DX advantage. The file-based routing still helps, but less so.

5. You have strict bundle size requirements and need granular control Expo Router does automatic code splitting, but you can't easily override its splitting decisions. React Navigation lets you be more explicit about what's lazy-loaded and when.

6. You're building a custom expo-config-plugin-heavy native module setup Some very specialized native configurations can conflict with Expo Router's metro configuration expectations. If you're deep in custom native code, test early.

15. Situations Where React Navigation Still Makes More Sense

React Navigation's Enduring Strengths

React Navigation isn't a legacy library being replaced by Expo Router. It's a mature, actively maintained library that remains the right choice in important scenarios.

Scenario 1: Bare React Native projects Any project not using Expo uses React Navigation. It's the standard for non-Expo React Native. No debate here.

react-native init MyApp
→ React Navigation. Full stop.

Scenario 2: Maximum navigation control React Navigation exposes every part of its internals. Custom navigators. Custom transitions. Intercepting navigation actions. Headless navigation (navigating from outside React).

// React Navigation: full control over custom navigators
function MyCustomNavigator({ state, navigation, descriptors, router }) {
  // Build literally any navigation pattern you want
}

Expo Router doesn't expose this level of control. If you need it, React Navigation is the answer.

Scenario 3: Non-standard screen presentation patterns Some apps have unique navigation flows that don't fit the Stack/Tab/Modal model:

• A canvas-based app where screens slide horizontally based on gestures
• A kiosk app with no back navigation at all
• An onboarding flow with branching paths and custom progress tracking

React Navigation gives you the primitives. You build the pattern.

Scenario 4: Existing team expertise A team that has two years of React Navigation experience, has built their own typed navigation layer, and has established patterns for auth, deep links, and state — they're probably more productive staying with what they know than switching to Expo Router.

The DX improvement of Expo Router is real, but it's measured in developer-hours per feature. If your team is already efficient with React Navigation, the learning curve of switching has a cost too.

Scenario 5: Hybrid apps with significant web WebView content Some React Native apps are mostly WebView-based, with native screens for authentication and settings. Expo Router's web routing is a strength — but if your "web" is already a WebView and not a React web app, this advantage disappears.

Scenario 6: Integration with third-party navigation solutions Some enterprise apps need to integrate with third-party navigation SDKs — navigation analytics, A/B testing of flows, remote-controlled navigation. These tools are built against React Navigation's API. Expo Router's abstraction layer can create compatibility issues.

The Honest Summary

React Navigation vs Expo Router — Honest Decision Matrix
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

                          React     Expo
Decision Factor           Nav       Router    Winner
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Using Expo ecosystem?      ○         ●        Expo Router
Bare React Native?         ●         ○        React Navigation
New project?               ○         ●        Expo Router
Existing codebase?         ●         ○        React Navigation
Beginner friendly?         ○         ●        Expo Router
Max nav control?           ●         ○        React Navigation
Web + Native parity?       ○         ●        Expo Router
Non-Expo native modules?   ●         ○        React Navigation
Team DX & speed?           ○         ●        Expo Router
Custom nav patterns?       ●         ○        React Navigation
Auto deep links?           ○         ●        Expo Router
Large team scalability?    ○         ●        Expo Router
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
● = Advantage    ○ = Disadvantage/Neutral

The Bottom Line

Here's the no-fluff answer:

Start new Expo projects with Expo Router. The file-based routing, automatic deep links, built-in auth guard pattern, shared layouts via _layout.tsx, and auto-generated TypeScript types are a genuine DX improvement. The mental model is easier. The team onboarding is faster. The folder structure is the documentation.

Keep React Navigation for existing projects and bare React Native. Migrating a large navigation system for DX gains is almost never worth it. React Navigation is not going anywhere — it's actively maintained, it powers Expo Router internally, and it will be a first-class choice for years to come.

Remember: Expo Router is React Navigation. There's no performance tradeoff, no compatibility risk, no "choosing the less proven library." You're using the same engine with better conventions. That's not a compromise — that's progress.

The best architecture choice is the one your team ships with. Pick what your team can move fast with, and go build something.

Quick Reference Summary

Published as part of the Mobile Development Cohort — React Native track.

Next up: Building a production-grade auth flow with Expo Router + Zustand + MMKV.

Architecture isn't about the code you write. It's about the decisions you make before writing any code.

If you've ever opened a React Native project and stared at a components/ folder with 200 files in it, this article is for you. Today, we're going to think like the engineers who built Instagram, WhatsApp, Uber, and Netflix — and explore how you'd structure those apps if you were starting from scratch today, using Expo Router.

This isn't a UI clone tutorial. We're not building Instagram's Reels UI or Uber's map screen pixel-by-pixel. We're thinking about architecture — how to organize code, navigate between screens, handle state, sync data in real time, and scale without losing your mind.

Let's go.

1. Why Simple Folder Structures Fail at Scale

The "It Works on My Machine" Phase

Every React Native project starts simple. You have a screens/ folder, a components/ folder, maybe a utils/ folder, and a single App.tsx. It's clean, understandable, and everyone on the team (usually just you) knows where everything lives.

Then the app grows.

You add authentication, then a profile screen, then a feed, then notifications, then settings, then DMs. Suddenly your screens/ folder has 40 files, your components/ folder has 150+ components with names like FeedCard.tsx, FeedCardNew.tsx, and FeedCardV2Final.tsx. Your App.tsx has 600 lines of nested navigators. New team members spend two days just understanding the project before writing a single line of code.

This is not a theoretical problem. This is how most apps die.

What Goes Wrong

The root causes of a failing folder structure are almost always the same:

• Feature logic is split across the codebase. The authentication screen, its API call, its state slice, its validation logic, and its UI components all live in different folders. To understand auth, you jump between 6 different directories.

• No clear ownership. When something breaks, no one knows which folder "owns" the broken piece.

• Shared components become a graveyard. A components/ folder that's shared by everyone is owned by no one. It accumulates dead code that nobody dares delete.

• Routing becomes a monolith. A single navigation file managing 40+ screens becomes unmaintainable.

The Mental Shift: From Files to Features

The solution isn't a better folder name. It's a different way of thinking. Instead of organizing code by type (all screens together, all components together), you organize it by feature (everything related to messaging lives in the messaging/ module).

This is the shift from small-app thinking to production engineering thinking. And Expo Router makes this shift natural.

2. Why Architecture Matters in React Native Applications

Mobile Has Unique Constraints

Web apps can get away with messy architecture longer because the browser handles a lot — routing, caching, back navigation. In React Native, you own all of it.

• Navigation is your responsibility. There's no browser URL bar. Every screen transition, deep link, and back press is code you write and maintain.

• Performance is visible. A janky scroll, a slow startup, a flickering list — users feel it immediately. Architecture choices directly affect frame rate.

• Offline behavior is expected. Mobile users expect apps to work in tunnels, on planes, and with bad connectivity. This requires deliberate caching and sync architecture.

• Bundle size matters. A 50MB app has lower install rates than a 10MB app. Architecture decisions affect how code is split and loaded.

The Cost of Rearchitecting Later

Bad architecture in mobile is especially painful to fix because:

1. Deep links are tied to your navigation structure. Changing routes breaks links shared by users.

2. State management changes require rewriting screens, not just services.

3. API layer changes ripple across every screen that touches data.

The engineers at Instagram, Uber, and Netflix didn't get their architecture right because they were smarter. They got it right because they invested in it early, before scale made changes expensive.

3. Folder Architecture Using Expo Router

How Expo Router Changes Everything

Expo Router brings file-based routing to React Native — the same mental model as Next.js. Your file structure is your navigation structure. This is a massive architectural win because:

• Routes are self-documenting. You can read the folder structure and understand the app's navigation.

• Deep linking works automatically. Every route is a URL.

• Shared layouts are trivial. A _layout.tsx wraps all screens in its directory.

• Code splitting is natural. Features map to folders, folders map to routes.

Production-Grade Expo Router Folder Structure

Here's how a production app at scale would be structured:

my-app/
├── app/                          # Expo Router root — file-based routes
│   ├── _layout.tsx               # Root layout (fonts, providers, auth guard)
│   ├── index.tsx                 # Entry redirect (to onboarding or home)
│   │
│   ├── (auth)/                   # Auth group — no tab bar
│   │   ├── _layout.tsx
│   │   ├── login.tsx
│   │   ├── register.tsx
│   │   └── forgot-password.tsx
│   │
│   ├── (tabs)/                   # Main app tab layout
│   │   ├── _layout.tsx           # Tab bar configuration
│   │   ├── index.tsx             # Home / Feed
│   │   ├── explore.tsx
│   │   ├── notifications.tsx
│   │   └── profile.tsx
│   │
│   ├── (modals)/                 # Full-screen modals
│   │   ├── _layout.tsx
│   │   ├── create-post.tsx
│   │   └── story-viewer.tsx
│   │
│   └── +not-found.tsx            # 404 screen
│
├── src/
│   ├── features/                 # Feature-based modules
│   │   ├── auth/
│   │   │   ├── components/
│   │   │   ├── hooks/
│   │   │   ├── services/
│   │   │   ├── store/
│   │   │   └── types.ts
│   │   ├── feed/
│   │   ├── messaging/
│   │   ├── profile/
│   │   ├── notifications/
│   │   └── player/               # e.g., for Netflix-style video
│   │
│   ├── shared/                   # Truly shared, stable code
│   │   ├── components/           # Button, Avatar, Card, etc.
│   │   ├── hooks/                # useDebounce, useThrottle, etc.
│   │   ├── utils/
│   │   └── constants/
│   │
│   ├── lib/                      # Third-party integrations
│   │   ├── api/                  # Axios/Fetch instance + interceptors
│   │   ├── socket/               # WebSocket/Socket.io client
│   │   ├── storage/              # MMKV / AsyncStorage wrapper
│   │   └── analytics/
│   │
│   └── store/                    # Global state (Zustand / Redux)
│       ├── auth.store.ts
│       ├── app.store.ts
│       └── index.ts
│
├── assets/
├── constants/
└── app.json

Understanding Route Groups

Expo Router's route groups (folders wrapped in parentheses) are the architectural backbone of a production app. They:

• Group screens that share a layout without affecting the URL/path.

• Allow you to have tab-bar screens and auth screens as completely separate layout trees.

• Enable modals, drawers, and stacks to coexist cleanly.

(auth)/     → No tab bar, simple stack
(tabs)/     → Tab bar always visible
(modals)/   → Slides over current screen

This maps directly to how Instagram works: the feed, explore, and profile screens share a tab bar. The story viewer and DM composer are modals. The login screen has none of it.

4. Feature-Based Separation in Large Applications

One Module, One Responsibility

Feature-based architecture means each feature directory contains everything it needs:

src/features/messaging/
├── components/
│   ├── MessageBubble.tsx
│   ├── ChatHeader.tsx
│   └── TypingIndicator.tsx
├── hooks/
│   ├── useMessages.ts
│   ├── useSocket.ts
│   └── useTypingStatus.ts
├── services/
│   ├── messaging.api.ts         # API calls
│   └── messaging.socket.ts      # Socket event handlers
├── store/
│   └── messaging.store.ts       # Zustand/Redux slice
└── types.ts

When a new engineer joins and needs to work on chat, they go to src/features/messaging/. That's it. Everything they need is there.

Rules for Feature Modules

1. Features can use shared/, but shared cannot import from features.

2. Features should not directly import from other features. Cross-feature communication goes through the global store or events.

3. API calls live in the feature's services/ folder, not in a global API folder.

4. If a component is used in more than two features, it graduates to shared/components/.

WhatsApp Example: Feature Boundaries

In a WhatsApp-style app, the feature breakdown would look like:

Each of these is a self-contained unit. The messaging/ feature talks to the contacts/ feature only through shared store or passed props — never through a direct import.

5. Navigation Architecture for Scalable Apps

The Three Navigation Primitives

Every mobile app, no matter how complex, is built on three navigation primitives:

1. Stack — Push/pop screens. Used for drilling into detail views.

2. Tabs — Side-by-side top-level sections. The bottom tab bar.

3. Modal — Overlays the current context. Full-screen modals, bottom sheets.

Expo Router composes these naturally through nested layouts.

Navigation Hierarchy for Large Applications

Root Layout (_layout.tsx)
├── Auth Stack  ← if not authenticated
│   ├── Login
│   ├── Register
│   └── Forgot Password
│
└── App (Tabs)  ← if authenticated
    ├── Tab: Feed
    │   └── Stack
    │       ├── Feed Home
    │       ├── Post Detail
    │       └── Comment Thread
    ├── Tab: Explore
    │   └── Stack
    │       ├── Explore Home
    │       └── Search Results
    ├── Tab: Messages
    │   └── Stack
    │       ├── Conversations List
    │       └── Chat Room
    └── Tab: Profile
        └── Stack
            ├── Profile Home
            └── Edit Profile

Modals (rendered above tabs)
├── Camera / Create Post
├── Story Viewer
└── Share Sheet

In Expo Router, this becomes:

// app/(tabs)/_layout.tsx
export default function TabLayout() {
  return (
    <Tabs>
      <Tabs.Screen name="index" options={{ title: "Home" }} />
      <Tabs.Screen name="explore" options={{ title: "Explore" }} />
      <Tabs.Screen name="messages" options={{ title: "Messages" }} />
      <Tabs.Screen name="profile" options={{ title: "Profile" }} />
    </Tabs>
  );
}

Instagram-Style Navigation

Instagram's navigation is a master class in this pattern:

• Main tabs: Home, Search, Reels, Shop, Profile.

• Tapping a post from the feed pushes a detail screen within the tab's stack.

• Stories open as a modal — they slide up over the feed without changing your tab context.

• The DM icon in the top-right opens another stack that sits outside the tab layout.

With Expo Router, the stories and DMs are modals. The feed detail is a stack push. No special navigation library configuration needed — the folder structure defines it all.

6. Authentication Flow Architecture

The Protected Route Pattern

The most critical navigation decision in any app is: how do you guard routes from unauthenticated users?

With Expo Router, the clean pattern is to handle this in the root layout:

// app/_layout.tsx
export default function RootLayout() {
  const { isAuthenticated, isLoading } = useAuthStore();

  if (isLoading) return <SplashScreen />;

  return (
    <Stack>
      {isAuthenticated ? (
        <Stack.Screen name="(tabs)" options={{ headerShown: false }} />
      ) : (
        <Stack.Screen name="(auth)" options={{ headerShown: false }} />
      )}
    </Stack>
  );
}

Authentication Flow with Protected Routes

App Launch
    │
    ▼
Check token in SecureStore / MMKV
    │
    ├── Token exists → Validate with server
    │       ├── Valid   → Navigate to (tabs)/
    │       └── Expired → Refresh token
    │               ├── Success → Navigate to (tabs)/
    │               └── Fail    → Navigate to (auth)/login
    │
    └── No token → Navigate to (auth)/login

Login Screen
    │
    ▼
POST /auth/login
    │
    ├── Success → Store access + refresh token → Navigate to (tabs)/
    └── Failure → Show error, stay on (auth)/login

Token Management Layer

Never scatter auth logic across screens. Centralize it:

src/lib/auth/
├── tokenStorage.ts     # Secure read/write of access & refresh tokens
├── tokenRefresh.ts     # Interceptor logic for expired tokens
└── authGuard.ts        # Route protection hook

The API client (Axios/Fetch) uses an interceptor that automatically attaches the access token to every request, and automatically refreshes it on 401 responses — without any screen knowing this is happening.

Biometric Authentication (Instagram/WhatsApp Pattern)

Large apps also implement app-lock via biometrics:

App returns to foreground
    │
    ▼
Check if biometric lock is enabled
    │
    ├── Enabled → Show biometric prompt (Face ID / Fingerprint)
    │       ├── Success → Unlock app
    │       └── Fail    → Show PIN fallback
    │
    └── Disabled → Resume directly

7. State Management Strategies for Large Apps

Don't Use One Solution for Everything

The most common mistake is reaching for Redux for everything. Production apps use multiple state management layers based on what kind of state it is:

API Layer + State Management Flow

User Action (e.g., pulls to refresh feed)
    │
    ▼
React Query: useFeedQuery()
    │
    ├── Cache hit → Return stale data immediately, refetch in background
    │
    └── Cache miss / stale
            │
            ▼
        API Service Layer (src/lib/api/)
            │
            ├── Attach auth token (interceptor)
            ├── POST /feed?cursor=...
            └── Response
                    │
                    ├── Success → Update React Query cache → UI re-renders
                    └── Error   → Retry logic → Error boundary

Zustand for Global UI State

Zustand is the sweet spot for global UI and auth state — lightweight, no boilerplate, works perfectly with Expo Router:

// src/store/auth.store.ts
interface AuthState {
  user: User | null;
  isAuthenticated: boolean;
  setUser: (user: User) => void;
  clearAuth: () => void;
}

export const useAuthStore = create<AuthState>((set) => ({
  user: null,
  isAuthenticated: false,
  setUser: (user) => set({ user, isAuthenticated: true }),
  clearAuth: () => set({ user: null, isAuthenticated: false }),
}));

Netflix Pattern: Hybrid State

Netflix uses a layered state strategy:

• TanStack Query for content catalog (search results, category rows, show details — all server state with aggressive caching).

• Zustand for playback state (current episode, playback position, quality setting).

• MMKV for persisted watch progress (synced to server periodically, not on every second).

• Context API for player UI state (overlay visibility, control bar) — scoped and doesn't need to be global.

8. API Handling and Networking Layers

The API Client Architecture

Never call fetch() directly in a component. Build a proper API layer:

src/lib/api/
├── client.ts          # Axios instance with base URL, timeout
├── interceptors.ts    # Auth token attach, refresh on 401, error logging
├── endpoints.ts       # Centralized endpoint constants
└── types.ts           # API response types

// src/lib/api/client.ts
const apiClient = axios.create({
  baseURL: process.env.EXPO_PUBLIC_API_URL,
  timeout: 10000,
  headers: { 'Content-Type': 'application/json' },
});

// Attach token on every request
apiClient.interceptors.request.use(async (config) => {
  const token = await tokenStorage.getAccessToken();
  if (token) config.headers.Authorization = `Bearer ${token}`;
  return config;
});

// Handle 401 — refresh token silently
apiClient.interceptors.response.use(
  (res) => res,
  async (error) => {
    if (error.response?.status === 401) {
      await refreshAccessToken();
      return apiClient(error.config); // retry
    }
    return Promise.reject(error);
  }
);

Feature-Level Service Files

Each feature owns its API calls:

// src/features/feed/services/feed.api.ts
export const feedApi = {
  getFeed: (cursor?: string) =>
    apiClient.get<FeedResponse>('/feed', { params: { cursor } }),
  likePost: (postId: string) =>
    apiClient.post(`/posts/${postId}/like`),
  getComments: (postId: string) =>
    apiClient.get<CommentsResponse>(`/posts/${postId}/comments`),
};

Uber's Location Data Strategy

Uber doesn't hit a REST API for live location updates. They use a combination:

• REST API for ride requests, pricing, driver info — standard request/response.

• WebSocket for live driver location updates — continuous stream.

• HTTP polling as fallback when WebSocket disconnects.

• Client-side prediction — the map moves the pin smoothly between received coordinates using interpolation, so it doesn't jump every second.

9. Real-Time Systems

Chat Systems (WhatsApp Architecture)

WhatsApp's real-time chat is built on reliable message delivery with guaranteed order. In a React Native app, this translates to:

src/features/messaging/
├── services/
│   └── messaging.socket.ts     # All socket event logic
└── store/
    └── messaging.store.ts      # Local message state

The messaging socket layer:

// Simplified socket event flow
socket.on('message:receive', (message: Message) => {
  // 1. Optimistically add to local store
  messagingStore.addMessage(message);
  // 2. Persist to local DB (WatermelonDB / SQLite)
  localDb.messages.insert(message);
  // 3. Send delivery receipt
  socket.emit('message:delivered', { messageId: message.id });
});

Realtime Messaging Architecture Flow:

User types and sends message
        │
        ▼
Optimistic update → Add to local state immediately (show in UI)
        │
        ▼
Emit via WebSocket: socket.emit('message:send', payload)
        │
        ├── Server ACK received
        │       └── Update message status: sent → delivered
        │
        └── ACK timeout (5s)
                └── Queue for retry → Re-emit on reconnect

Live Updates (Instagram Feeds)

Instagram's feed doesn't use WebSockets for new posts — that would be too expensive at scale. Instead, they use:

• Long polling or Server-Sent Events (SSE) for like counts and comment counts.

• Background refresh every 60 seconds when the app is in focus.

• Push notifications for activity (new follower, comment) — which deep link into Expo Router routes.

// Using TanStack Query for Instagram-style live feed data
const { data, refetch } = useQuery({
  queryKey: ['feed'],
  queryFn: feedApi.getFeed,
  refetchInterval: 60_000, // refetch every 60s
  staleTime: 30_000,       // treat data as fresh for 30s
});

Ride Tracking (Uber Architecture)

Uber's live tracking is one of the most technically demanding real-time features in any mobile app:

Driver's phone
    │ emits location every 4s via WebSocket
    ▼
Location service (backend)
    │ broadcasts to rider's WebSocket channel
    ▼
Rider's React Native app
    │
    ├── Receives coordinate {lat, lng, bearing}
    ├── Stores in Zustand: rideStore.updateDriverLocation()
    ├── Animates marker on map (interpolation over 4s)
    └── Updates ETA label

In React Native with Expo Router:

// src/features/ride/hooks/useDriverLocation.ts
export function useDriverLocation(rideId: string) {
  const updateDriverLocation = useRideStore((s) => s.updateDriverLocation);

  useEffect(() => {
    const channel = supabase
      .channel(`ride:${rideId}`)
      .on('broadcast', { event: 'location' }, ({ payload }) => {
        updateDriverLocation(payload);
      })
      .subscribe();

    return () => supabase.removeChannel(channel);
  }, [rideId]);
}

10. Offline-First Support and Caching

Why Offline Matters More on Mobile

A web user with no internet just sees an error. A mobile user with no internet expects the app to work — or at least show them their last known state. Instagram still shows your feed. WhatsApp still shows your conversations. Spotify still plays downloaded music.

Offline-first architecture means: assume the network is unreliable and design around it.

The Offline Cache Synchronization Flow

App opens / user navigates to screen
        │
        ▼
Check local cache (MMKV / WatermelonDB / SQLite)
        │
        ├── Cache hit → Render immediately from cache
        │       │
        │       └── If online → Fetch fresh data in background
        │               └── On response → Diff and update cache → Re-render if changed
        │
        └── Cache miss + offline → Show empty state with "No connection" UI
                       + offline → Queue actions (likes, messages) for retry on reconnect

Tools for Offline-First in React Native

WhatsApp's Offline Message Queue

// When offline, queue outgoing messages
async function sendMessage(chatId: string, content: string) {
  const message = createOptimisticMessage(content); // status: 'queued'
  messagingStore.addMessage(message);               // show in UI

  if (!networkStore.isOnline) {
    await messageQueue.enqueue({ chatId, message }); // persist to local DB
    return;
  }

  try {
    await messagingApi.sendMessage(chatId, content);
    messagingStore.updateStatus(message.id, 'sent');
  } catch {
    messagingStore.updateStatus(message.id, 'failed');
  }
}

// Drain the queue on reconnect
networkStore.onReconnect(async () => {
  const queued = await messageQueue.getAll();
  for (const item of queued) {
    await sendMessage(item.chatId, item.message.content);
    await messageQueue.remove(item.id);
  }
});

11. App Startup Optimization Techniques

The Problem: Slow Cold Start

Cold start time — the time from tapping the app icon to seeing your first interactive screen — directly impacts retention. Every second of loading loses users. Facebook's internal studies have shown that every 100ms of startup latency reduces engagement meaningfully.

App Startup Lifecycle Optimization

App process starts
    │
    ▼
[Phase 1: Critical bootstrap — must be fast]
    ├── Load fonts (preload in _layout.tsx with expo-font)
    ├── Read auth token from MMKV (synchronous, no async)
    └── Determine initial route (auth or tabs)
    │
    ▼
[Phase 2: Render first screen — show something immediately]
    ├── Show splash screen or skeleton
    └── Start background tasks (non-blocking):
            ├── Validate token with server
            ├── Prefetch critical data (e.g., first page of feed)
            └── Initialize analytics, crash reporting
    │
    ▼
[Phase 3: Hydration — fill in the real content]
    ├── Replace skeletons with real data
    ├── Initialize WebSocket connection
    └── Schedule background sync

Key Optimization Techniques

1. Use MMKV instead of AsyncStorage for token reads MMKV is synchronous and 10x faster than AsyncStorage. Reading the auth token on startup becomes instantaneous.

2. Defer non-critical initialization Don't initialize analytics, Sentry, or push notifications in the root layout. Initialize them after the first screen renders.

3. Prefetch above-the-fold data While showing the splash/skeleton, start fetching the feed. By the time the user sees the feed, data is already loading.

4. Use expo-splash-screen to control dismissal Don't dismiss the splash screen until fonts are loaded and the auth check is complete. This prevents a flash of unstyled content.

// app/_layout.tsx
SplashScreen.preventAutoHideAsync();

export default function RootLayout() {
  const [fontsLoaded] = useFonts({ ... });
  const { isAuthChecked } = useAuthStore();

  useEffect(() => {
    if (fontsLoaded && isAuthChecked) {
      SplashScreen.hideAsync();
    }
  }, [fontsLoaded, isAuthChecked]);
}

5. Lazy-load heavy screens Netflix's video player, Uber's map, Instagram's camera — these shouldn't be loaded at startup. Use React's lazy() and Suspense to load them on demand.

12. Performance Considerations in Production Apps

The Performance Killers

Most React Native performance issues come from the same handful of mistakes:

Instagram's Feed Performance

Instagram's feed scrolls at 60fps with high-res images and videos. The key techniques:

• FlashList (by Shopify) instead of FlatList — recycles cell components more efficiently.

• Progressive image loading — show a blurred thumbnail, then full image.

• expo-image — built-in disk and memory caching, reduces re-downloads.

• Virtualization — only the ~5 items visible + ~10 off-screen are rendered.

• Video pause on scroll — videos pause when they leave the viewport, freeing GPU resources.

Netflix's Video Playback Architecture

Netflix has the most demanding performance requirements:

• Adaptive bitrate streaming (HLS/DASH) — quality adjusts to network speed automatically.

• Pre-buffering — the next episode starts buffering 30s before the current one ends.

• Background download — offline downloads happen in the background via expo-file-system + react-native-background-downloader.

• Playback state isolation — the player's Zustand store is separate from the app store. Player state changes don't cause the rest of the app to re-render.

13. Shared Layouts and Nested Routing in Expo Router

The Power of _layout.tsx

Every folder in your app/ directory can have a _layout.tsx that wraps all screens inside it. This is how you share headers, tab bars, sidebars, and context providers without repeating them.

app/
├── _layout.tsx              ← Wraps EVERYTHING (fonts, providers, auth guard)
├── (tabs)/
│   ├── _layout.tsx          ← Wraps all tabs (tab bar itself)
│   └── (feed)/
│       ├── _layout.tsx      ← Wraps feed screens (shared header context)
│       ├── index.tsx        ← /feed
│       └── [postId].tsx     ← /feed/123

Dynamic Routes for Content Apps

Dynamic routes ([id]) are essential for content apps:

app/(tabs)/(feed)/
├── index.tsx              → /feed
├── [postId].tsx           → /feed/abc123
└── [postId]/
    └── comments.tsx       → /feed/abc123/comments

This means deep links like myapp://feed/abc123 work automatically. No manual deep link handling. Expo Router resolves it to the right screen.

Nested Navigation: Instagram DMs Example

Instagram's DM flow sits outside the main tab structure — it's like a second app that slides over:

app/
├── (tabs)/                 ← Main tab navigation
│   ├── index.tsx           ← Feed
│   └── profile.tsx         ← Profile
│
└── (messages)/             ← Separate stack, slides over tabs
    ├── _layout.tsx         ← Stack navigator, no tab bar
    ├── index.tsx           ← Conversations list
    └── [chatId].tsx        ← Individual chat room

When a user taps the DM icon, they're pushed into the (messages) stack, which renders over the tabs but preserves the tab bar state underneath.

14. Scalability Challenges and How Each App Solves Them

Instagram: Feeds and Media at Scale

Challenge: A user follows 1,000 accounts. Computing a personalized feed of recent posts from all of them in real time is expensive. Serving full-resolution images to 2 billion users is expensive.

Architecture decisions:

• Fan-out on write vs. fan-out on read: Instagram pre-computes feeds for most users (fan-out on write) but computes feeds on-demand (fan-out on read) for users who follow celebrities with 100M+ followers.

• CDN for media: Every image goes through a CDN. The app requests the image at the exact pixel dimensions needed, not the original.

• Progressive JPEG + lazy loading: Feed items below the fold are not loaded until the user scrolls near them.

• Expo Router impact: The feed screen uses a dynamic route for post details ([postId].tsx). Stories use a modal layout to avoid breaking feed scroll position.

WhatsApp: Realtime Messaging at Scale

Challenge: 100 billion messages per day. Message ordering must be consistent. Messages must arrive even if the receiver is offline for 30 days.

Architecture decisions:

• XMPP-derived protocol: Messages are not stored on servers by default — end-to-end encrypted and pushed to device.

• Message queue for offline delivery: Messages are held on server until the device comes online, then delivered in order.

• Local database first: All messages are written to local SQLite (WatermelonDB in React Native equivalent) first. The server is the source of truth for delivery but the device DB is the source of truth for display.

• Expo Router impact: Chat rooms are dynamic routes ([chatId].tsx). Typing indicators and online status come from a WebSocket layer, completely separate from the API layer.

Uber: Maps and Live Location at Scale

Challenge: Tracking millions of active drivers and riders simultaneously. Matching the nearest available driver in real time. Routing that updates as traffic changes.

Architecture decisions:

• Geohashing: The map is divided into grid cells. Uber only tracks drivers in cells near the rider's cell — not all drivers everywhere.

• Location updates decoupled from map rendering: The app receives location updates every 4 seconds but animates the map pin continuously using interpolation.

• Separate service for each phase: Ride request, driver matching, active ride tracking, and payment are handled by separate backend microservices. The app consumes them through a unified API gateway.

• Expo Router impact: The ride tracking screen is a modal that slides over the home map. It subscribes to a Supabase/Socket.io channel scoped to the active rideId — which comes from the route param.

Netflix: Heavy Content Delivery at Scale

Challenge: Streaming video to 300M subscribers simultaneously. Recommending content that keeps users watching. Handling content libraries with thousands of titles across dozens of markets.

Architecture decisions:

• Adaptive bitrate streaming: The player automatically switches between 240p and 4K based on current bandwidth.

• Open Connect (CDN): Netflix operates its own CDN appliances placed inside ISPs. Popular content is cached at the ISP level, dramatically reducing streaming latency.

• Recommendation as a separate layer: The recommendation engine runs server-side. The app just receives a list of contentId arrays for each row ("Continue Watching", "Top Picks for You"). It doesn't know the algorithm.

• Expo Router impact: Content is browsed via tabs (Home, Search, Downloads, Profile). The player is a full-screen modal with hidden navigation chrome. The contentId comes from a dynamic route ([contentId].tsx), making deep links from push notifications trivially simple.

15. Tradeoffs and Architectural Decisions at Scale

When to Use React Query vs. Zustand vs. Context

Tradeoffs Every Team Faces

Expo Router vs. React Navigation directly

Expo Router adds file-based conventions but requires your project to follow its folder structure. React Navigation gives more control at the cost of more boilerplate. For new projects, Expo Router wins. For migrating existing projects, the cost is higher.

TanStack Query vs. RTK Query

Both are excellent. TanStack Query has a smaller API surface and is framework-agnostic. RTK Query integrates tightly with Redux. If your team already uses Redux, RTK Query is less friction. If starting fresh, TanStack Query is simpler.

WatermelonDB vs. SQLite directly vs. no local DB

WatermelonDB is the right choice for apps with complex relational data (messaging, social feeds). SQLite directly is lower-level but works. Most apps don't need a local DB — MMKV + TanStack Query cache covers 80% of use cases.

Monorepo vs. separate repos

At Uber and Meta scale, different teams own different parts of the app. A monorepo (Nx or Turborepo) allows shared code while keeping team ownership clear. For solo developers and small teams, a single repo is fine.

The Most Important Decision: Invest in Architecture Early

Here's the honest truth: you will not build Instagram, WhatsApp, Uber, or Netflix. But you will build something that grows beyond what you planned for. The folder structure you choose in week one will still be the folder structure you're arguing about in year three.

Feature-based architecture with Expo Router's file-based routing gives you a system that scales with the team and the product. Not because it's perfect, but because its conventions force clarity — every engineer knows where to look, every new feature knows where to live.

That clarity, at scale, is everything.

Summary: Key Architectural Principles

Conclusion

The apps we admire didn't start with perfect architecture. They evolved. Engineers at Instagram and WhatsApp have written public post-mortems about the painful rewrites they had to do when their early structure couldn't scale. You have the advantage of learning from those rewrites.

Expo Router gives you a routing system that grows with you. Feature-based architecture gives you a codebase where every file has a home. A layered state strategy gives you the right tool for every type of data.

You don't need to build Instagram. You need to build your app in a way that won't embarrass you in two years. Start with structure. The features follow naturally.

In this article, we'll build a clear mental model — step by step — from the problem of slow DOM manipulation all the way to how React decides what (and what not) to update on screen.

No Fiber internals. No low-level implementation details. Just the mental model you actually need.

The Problem: Direct DOM Manipulation Is Expensive

Before React, the standard approach was to manipulate the DOM directly — with vanilla JavaScript or jQuery.

This worked fine for small pages. But as apps grew, a pattern emerged: every time something changed (a user typed in an input, a list item was added, a counter incremented), developers would reach into the DOM and update it directly.

document.getElementById('count').textContent = newCount;
document.querySelector('.user-list').innerHTML = buildListHTML(users);

The problem? The DOM is expensive to touch.

Every time you read from or write to the DOM, the browser may need to:

• Recalculate styles (style computation)

• Recompute the layout of elements (reflow / layout)

• Repaint affected areas on screen

These operations are not cheap. And when they happen on every keystroke, every data update, every state change — across a complex app with hundreds of elements — performance degrades fast.

The real bottleneck isn't JavaScript itself. JavaScript is fast. The bottleneck is the bridge between JavaScript and the browser's rendering engine. Every crossing of that bridge has a cost.

flowchart LR
    A["JS DOM write"] --> B["Style recalculation"]
    B --> C["Reflow / Layout"]
    C --> D["Repaint"]
    D --> E["Screen update"]

Every single DOM write can cascade through all of these steps — even for a one-character text change.

The Real DOM vs. The Virtual DOM

Let's get the distinction clear before going further.

The Real DOM

The Real DOM (Document Object Model) is the browser's structured, live representation of your HTML. Every element on the page — every <div>, <p>, <button> — is a node in a tree. When you change a node, the browser reacts immediately: it recalculates layout, triggers repaints, and updates what's on screen.

The Real DOM is powerful, but it's also heavyweight. Each node carries a lot of associated data — computed styles, event listeners, layout geometry. Touching it unnecessarily, or touching more of it than you need to, wastes time.

The Virtual DOM

The Virtual DOM is React's answer to this problem. It's not a browser API — it's a concept implemented entirely in JavaScript.

The Virtual DOM is simply a lightweight JavaScript object (a plain tree of objects) that describes what the UI should look like. It mirrors the structure of the Real DOM, but it's completely detached from the browser. Reading and writing to it is just object manipulation in memory — fast, cheap, and consequence-free.

Here's an oversimplified look at what a Virtual DOM node looks like:

// What React creates internally when you write JSX
{
  type: 'div',
  props: {
    className: 'card',
    children: [
      {
        type: 'h2',
        props: { children: 'Hello, World' }
      },
      {
        type: 'p',
        props: { children: 'Welcome back!' }
      }
    ]
  }
}

This is just a JavaScript object. No browser involvement. No layout recalculation. No repaint.

The key insight: React works with this cheap JavaScript tree first, and only touches the expensive Real DOM when it absolutely needs to.

Step 1 — Initial Render

When your React app loads for the first time, here's what happens:

1. React evaluates your components. Your top-level component (say, <App />) renders, its children render, their children render — all the way down the tree.

2. React builds a Virtual DOM tree. The result of all that rendering is a JavaScript object tree describing the full UI.

3. React translates this to the Real DOM. React takes the Virtual DOM tree and creates actual DOM nodes in the browser — createElement, appendChild, and so on.

4. The browser paints what it sees. The user now sees your UI.

flowchart TD
    A["Component Tree\n<App /> → <Header /> → <List />"]
    B["Virtual DOM Tree\n{ type: 'div', props: {...} }\nplain JS objects in memory"]
    C["Real DOM\n<div><header><ul>...\nbrowser creates and paints this"]

    A -->|"React renders components"| B
    B -->|"React commits to DOM"| C

At this point, React keeps a reference to the Virtual DOM tree it just created. This snapshot becomes important in the next step.

Step 2 — State or Props Change Triggers a Re-render

Now something happens. A user clicks a button. An API call returns data. A timer fires. Somewhere in your app, setState is called or a prop changes.

React needs to update the UI to reflect the new state. But it doesn't immediately reach into the Real DOM.

Instead, it does something much smarter.

Step 3 — React Creates a New Virtual DOM Tree

When state or props change, React re-renders the affected components — meaning it calls their render logic again and produces a new Virtual DOM tree representing what the UI should now look like.

This new tree exists entirely in memory. No browser touching. No layout. No paint. Just JavaScript objects.

flowchart TD
    A["User action\ne.g. button click"]
    B["setState called\nre-render scheduled"]
    C["Component function runs again"]
    D["New Virtual DOM tree produced\nexists only in JS memory"]

    A --> B --> C --> D

Now React has two Virtual DOM trees side by side:

• The old tree — snapshot from the previous render

• The new tree — just produced from the re-render

Both live in JavaScript memory. The browser has no idea any of this is happening yet.

Step 4 — Diffing (Reconciliation)

This is the heart of React's performance strategy.

React now compares the old Virtual DOM tree with the new one to find exactly what changed. This process is called diffing, and the broader algorithm that manages it is called reconciliation.

React walks both trees simultaneously, node by node, and asks: "Is this the same as before?"

flowchart LR
    subgraph OLD ["Old Virtual DOM"]
        O1["div.card"]
        O2["p: count 0"]
        O3["span: label"]
        O4["button: +"]
        O1 --> O2 & O3 & O4
    end

    subgraph NEW ["New Virtual DOM"]
        N1["div.card"]
        N2["p: count 1"]
        N3["span: label"]
        N4["button: +"]
        N1 --> N2 & N3 & N4
    end

    O2 -. "changed" .-> N2
    O3 -. "same" .-> N3
    O4 -. "same" .-> N4

Here's how React handles each node during diffing:

• If a node's type changed (e.g. a <div> became a <span>), React tears out the entire old subtree and builds a fresh one.

• If the type is the same but props changed (e.g. className or text content), React notes which specific attributes need updating.

• If nothing changed on a node, React skips it entirely.

Keys and list diffing

React also uses key props on list items to make this comparison smarter. Without keys, React might mistake an item being removed for all subsequent items shifting — causing unnecessary re-renders. With keys, React can identify exactly which item was added, removed, or moved.

// Without key — React can't tell which item changed
{items.map(item => <li>{item.name}</li>)}

// With key — React tracks each item precisely
{items.map(item => <li key={item.id}>{item.name}</li>)}

O(n) diffing

React's diffing algorithm is designed to run in O(n) time (linear) rather than the theoretical O(n³) complexity of a naive tree comparison. This is possible because of some practical heuristics React makes about how UIs actually change — for example, that components rarely change type, and that siblings at the same level are usually stable.

Step 5 — Applying Minimal Updates to the Real DOM

After diffing, React has a precise, minimal patch — a list of only the changes that need to be made to the Real DOM.

Maybe out of 200 nodes in your component tree, only one <p> tag's text content actually changed. React will update only that one node.

flowchart TD
    A["Diff result:\nOnly p text changed — count 0 to 1"]
    B["Real DOM\n· div — unchanged\n· p  ← UPDATED\n· span — unchanged\n· button — unchanged"]
    C["Browser repaints only the affected area"]

    A -->|"React writes minimal patch"| B
    B --> C

This is the key performance win. Instead of rebuilding entire sections of the DOM (or doing a wholesale innerHTML replacement), React makes surgical, targeted updates — touching only the nodes that actually need to change.

Why This Approach Improves Performance

Let's tie it all together.

The Virtual DOM isn't fast because JavaScript is fast (though that helps). It's fast because of what it avoids.

By computing changes in memory first — comparing two plain JavaScript trees — React minimizes the number of times it has to touch the Real DOM. And since Real DOM operations are the expensive part, doing fewer of them (and doing them only where necessary) is what makes React UIs snappy.

Think of it like a diff tool for code. Instead of rewriting an entire file when one line changes, you apply only the changed lines. The Virtual DOM is React's "diff tool" for UI.

The Full Picture: Render → Diff → Commit

Here's the complete React update lifecycle in one diagram:

flowchart TD
    TRIGGER["State / Props change"]

    subgraph RENDER ["Render Phase — pure computation, interruptible"]
        R1["Run component functions"]
        R2["Build new Virtual DOM tree"]
        R3["Diff old tree vs new tree"]
        R1 --> R2 --> R3
    end

    subgraph COMMIT ["Commit Phase — synchronous DOM writes"]
        C1["Apply minimal patch to Real DOM"]
        C2["Run useEffect / lifecycle methods"]
        C3["Browser paints updated UI"]
        C1 --> C2 --> C3
    end

    TRIGGER --> RENDER
    RENDER -->|"List of changes"| COMMIT

React keeps these two phases separate deliberately. The render phase is pure computation — no side effects, and in modern React it can even be paused or interrupted for higher-priority updates. The commit phase is where the actual DOM writes happen, and it runs synchronously to completion.

A Note on React Fiber

React's current architecture (React Fiber, introduced in React 16) builds on these ideas but goes further — it makes the render phase interruptible and prioritizable, so high-priority updates (like user input) can jump ahead of lower-priority ones (like background data fetches).

But Fiber's internal mechanics are a deeper dive for another article. The mental model above — Virtual DOM as a lightweight description, diffing to find minimal changes, committing those changes to the Real DOM — is the foundation that everything else is built on.

Summary

• The Real DOM is expensive to update frequently due to reflow, repaint, and style recalculation costs.

• The Virtual DOM is a lightweight JavaScript representation of the UI, kept entirely in memory.

• On initial render, React builds a Virtual DOM tree and translates it to the Real DOM.

• On state or props changes, React re-renders components and builds a new Virtual DOM tree in memory.

• Diffing (reconciliation) compares the old and new trees to find exactly what changed.

• React then applies a minimal patch to the Real DOM — touching only what needs to change.

• This approach dramatically reduces unnecessary browser work, keeping UIs fast and efficient.

The next time you call setState, remember: React isn't updating the DOM. It's updating a JavaScript object first, doing the math, and then making only the moves that matter.

Found this useful? Share it with someone learning React. Drop any questions in the comments below!

What we'll cover

The problem Kafka solves → message streams → topics → producers → partitions → consumer groups → how Kafka stays fast and safe. All with zero jargon.

What problem does the Kafka solve?

Modern apps are not simple request-response machines anymore. Think about what happens when you order something on Swiggy:

• The order service records your order

• The payment service charges your card

• The restaurant gets notified

• The delivery partner app updates

• Your notification system pings you

All of this happens simultaneously, triggered by a single button tap. How do you connect all these services without creating a tangled mess where every service talks directly to every other service?

Without Kafka, it's like every person in a stadium trying to whisper their message directly to everyone else. Chaos. With Kafka, there's one giant PA system that everyone can broadcast to — and anyone who cares can listen.

The old way was direct service-to-service calls. If the notification service is down, the order can't complete. Services become tightly coupled — break one, break all. Kafka decouples them entirely.

What Is a Message Stream?

Think of a river. Water flows continuously — it doesn't stop because you aren't looking at it. A message stream is exactly that: a continuous, never-ending flow of events.

Every click, every purchase, every sensor reading, every login — these are all events. They happen constantly, in real time. A message stream is just a way to capture all of these events as they happen and make them available to whoever needs them.

Imagine a live cricket match scoreboard. Every run, every wicket, every over — it's a stream of events. You can tune in now and see what's happening. Or you can rewind and check what happened in the 3rd over. The stream doesn't care when you arrive — it just keeps flowing.

Kafka is the system that captures, stores, and serves that stream to as many listeners as need it.

Kafka as a Central Message Pipeline

Instead of services talking to each other directly, every service talks to Kafka. Kafka sits in the middle — a central nervous system for your entire application.

Producers don't know or care who reads their messages. Consumers don't know or care who produced them. Kafka is the only thing they both trust. This is called loose coupling — and it's the foundation of every scalable system you've ever used.

Producers: Who Sends Messages

A producer is any application that sends (publishes) messages to Kafka. Think of producers as reporters — they have news to share and they file it with the news agency (Kafka). They don't worry about who reads the article. That's not their job.

A weather station is a producer. It measures temperature every second and sends the reading to Kafka: "Mumbai, 36°C, 2:30 PM." It doesn't know whether the AC company, the news app, or the agriculture dashboard will read it. It just keeps sending.

Producers are responsible for two things: what message to send, and which topic to send it to. That's it. Kafka handles the rest.

Real world producers

Your app's backend, IoT sensors, mobile apps, payment gateways, microservices, log aggregators — anything that generates events is a producer.

Topics: How Messages Are Grouped

Kafka doesn't just dump all messages in one pile. It organizes them into topics — named categories, like folders on your computer.

Think of a newspaper. There's a Sports section, a Business section, a Local News section. Producers (journalists) file their stories to the right section. Readers (consumers) subscribe to only the sections they care about. Topics work exactly like this.

You might have topics like orders, payments, user-events, inventory-updates. Producers write to a specific topic. Consumers read from a specific topic. Clean, organized, zero confusion.

Topics in Kafka are persistent — messages don't disappear after being read. They stay for a configurable amount of time (say, 7 days). This means a new service can join and replay the entire history from day one. That's powerful.

Partitions: Why Messages Are Split Internally

Here's where it gets really clever. Each topic is split into multiple partitions — think of them as parallel lanes on a highway.

Imagine a single-lane road from Mumbai to Pune. 10,000 cars, one lane — you'll be there tomorrow. Now imagine a 6-lane expressway. Same 10,000 cars, but they spread across lanes and move in parallel. Partitions are those lanes.

Each message within a partition gets a sequential number called an offset — like a page number in a book. Consumers bookmark their offset so they know exactly where they left off. Messages within a single partition are always in order. Across partitions, Kafka makes no ordering guarantee — but that's usually fine.

How does Kafka decide which partition?

If you send a message with a key (e.g., user-id: 42), Kafka hashes that key and routes all messages with the same key to the same partition — guaranteeing order for that user. No key? Kafka spreads messages across partitions evenly.

Why Kafka Is Fast and Scalable

Kafka handles millions of messages per second. Here's the honest reason why:

Sequential disk writes. Most databases do random reads and writes all over the disk — slow. Kafka only appends to the end of a log file, sequentially. Your hard drive was designed to do exactly this quickly. Kafka exploits this to its fullest.

Zero-copy transfer. Normally, reading a file and sending it over the network copies the data 4 times through memory. Kafka uses a Linux trick called sendfile that sends data directly from disk to network in 1 step. This alone gives it a 60–70% throughput boost.

Batching + compression. Rather than sending 1 message at a time, Kafka groups messages into batches, compresses them (GZIP/Snappy), and sends the whole batch in one network call. Fewer round trips = faster.

Horizontal scaling via partitions. Want to handle 10× more traffic? Just add more partitions and more consumer instances. Linear scale-out, no complex re-architecture needed.

Why Kafka is fast — summary

• Appends to disk sequentially (fast I/O)

• Zero-copy data transfer over network

• Batches messages, compresses payloads

• Scales horizontally by adding partitions

• Messages stay in memory (OS page cache) for hot reads

What are Consumer Groups ?

A consumer group is a team of consumers that work together to read a topic. Kafka divides the partitions among the group members so each partition is handled by exactly one consumer at a time.

Imagine 3 postmen delivering letters in a colony. Instead of all 3 delivering to every house (wasteful), each postman covers a different street. Together they cover the whole colony faster. That's a consumer group — divide the work, finish faster.

The magic rule: one partition → one consumer within a group. So if you have 6 partitions and 3 consumers in a group, each consumer handles 2 partitions. Add a 4th consumer? Rebalancing happens automatically — Kafka redistributes the partitions.

How Work Is Shared Across Consumers

This is where Kafka gets really powerful. Not only can one consumer group share a topic's load — multiple independent consumer groups can read the same topic simultaneously, each getting a full copy.

Group A's offset and Group B's offset are completely independent. Group B can be 10 minutes behind Group A — Kafka doesn't care. The messages sit there for both groups to read at their own pace. This is radically different from a traditional queue where a message is gone once one consumer reads it.

How Kafka Keeps Messages Safe and Ordered

Kafka's durability story is built on two pillars: replication and acknowledgment.

Replication

Every partition has one leader (handles all reads and writes) and multiple replicas (silent backups on other machines). If the leader crashes, one replica instantly becomes the new leader. Your data doesn't disappear — it was already copied. You configure the replication factor (usually 3 in production).

Acknowledgment (acks)

When a producer sends a message, it can ask for different levels of confirmation:

• Fire and forget — no confirmation. Fastest. Data loss possible.

• Leader confirms — one server confirms. Good balance.

• All replicas confirm — maximum safety. Use for financial data.

Ordering

Within a single partition, messages are strictly ordered — they are always appended and always read in sequence. Consumers track their position using offsets. If a consumer crashes and restarts, it picks up from the last committed offset — no message is skipped, no message is processed twice (with idempotent producers enabled).

Conclusion

So — What Is Kafka, Really?

Kafka is a distributed commit log that acts as the central nervous system of modern applications. It decouples producers from consumers, stores messages durably, scales horizontally through partitions, and lets multiple independent systems consume the same stream of events.

The one-line version: Kafka is a bulletin board that never erases anything, where everyone has their own bookmark, and new readers can start from the very first message.

What Is It?

The Thundering Herd Problem happens when a large number of servers simultaneously request the same resource that just became unavailable — most commonly, an expired cache key.

Think of it like a store opening at 9 AM. 500 people are waiting outside. The second the door unlocks — everyone rushes in at once. The shelves collapse. The staff can't cope. Not because there were too many people overall — but because they all arrived at the exact same moment.

Replace the store with your database. Replace the crowd with your app servers. Replace 9 AM with the moment your Redis TTL hits zero.

⚡ It's not a volume problem — it's a synchronization problem. A thousand requests over an hour is fine. A thousand requests in the same millisecond on a cold cache is an outage.

How It Happens

Here's the failure sequence when a cached key expires at peak traffic:

• T=0 · Cache key dropsT+1ms ·

• All servers: cache MISST+2ms ·

• All servers fire DB queriesT+50ms ·

• DB connection pool fullT+500ms ·

• Timeouts → retries (worse)T+10s · 503 errors · site down

The naive code pattern that causes this looks completely innocent:

//Looks fine. Will destroy your DB under load. 
async function getData() { let data = await redis.get('homepage'); if (!data) { 
// All 50 servers hit this at the same millisecond 
data = await db.query('SELECT * FROM articles LIMIT 20'); await redis.set('homepage', data, 'EX', 60); } return data; }

Real-World Examples

Hotstar / IPL — 35M concurrent viewers. A scorecard cache expiring mid-match means tens of thousands of simultaneous DB queries. Hotstar built custom thundering-herd-resistant layers for exactly this.

Netflix — A show drops at midnight. Millions check "What's New" simultaneously. If recommendation caches were set to expire at midnight, the herd triggers at peak traffic. Netflix uses probabilistic refresh to prevent it.

Stack Overflow — Documented using mutex-based cache refresh so only one thread rebuilds any given key, regardless of how many readers are waiting.

How to Fix It

Five battle-tested techniques — pick based on your use case:

🔒 Cache Locking (Mutex) — Only one server refreshes the cache. Others wait 50ms or return stale data. One DB query instead of fifty.

const lock = await redis.set('key:lock', '1', 'NX', 'EX', 5); if (lock) { // Only this server hits the DB 
const data = await db.query(...); await redis.set('key', data, 'EX', 60); await redis.del('key:lock'); } else { await sleep(50); // wait and retry — cache will be warm }

🎲 TTL Jitter (Simplest fix) — Add random variance to every TTL. Keys expire at different times instead of all at once. One line of code.

// Instead of a fixed 60s for everyone: 
const ttl = 60 + Math.floor(Math.random() * 20 - 10); // 50–70s 
await redis.set('key', data, 'EX', ttl);

🔀 Request Coalescing — 500 requests for the same key collapse into 1 upstream DB query. All 500 wait for that single result.

⏱ Exponential Backoff — Retry after 1s → 2s → 4s → 8s. Prevents retry storms from amplifying the failure.

♻ Background Refresh — Refresh cache at T+55s before the 60s TTL expires. Cache is always warm. Users never cause a miss.

📌 Key Takeaways

• Cache expiry + many servers = synchronized DB flood

• It's a synchronization problem, not a traffic volume problem

• Adding more servers makes itworse

• TTL jitter is the simplest fix — always add it

• Cache locking (mutex) is the most complete fix

• Background refresh gives the best user experience

Javascript, the Dynamic language which powers the web, is constantly evolving. While this evolution brings exciting new features, it can also create compatibility issues, older browsers might not support the latest Javascript features, leading to broken functionality and a frustrating user experience. This is where polyfills come to the rescue.

What is a Polyfill?

A polyfill is a piece of Javascript code that provides modern functionality in older browser environments which don’t natively support it. Think of it like a “Filling in the gaps” mechanism. It replicates a new feature's behaviour so your code can run smoothly across different browsers, ensuring a consistent user experience.

Why use polyfills?

The primary reason for using polyfill is to avoid situations where your web application or website doesn’t work on older browsers due to a lack of support.

Future-proofing code: Using polyfill, you can start modern Javascript features today. knowing that your code will continue to work even as browsers evolve.

Let us now create our polyfills.

We will start will forEach () function/method used in the iteration of the array.

ForEach() function/method Array instance executes a provided function once for each array element.

// Fist we will name the method myForEach, second we will check if the method exist in the array, 
// If it doesnt then only we can create it
if(!Array.prototype.myForEach) {
Array.prototype.myForEach = function(callback) {
 for(let i = 0; i < this.length; i++) { // we are using for iterating over the array and then 
            callback(this[i], i); // for each element the callback , which we have passed here as an argument
        }                         // callback function is called with two arguments
}
}
// now lets demonstrate it
const arr1 = [1, 2, 3, 4 , 5];
arr1.myForEach((e) => {
console.log(e)
}); // this will print each element in the array in the console.

The second polyfill we will have a look at is the map() function/method, instances create a new array populated with the results of calling a provided function on every element in the calling array.

// myMap method creation
// again we will check if it exists, if it doesnt then we will create a myMap()
if(!Array.prototype.myMap) {
Array.prototype.myMap = function(userFunction) {
const resultArray = []; //initialising an empty array
         // this points to an exisiting context
for(let i=0; i < this.length; i++) {
// now we will push the userFunction with two arguments which are index and the element of the array 
// by using this
resultArray.push(userFunction(i, this[i]);
}
return resultArray[] // returnning the new array 
}
};

// testing this myArray polyfill
const arr1 = [1, 2, 3, 4 , 5];
const newArr = arr.myMap((e) => e ** 2);
// 
console.log(newArr) // => the output will be [1, 4, 6, 8, 10]

The third polyfill we will create is the filter method in Javascript.

The filter() method creates a new array filled with elements that pass a test provided by a function.

// lets create our own filter method
if(!Array.prototype.myFilter) {
Array.prototype.myFilter = function(userFunction) {
// initializing an new array
const resultArray = [];
// now we will again use a for loop to iterate
for(let i=0; i < this.length /* currentArray length */; i++) {
// The loop will iterate with each element in the array;
if(userFunction(this[i]) {
resultArray.push(this[i)
// element in the array, the function userFunction (which is a callback function passed to myFilter) is called with the current element (this[i]). If userFunction returns true, the element is added to the res array.
}
}
return resultArray;
}
}

// now  check the myFilterFunction
const arr1 = [1, 2, 3, 4 , 5];
const newFilter = arr1.myFilter(e => e %2===0);
console.log(newFilter); // output will be 2 and 4 , as it is divisble by 2 and the remainder is 0.

Conclusion

Polyfills are an essential tool for JavaScript developers who want to build modern, feature-rich web applications that work seamlessly across all browsers. By understanding how polyfills work and how to use them effectively, you can write cleaner, more efficient code while ensuring a consistent and enjoyable user experience for everyone.

In the year 2405, the Starship Enterprise, under the command of Captain Jean-Luc Picard, embarks on a critical mission to establish peaceful relations with the mysterious alien civilization known as the Xelthorians. The Xelthorians possess advanced technology that could revolutionize Starfleet’s understanding of the cosmos. However, their language and data formats are unlike anything the Federation has encountered before.

To facilitate communication and data exchange, Captain Picard and his crew must serialize their data to send it across subspace communication channels and deserialize the data they receive from the Xelthorians.

Serialization: Preparing Data for Subspace Transmission.

Ensign Wesley Crusher, the young and brilliant Starfleet officer, is tasked with preparing the crew's data for transmission. He gathers crucial information about the Enterprise's crew and uses JavaScript to serialize it. This process converts the data into a string format, making it easy to send over subspace communication.

// Data to be serialized
const starFleetOfficer = {
name: "Jean-luc Picard",
rank: "Captain",
ship: "USS Enterprise",
species: "Human",
}
// Serialize the object 
const serializedData = JSON.stringify(starFleetOfficer)
console.log(serializedData);
// the output will be
// {"name":"Jean-luc Picard","rank":"Captain","ship":"USS Enterprise","species":"Human"}

Lieutenant Commander Data, the android officer renowned for his computational prowess, receives a transmission from the Xelthorians. The data arrives in a serialized format, a jumble of characters that only a machine could understand. Using JavaScript, Data deserializes the alien data, converting it back into a readable object.

Deserialization: Deciphering the data

Now Deserialization is a process to convert the JSON string into a Javascript object again, and it is done by using JSON.parse.

const deserializedData = JSON.parse(serializedData);
console.log(deserializedData);
// the output will be again 
/*1111
{
  name: 'Jean-luc Picard',
  rank: 'Captain',
  ship: 'USS Enterprise',
  species: 'Human'
}
*/

Here is how it works in a flow diagram

The exchange of bridging worlds

The bridge of the Enterprise buzzes with activity as Wesley and Data work together. The crew’s data, now serialized, is transmitted to the Xelthorians. Moments later, Data deciphers the incoming alien data.

// Crew data to be sent
const crewData = [
    { name: "William Riker", rank: "Commander", ship: "USS Enterprise", species: "Human" },
    { name: "Data", rank: "Lieutenant Commander", ship: "USS Enterprise", species: "Android" }
];

// Serialize the crew data
const serializedCrewData = JSON.stringify(crewData);
console.log("Sending data:", serializedCrewData);
// Output: Sending data: [{"name":"William Riker","rank":"Commander","ship":"USS Enterprise","species":"Human"},{"name":"Data","rank":"Lieutenant Commander","ship":"USS Enterprise","species":"Android"}]

// Data received from the alien ship
const receivedData = '[{"name":"Spock","rank":"Commander","ship":"USS Enterprise","species":"Vulcan"},{"name":"Uhura","rank":"Lieutenant","ship":"USS Enterprise","species":"Human"}]';

// Deserialize the received data
const deserializedAlienData = JSON.parse(receivedData);
console.log("Received data:", deserializedAlienData);
// Output: Received data: [ { name: 'Spock', rank: 'Commander', ship: 'USS Enterprise', species: 'Vulcan' }, { name: 'Uhura', rank: 'Lieutenant', ship: 'USS Enterprise', species: 'Human' } ]

Conclusion

Thanks to the combined efforts of Wesley Crusher and Lieutenant Commander Data, the Enterprise successfully establishes a data exchange with the Xelthorians. This achievement marks a significant step towards understanding and cooperation between two civilizations, bringing the galaxy closer to a future of peace and harmony.

With serialization and deserialization, the crew of the Enterprise bridges the gap between worlds, proving once again that in the vastness of space, knowledge and collaboration are the keys to unlocking new frontiers.

And so, the mission continues, boldly going where no one has gone before... 🖖😄

In the world of programming, objects play a crucial role, they are fundamental building blocks which assist in structuring and organizing code. Objects play a pivotal role in Javascript, enabling developers to create complex structures and manage information efficiently. In this article, we will explore the core concepts of Javascript objects, their properties, and various use cases.

What is a Javascript Object?

A Javascript object is a collection of key-value pairs, where each key is known as a property name and is associated with a value. These values can be of any data type, including strings, numbers, arrays, or even objects. Objects are versatile and provide a way to group related data and functionalities.

Here is a way of declaring a Javascript object:

const person = {
name: "Spiderman",
age: "30",
alias: "Peter parker",
occupation: "Super hero",
}

Accessing and modifying properties of Object -

You can access and modify the properties of a Javascript object using either a dot notation or bracket notation. Let’s take a look at both approaches:

Dot notation

console.log(person.name); // output will be Spiderman
// modifying the age of spider man, lets make him old
person.age = 40;
console.log(person.age) // output will be the age is 40

Bracket notation

console.log(person["occupation"]); // output will be Super hero
// let us change it to photographer
person["occupation"] = "Photographer";
console.log(person["occupation"]); // Out put will be Photographer

Adding and deleting properties

Javascript objects are dynamic, meaning you can add or delete properties even after the object has been created.

Adding a property

// lets add nemesis of spiderman to the list
person.nemesis = "Green goblin";
console.log(person.nemesis); // output: green goblin

Deleting a property:

// lets delete or terminate the nemesis
delete person.nemesis;
console.log(person.nemesis); // output: undefined

Methods in Javascript Objects

Methods are functions that are associated with objects. They allow objects to perform actions and manipulate their own data. You can define and use a method within an object:

const person = {
name: "Spiderman",
age: "30",
alias: "Peter parker",
occupation: "Super hero",
greet: function() {
console.log("Hello, my name is " + this.name);
},
league: {
avengers: "yes",
},
};

person.greet() // Output will be, Hellom my name is Spiderman

Iterating in objects

To iterate over the properties of an object, you can use a for...in loop. This will allow to access each key-value pair within the object.

for (let key in person) {
  console.log(key + ": " + person[key]);
}

Copying objects

There are several ways to copy Javascript objects, ranging from shallow to deep copies.

Shallow copy

A shallow copy of an object creates a new object that references the same memory locations as the original object for nested objects or arrays. This means that changes to nested objects or arrays in the copied object will also affect the original object and vice versa.

Example with Object.assign();

const newPerson = Object.assign({}, person);

// modifying the nest object in the shallow copy
newPerson.league.avengers = "No";
console.log(person.league.avengers); // No
console.log(newPerson.league.avengers); // No

Example with spread operator.

const newPerson = {...person}

// modifying the property in the shallow copy
newPerson.league.avengers = "Maybe";
console.log(person.league.avengers); // Maybe; 
console.log(newPerson.league.avengers); // Output will be Maybe

Deep copy

A deep copy of an object creates a new object with a completely separate memory location for all nested objects and arrays, this means that changes to the objects, nested objects and arrays in the copied object will not affect or change the original object and vice versa.

Example with JSON methods:

One common method to create a deep copy is by using JSON.stringify() and JSON.parse(). This method converts the object to a JSON string and then parses it back into a new object.

const deepCopy = JSON.parse(JSON.stringify(person));

// modifying the value of the object in the deep copy
deepCopy.league.avengers = "No";
console.log(person.league.avengers) // Maybe
console.log(deepCopy.league.avengers) // No

Understanding Objects is very important for writing robust and maintainable JS code, especially when you are working with large and complex data structures.

What is Javascript?

Have you ever wondered, when you click on a button for example pay button, you are taken to a form and then you fill the form on the website, and the payment is made, my friend that is nothing but Javascript working behind the scenes of the web application or web site 😱. Yes, you heard it right Javascript is the one that makes the website interactive for users. Now you know that, we will look into some of its basics and features.

Welcome to the magic world of Variables

A thought must have come to your mind, what are variables, in layman's terms variables are nothing but containers where you can store your data or values. They are also known as identifiers.

Here you can see various containers or boxes and their values.
There are two ways of declaring variables: the Implicit and the Explicit ways.

Explicit way

The word explicit itself says that we are specifying a keyword/identifier before the variable name.

There are three ways in which you can declare a variable in JavaScript. var let and const. Before getting to that, we should know how to declare variables and follow certain standards.

Var

var a = 10;

when we console.log here we will get the value of a is 10. Var is the global scope, which means when we declare a with the var keyword, it is accessible everywhere in the javascript program.

But var is loosely scoped, which means it can be re-declared or re-assigned, what it did was it caused problems in behaviour, bugs, and memory leaks, especially in large-scale projects and complex codebases.

var name = "ojas"

console.log(name);

function greet() {
console.log(`Welcome ${name}`}
}
greet(); // When I call greet function here it is taking the name reference from the top.

The limitation of var is used to show unexpected behaviour in the program, which results in bugs, mostly in large and complex projects. To solve that two identifiers were introduced in Javascript standards and those were let and const.

let

let identifier was introduced in ES6 standards, it is re-assignable, block-scoped, optionally initializing each to a value.

// demonstrating this through a code snipet
let x=10; 
console.log(x)  // initialized with a value 10.
// let x = 20; 
// Uncaught SyntaxError: redeclaration of let x . What this says is you cannot redeclare x again.
// we are also able to change the value here
x = 20;
console.log(x) // so here it will give you x = 20 here. so we can change the value of x which is declared by using let keyword.
// But there is more, when we declare x again inside a curly braces , then lets see what happens
{
 let x = 30;
 console.log(x) // x = 30;  here something thing happened, we were able to re declare the x variable and assign different value to it.
}

const

const identifier was also introduced in ES6 standards, It is block scoped local variable. The value of const cannot be changed/reassigned but if it is an object then the values can be added, updated or removed.

const x; // We need to initialize the value of a const, it cannot be kept uninitialized
const x = 20;
x = 30; // in this case this will give an error which will tell value of const cannot be reassigned.

const names = ['Abhijit', 'Bipin', 'Ojas'];
names.push('Aniket') // here we are adding Aniket in the array which is behind the scenes an object so, we can add this value in the array even if it is defined using a const keyword.

Implicit Declaration

When a user declares or assigns a value to a variable which is not declared by using var, let or const, Javascript declares that variable for you, but caution, when you declare a variable implicitly then it is always global scope, even if it is declared in a function body.

// example of declaration of variable implicitly
name = "Abhijit"
console.log(name) // this will print Abhijit in the console.

Difference between var let and const.

This is a brief introduction to variables in JavaScript and how they work.

Datatypes in Javascript

There are two main sets of data types in JS, those are primitive and non-primitive data types.

Primitive data types

• String: Represents a sequence of characters, for example, "Hello, World!".

• Number: Represents both integer and floating-point numbers, for example, 42 or 3.14.

• Boolean: Represents a logical entity and can have two values: true or false.

• Undefined: A variable that has been declared but not yet assigned a value.

• Null: Represents the intentional absence of any object value. It is treated as falsy for boolean operations.

• Symbol: A unique and immutable primitive value used as the key of an object property. For example, Symbol('description').

• BigInt: Represents integers with arbitrary precision. It can handle values larger than the Number type. You can create a BigInt by appending n it to an integer, for example, 123n.

Non-primitive data types

• Objects

• Arrays

// Object data type
const person = {
name:"Abhijit",
age:"20",
}

// array data type
const fruits = ['Apple', 'Banana', 'Grapes']

This is a brief introduction to data types in JavaScript.

This is the year 1987; we are at Jungle base.

let level1 = ['grunt', 'grunt', 'flying-soldier', 'grunt'];
// we need to engage now
console.log(level1, "Engage");

Methods in arrays

Shift()

Our hero, Bill needs to take out a few enemies. here we use the shift method, which removes element from the start.

level.shift();
level.shift();
console.log(level1) // output is level1=['flying-soldier', 'grunt'] 'grunt' and 'grunt' are terminated

Push()

Now we go ahead in the level, and new enemies appear, here we use the push method, so that we can insert new elements in the level1

level1.push("Soldier-sniper", "scuba-soldier", "Boss");
console.log(level1); // here we get level1=['flying-soldier', 'grunt', 'Soldier-sniper', 'scuba-soldier', 'Boss']

indexOf()

Now I want to locate where is the enemy in the jungle stage, so for this, we will use the index of method which will give me the position of the ‘Boss’ in the level1 array

console.log(level1.indexof("Boss"); // the index will be 4. so my final enemy is located at the last.

Concat()

Now we have to give the bill a power, but it is in a different section of the level, we need to bring it to level 1, how can we do that, We use the concat method, which will merge two arrays

let powerups = ['S-wide range bullets', 'L Laser-gun', 'grunt'];
console.log(level1.concat(powerups)); //  here we get level1=['flying-soldier', 'grunt', 'Soldier-sniper", 'scuba-soldier', 'Boss', 'S-wide range bullets', 'L Laser-gun']

filter()

Now we must remove Grunt from the level 1 jungle, so what should we do? we should perform the filter method; it will return a new array which passes the test case

let onlyGrunts = level1.filter((e) => e === 'grunt'); // it will filter out grunts

pop()

Now we need to eliminate mid-level enemies like “scuba soldiers”, and it is placed at the end of the enemies array, how will we do that, we will use the pop method, pop removes the last element in the array

let enemies = ['Grunts', 'Soldier-sniper', 'Scuba-soldiers'];
console.log(enemies.pop()); // output is the last element ie the Scuba-soldier is removed from the enemies list

includes()

Now, Bill needs to check if the enemies contain sniper-soldier and how he will be able to do that. we use the includes method, This will check if the array consists of that specific value or not

console.log(enemies.includes('Soldier-sniper')); // This will give the result as true to Bill./

map()

We need to describe each element in the level1 array like what bill has encountered

// let level1 = ['grunt', 'grunt', 'flying-soldier', 'grunt'];
console.log(level1.map((item => item + ' encountered!')); //  let level1 = ['grunt encountered', 'grunt encountered', 'flying-soldier encountered', 'grunt encountered'];

slice()

now Bill needs to extract a portion of the level, we can achieve this by using the slice method, which extracts a section of an array and returns a new array

let levelSection = level1.slice(0, 2); // Extracts elements from index 0 to 2
console.log(levelSection);

unshift()

Now we need to add some staring area like a wall in the level1 array, how are we going to achieve it by using the unshift method, Adds one or more elements to the beginning of an array, like adding a new starting area to the level

level1.unshift('wall', 'wall');
console.log(level1) // The output will be level1 = ['wall', 'wall', 'grunt encountered', 'grunt encountered', 'flying-soldier encountered', 'grunt encountered'];

Conclusion

Here I have tried to explain few method of arrays in javascript in a fun way.

The Internet is a worldwide network linking various devices, including (computers, mobiles, tablets, and gaming consoles like PS5 and Xbox). Consider it a freeway, where traffic(Data) from one plays to another and vice versa from millions of endpoints. Everything is connected via cables and it’s an extensive network for connecting the entire globe, a large amount of cables are laid under seas and oceans, as shown in the figure below.

Image credits: https://www.submarinecablemap.com/

Working the Internet.

Now that we have a basic idea of what the Internet is, let us dive deep and understand, how it works.

Physical Medium: A device connects through the medium of Wires.

a Fibre Optic Cables: These are used for long-distance connectivity, which transmits the data as light.

b Ethernet cables: These are used for short-distance connectivity, which also ensures direct connectivity and fast transmission.

Explaining it further

• Physical devices like Computers, Mobile, and Tablets send and receive the data.

• Internet Service Provider(ISP): These service providers play a key role in or you can say it is a medium which helps you connect to the internet.

• Routers: Routers are nothing but a mechanism which guides your signal or information to reach the proper source.

• Servers: Servers serve you the information, like a restaurant person getting your food and serving it. It also holds the information.

The basic flow of how the internet works is shown in the figure above.

Travel journey of the data

Consider an example when you hit www.google.com in your browser what does happen exactly behind the scenes? Let’s dive in.

1 User enters www.google.com in the browser, but the computer doesn’t understand www.google.com, what does the internet do it asks where can I find this www.google.com.

2 (Tada!) Here comes the DNS(Domain name service). This is a digital phonebook or Yellow Pages of the internet. It translates google.com to an address that the internet understands ie 142.250.191.78. so the whole crux is the DNS does is simply translate the google.com to an Internet Protocol address, which is then read correctly by the internet.

3 After the DNS is done locating the address, the web request is made in action.

• Routers: This gives a proper direction to the request.

• ISP(Internet Service Providers): These are the ones issuing connections and initiating the requests through them to the global network.

• Servers: The end destination where the request is fulfilled and the response is sent back to the client's computer.

Request/Response Cycle

The above figure shows the dataflow via the internet to the server and back.

Conclusion

This is an overview of how the internet works.

Let’s begin from scratch. What is GIT?
Git is a version control system designed to track project file changes. It also helps you collaborate with other developers, so you can work in sync. It was developed by Linus Torvalds in 2005. It also allows or is used in integrating code snippets from different branches. Terms like Branches and Merges can be a little overwhelming a bit, but relax in this tutorial you will be familiar with these terms/jargon. Git is an open-source software, where you can install it on Windows, Linux and Mac.

What is a Repository?
A repository is nothing but a folder, where your files are stored. The repositories are of two types.

a) Local Repository - The folder which is present locally ie on your computer

b) Remote Repository - The folder which is located on the central server(Accessible to all).

Getting Started with Git.

To install git you can visit the official website https://git-scm.com/downloads.

You will land up here, and then according to your operating system, you can download the specific version of GIT. GIT is free to use and open source.

After you have done the installation, you can check if the git is installed properly or not by using this command in your command prompt or bash shell.

git --version
git version 2.24.0.windows.2

You will see the version details like “git version 2.24.0.windows.2**” which is also shown in the above snippet.**

Creating an account with GitHub

One more thing we need to do is create an account with Github. A brief introduction to GitHub

GitHub is a web-based platform that uses Git for version control. It allows developers to host and manage their code repositories, collaborate on projects, and track changes. Think of GitHub as a social network for programmers, where you can share your projects and contribute to others' work.

Step 1

Go to GitHub and you will see the homepage.

Step 2
Click on the Signup button on the top right corner. You will proceed with filling out the information given in the image below.

Step 3
After successfully creating an account, You will be able to sign in to the GitHub website or app.
Now that we have covered this basic sign-up tutorial, we will move ahead to understanding the basics of Git, commands like Git init, Git add, Git commit and so on.

Git Basics Guide

We will be going through some commands
1 Initialization

Create a folder named chaicode-cohort

go into that folder.

// for initialzing the git repository type the following command
git init

what this will do is, it will create a .git folder in your chaicode-cohort folder and also which will be hidden

You can see it in the above image, for viewing all the folders in the chain code-cohort, we need to run the command ie using the git bash is ls -a, it will show you all the subdirectories including the .git folder.

When you open this folder in your code editor, ie the VS code editor, this is how it will appear

Now if you closely look, By default git will show U besides the file name, which means the files are not being tracked, now to track those files we need to add them to git or let git know that, start tracking those files.

for that we use

git add filename // for single file 
git add . // for multiple files

This is also known as the staging stage, that is we are making git aware that these files are the ones you should keep track of.

// now suppose I want to track a single file.
git add one/info.txt

after using that command now you can see the info.txt U ie untracked status is changed to A, now it will be tracked or staged for commit. But this doesn’t work for large file sets, so we use

git add . // This commands adds everything to staged for commit

Here, when I use that command then you can see every untracked file U is changed to A ie it will be tracked from now on.

Now that the files are staged and ready to be committed in the git folder, we will commit them and specify a message, so that it can be easy for us to keep track of the commit.
for that, we use the command

git commit -m "First commit for chaicode-cohort"

When I use the commit command with a message, you can see that there are 3 files changed and 2 insertions are made.

For viewing the Commit ID, we can use the git log

git log

When I use this command it gives me the information regarding the commit. like, commit id, Author, Message and date.

Now what if I make a change in info.txt and I want to see what change I made then “git status” is the command you will use

git status

You can see in the above image that I have made some changes in the info.txt and git is amazingly tracking it and showing it to me when I use the git status command.

after committing the new change and logging, we can see that my head has shifted to the new commit.

Now that we have the gist of how git works. we will move ahead to Github.

We will create a Remote repository accessible to all of our peers.

GitHub basics

Now we need to push the local repository to the centralized place where our peers can access it.

Go to GitHub and log in. You will see the home screen.

Here you can see the new button. click on it.

In the above image, enter the repository name, chaicode-cohort in our case and then click on Create the repository.

Now we see the repository is created in GitHub, now we need to push our existing repository over here

so first we need to add the remote folder or remote origin folder here.

git remote add origin https://github.com/abhijitWebDev/chaicode-cohort.git

after adding the folder we need to set the branch ie in our case the master branch

git branch -M main

after setting the branch to main, now we are ready to push the code to the centralize folder

git push -u origin main

after using this command our local repository is also uploaded in our central repository.

The above image shows that it is successfully uploaded and now anyone can use this by cloning it.

I hope this is helpful.

Things to be followed while committing to the central repository

1 The commit message should be clear and the description should be to the point

2 The message body should contain whether it is a feature / bug-fix / documentation-improvement

3 The message body should contain the task id for ex “Ch-5506” like a task id.

4 Always take a pull from the main branch or release branch.

5 Create branches from the release branch or main branch.

6 Commit your changes more frequently.

This concludes the tutorial.

State management is a fundamental aspect of building modern web applications with React. It involves managing the data that drives component behaviour and appearance dynamically. From simple values to complex objects, a React state reflects the application’s current UI and logic state.

Here are some key points about state management in React:

1 Component State

• React Component can have its own local state.

• The state is mutable and can be updated with the setState method.

• Example

    import React, { useState } from 'react';
  
    function Counter() {
      const [count, setCount] = useState(0);
  
      const increment = () => {
        setCount(count + 1);
      };
  
      return (
        <div>
          <p>Count: {count}</p>
          <button onClick={increment}>Increment</button>
        </div>
      );
    }
  

  This code snippet is a React functional component called Counter. It uses the useState hook from React to manage state. The useState(0) initializes a state variable count with an initial value of 0, and setCount is the function used to update the count state.

  The component renders a <div> containing a <p> element that displays the current value of count, and a <button> element labeled "Increment". When the button is clicked, the increment function is called, which updates the count state by incrementing it by 1. This triggers a re-render of the component with the updated count value displayed.

  2 Lifting the state up

• To share the state between components, you can lift it up to a common ancestor.

• This allows multiple child components to access and modify the same state.

Example

function Parent() {
  const [message, setMessage] = useState('');

  return (
    <div>
      <Child1 message={message} />
      <Child2 setMessage={setMessage} />
    </div>
  );
}

This code snippet defines a React functional component called Parent. Inside the component, it uses the useState hook to create a state variable message initialized with an empty string and a function setMessage to update the message state.

The Parent component returns a <div> element containing two child components: Child1 and Child2. It passes the message state variable as a prop to Child1 and the setMessage function as a prop to Child2. This allows communication between the parent component and its child components by passing data and functions as props.

Why is State management needed?

State management is essential in React for several reasons:

1. Maintaining Component State: React components can have an internal state that determines their behaviour and appearance. State management allows components to store and update their state, triggering re-renders when the state changes.

2. Handling User Input: State management is crucial for handling user interactions like form inputs, button clicks, and other events. By managing state, React components can respond to user input and update the UI accordingly.

3. Managing Data: React applications often need to fetch and display data from APIs or other sources. State management enables components to store and manage this data, ensuring that the UI reflects the latest information.

4. Passing Data Between Components: State management facilitates passing data between components in a React application. By lifting the state up to a common ancestor or using context, components can share and update data effectively.

5. Optimizing Performance: Efficient state management can help optimize performance by minimizing unnecessary re-renders. React's reconciliation process compares the current state with the previous state to determine what needs to be updated in the DOM.

Overall, state management in React is crucial for building interactive, dynamic, and data-driven applications while maintaining a clear and predictable flow of data and updates throughout the component hierarchy.

In the further blogs, we will analyze various tools of state management.