What did the parent type or say, and is this home-made, restaurant, or packaged food?
Text, voice-transcribed text, photo + notes, or barcode enters Add Meal. User-selected home_made is carried into both parsing and nutrition estimation.
- In
- raw phrase / image, mode
- Out
- cache identity + parser request
Failure, impact & fix
Failure: wrong source mode. Impact: restaurant/packaged priors may be applied incorrectly. Fallback: home-made is the default. Fix: expose the selected mode at confirmation and persist source provenance.
Have we seen this exact phrase before and already know what it means?
AsyncStorage key includes mode, source lane and normalized text (or image hash + note hash). A draft is reusable only after 5 observations, 4 matching signatures, stable grams/recipe, and before its 30-day expiry.
Validated hit?YES → skip primary parse AINO → Step 03
Failure, impact & fix
Failure: stale or semantically over-broad cache key. Impact: repeatable wrong structure. Fallback: schema/model/expiry guards reject incompatible entries. Fix: add prompt-version and locale to identity; instrument cache hit quality.
AI breaks down the phrase: what food is this, how much, what unit, and how was it prepared?
parseStructuredMealWithGemini requests name, canonical name, amount, unit, food type, quantity kind, estimated grams, per-100g nutrition, modifiers, recipe assumption, qualifiers and expected calorie range.
Likely interpretationmakhana · amount 1 · unit katori · cooked_dish/direct_food? · qualifier roasted
Failure, impact & fix
Failure: timeout, malformed JSON, hallucinated amount or lane. Impact: every downstream choice can drift. Fallback: simple local structure parser; image flow can retry image then text notes. Fix: retain raw-vs-parsed diff, enforce confidence thresholds, ask user when food type changes the recipe path.
background, non-blocking
Remember this interpretation for next time (happens in the background, doesn't slow you down).
Fresh Gemini drafts are accumulated in AsyncStorage. Structural signature, estimated-gram spread, calorie spread, and cooked-dish ingredient Jaccard similarity decide whether a future validated entry can be written.
Failure, impact & fix
Failure: background write fails. Impact: cost/latency only; current meal continues. Fallback: next request calls AI again. Fix: telemetry for write failures and bounded retry.
What utensils does this family use? What's their katori size? What staples (ghee, milk) do they have saved?
Load child-specific serveware and essentials from Firestore, normalize profiles, fill absent fields from system defaults, and pre-warm the food DB: memory → AsyncStorage (30-day) → Firestore nutrition_foods → bundled database.
Failure, impact & fix
Failure: Firestore offline or profile absent. Impact: personalization is lost. Fallback: system serveware/default essentials and bundled food DB. Fix: show a “using defaults” trust flag and sync-age.
What exactly is this food? Match it to a known food in our database — check aliases, regional names, and spelling variations.
Normalize the name, consult regional dish aliases, kitchen profile terms and top local DB matches. Exact matches win. Fuzzy matches require ≥60% query-token coverage and are scored for confidence.
Confident exact/local?YES → accept identityNO → cache / Gemini validation
Failure, impact & fix
Failure: “roasted makhana” maps to raw seeds or another regional alias. Impact: wrong density and nutrition basis. Fallback: up to five candidates go to identity validation. Fix: preserve preparation tokens in identity tests and expand makhana-specific aliases.
If we're unsure about the food identity, check if other users have already confirmed it, or ask AI to verify.
Firestore alias cache is checked first. Otherwise Gemini gets candidate IDs and has a 2-second race timeout. A non-low-confidence choice is written back asynchronously for all users.
Failure, impact & fix
Failure: cache poisoning or AI timeout. Impact: wrong shared identity or weaker local match. Fallback: top local/fallback rule or unresolved low confidence. Fix: version aliases, record evidence and add admin rollback.
How much food was eaten? If grams are given, use them. If it's "1 katori" or "2 pieces", convert to grams using the right method.
Mass/volume wins: g, kg, ml, l. Count units use a food-specific per-item weight. Serveware invokes its own branch. Missing or weak quantities may accept parser estimatedGrams; otherwise food default or 100g.
Failure, impact & fix
Failure: “1 katori” parsed without unit. Impact: generic default can replace vessel logic. Fallback: AI estimated grams, food default, then 100g. Fix: block confirmation for unrecognized quantity words.
A “katori” means different grams for different foods (rice is heavier than makhana). Use the family's katori size × how densely this food fills it.
Resolve “katori” alias. Use the child’s configured vessel amount when present, else the system vessel default. Then find a food-specific density/filled-mass from stable cache or estimate it with Gemini. The result is mass of roasted makhana in the katori, not a raw-piece count.
conceptuallykatori capacity × makhana packing/density → assumed_g
Failure, impact & fix
Failure: vessel size, fill level, packing, or density is wrong. Impact: every nutrient scales linearly by the same percentage. Fallback: user vessel → system default; stable density → AI density → local food density/default. Fix: let user calibrate empty/full vessel mass and choose “loosely filled / level / heaped.”
parallel hardening after a fast density estimate
In the background: confirm the density estimate over time. If it's been estimated many times consistently, lock it in for future use.
Validation attempts are recorded; stable density entries are reused and expire after 30 days. Repeated checks happen off the immediate response path.
Failure, impact & fix
Failure: unstable AI consensus. Impact: future requests keep estimating. Fallback: current fast estimate remains usable. Fix: store distribution/sample count, not only a point estimate; use measured reference data for common foods.
Is this a single ingredient (egg, milk, almond) or a cooked dish with a recipe (dal, omlette, paratha)?
Use structured draft plus local category/rule/profile signals. If no deterministic classification exists: memory cache → 90-day Firestore cache → Gemini. Failed classification returns ambiguous / low.
For roasted makhanaDIRECT → prepared food nutritionDISH → recipe decomposition
Failure, impact & fix
Failure: roasted snack misclassified as a multi-ingredient dish or raw direct food. Impact: recipe/oil may be added or omitted. Fallback: ambiguous path attempts safest available resolution. Fix: encode “dry roasted / oil roasted” as an explicit preparation decision.
DIRECT FOODCOOKED DISH
For single foods: look up nutrition per 100g directly. If it's a kitchen staple the family saved, use their specific brand/type.
Kitchen essential effective nutrition has priority when matched. Otherwise trusted parser nutrition / nutrition cache / Gemini estimate can supply per-100g values. Preparation modifiers may adjust the basis.
Failure, impact & fix
Failure: raw makhana basis used for oil-roasted snack. Impact: added fat/calories omitted. Fallback: qualifier-aware AI estimate. Fix: require explicit oil/ghee choice or default assumption surfaced to user.
For cooked dishes: find or build the recipe — what ingredients go in, and in what proportions?
Priority includes parser draft, versioned/curated/static/shared validated recipes, then Gemini repair/decomposition. Ingredients are expressed as raw amount per 100g cooked plus optional evaporation and modifiers.
Failure, impact & fix
Failure: “home made” contains unknown oil/ghee and salt. Impact: recipe is plausible, not household truth. Fallback: repaired/decomposed recipe or direct estimate. Fix: one-tap recipe confirmation: dry / oil / ghee and amount per batch.
For each ingredient in the recipe, find its nutrition per 100g — preferring the family’s saved brands/types over generic values.
For recipe ingredients, household kitchen profiles and packaged staple nutrition override generic values. Otherwise use known ingredient essential/static database values; self-healing AI can fill an unrecognized ingredient. Raw-to-cooked scaling is applied where packaged/raw density exceeds cooked expectations.
Failure, impact & fix
Failure: raw material missing, wrong brand, or raw/cooked basis confused. Impact: systematic recipe error. Fallback: static essential → local DB → bounded AI scaling/estimate. Fix: attach source citation and raw/cooked state to each ingredient; reject silent basis changes.
Some ingredients (rice, dal) have "raw" nutrition on the label but are eaten cooked. Convert raw values to cooked equivalents.
For grains/pulses and cooked-rule foods above 200 kcal/100g, use admin override, permanent Firestore factor cache, then bounded Gemini raw_per_100g_cooked; local recipe scaling remains the fallback.
Failure, impact & fix
Failure: permanent cache captures a bad factor. Impact: all future users inherit it. Fallback: bounds reject extreme AI values. Fix: expiry/version/cache audit and sample-based validation.
Add up all ingredient contributions, account for water loss during cooking, and calculate nutrition per 100g of the finished dish.
Scale each ingredient’s nutrient basis by its raw contribution, apply ingredient modifiers, sum, and account for yield/evaporation. This produces the dish’s normalized nutrients_per_100g.
Failure, impact & fix
Failure: yield, oil retention, or recipe ratios differ. Impact: per-100 density is wrong before portion scaling. Fallback: best-effort recipe/AI estimate. Fix: persist household recipes with batch raw weights and final cooked yield.
Multiply: nutrition per 100g × actual grams eaten ÷ 100. This gives the final calories, protein, fat, etc. for what the child actually ate.
Sanitize non-finite/negative values, then calculate each nutrient as per-100 value × assumed_g / 100. Attach item trust/provenance and sum item nutrients into meal totals.
makhana exampleNmeal = Nroasted makhana / 100g × resolved katori grams ÷ 100
Failure, impact & fix
Failure: plausible but wrong upstream values pass sanitation. Impact: precise-looking wrong result. Fallback: validator flags obvious mismatch only. Fix: uncertainty interval propagated from quantity + recipe + source confidence.
Does the result make sense? Check if calories from protein + fat + carbs add up correctly. Flag if something looks impossible.
Check macro-derived calories (4/4/9) against stated calories at a 25% threshold, near-zero food, expected calorie range, and low-confidence AI interpretation. Mark resolved, best_effort, or failed; do not automatically repair.
Failure, impact & fix
Failure: correlated errors remain internally consistent. Impact: validator accepts wrong values. Fallback: trace remains visible in saved data. Fix: independent reference-range validation by canonical food/preparation.
Save the final meal with all its nutrition, the source of every number, and the engine version — so we can trace and improve later.
addMeal re-sanitizes items, recomputes item and meal totals, attaches source mode and calculation/canonicalization/recipe versions, then writes under the child’s meals collection with trace fields.
Failure, impact & fix
Failure: write fails or schema strips a new field. Impact: meal is not stored or loses provenance. Fallback: UI reports failure; no offline write queue is evident in this path. Fix: queued idempotent writes, schema contract tests and save receipt.