deckyvault/docs/2025-05-05-sync-pwa-plan.md

Sync Unification, Nav Cleanup & PWA Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Unify all Steam game data pipelines through a single syncSteamGame function, remove the Updates nav link, and lay PWA groundwork.

Architecture: Adopt a stub-then-sync pattern: insert a minimal database row (appId + placeholder title), then immediately call syncSteamGame which populates all fields including reviews, error tracking, and playability. Two divergent insert locations are collapsed into one canonical path. PWA tasks are out of scope for this plan (per spec §7 — separate planning cycle).

Tech Stack: Next.js 15 (App Router), Elysia (API), Drizzle ORM (PostgreSQL), Vitest (testing)

---

File Structure

Action	File	Responsibility
Modify	lib/steam/sync.ts	Expose ensureSteamGame — the new unified entry point
Modify	lib/api/game-stub.ts	Replace direct Steam fetch + insert with ensureSteamGame
Modify	app/game/[id]/page.tsx	Replace createGameStub with ensureSteamGame
Modify	lib/routes.ts	Remove Updates nav entry
Create	lib/steam/__tests__/ensure-steam-game.test.ts	Unit tests for the new unified function

No new files beyond the test file. The PWA scope is explicitly deferred.

---

Phase 1: Expose ensureSteamGame in sync.ts

This phase creates the new unified entry point. It does not yet consume it — that happens in Phase 2. This isolates the core logic change so it can be tested independently.

Task 1: Add ensureSteamGame to lib/steam/sync.ts

Files:

• Modify: lib/steam/sync.ts

• Step 1: Add the ensureSteamGame function after the syncSteamGame export

Add the following function at the end of lib/steam/sync.ts (after the syncSteamGame function):

/**
 * Ensure a Steam game exists in the database with full data.
 * If the game doesn't exist, inserts a minimal stub then runs a full sync.
 * If the game already exists, returns it without re-syncing.
 *
 * This is the SINGLE ENTRY POINT for creating new Steam game records.
 * Use this instead of direct Steam API fetches + manual inserts.
 */
export async function ensureSteamGame(
  steamAppId: number
): Promise<{ game: typeof games.$inferSelect | null; created: boolean; error?: string }> {
  // 1. Check if game already exists
  const [existing] = await db
    .select()
    .from(games)
    .where(eq(games.steamAppId, steamAppId))
    .limit(1)

  if (existing) {
    return { game: existing, created: false }
  }

  // 2. Insert minimal stub
  const [stub] = await db
    .insert(games)
    .values({
      steamAppId,
      source: "steam",
      title: `Steam App ${steamAppId}`,
      storeUrl: `https://store.steampowered.com/app/${steamAppId}`,
      syncStatus: "pending",
    })
    .returning()

  // Handle race condition: if another request inserted first, UNIQUE constraint
  // on steamAppId will throw. Catch and return the existing record.
  if (!stub) {
    const [raceWinner] = await db
      .select()
      .from(games)
      .where(eq(games.steamAppId, steamAppId))
      .limit(1)
    return { game: raceWinner ?? null, created: false }
  }

  // 3. Run full sync (forceRetry=true to bypass staleness check on a brand-new record)
  const syncResult = await syncSteamGame(steamAppId, { forceRetry: true })

  if (!syncResult.success) {
    // Sync failed — return the stub (it has error tracking fields populated by
    // recordSyncFailure inside syncSteamGame). Re-fetch to get updated fields.
    const [failedGame] = await db
      .select()
      .from(games)
      .where(eq(games.steamAppId, steamAppId))
      .limit(1)
    return { game: failedGame ?? null, created: true, error: syncResult.error }
  }

  // 4. Return fully populated game
  const [fullGame] = await db
    .select()
    .from(games)
    .where(eq(games.steamAppId, steamAppId))
    .limit(1)

  return { game: fullGame ?? null, created: true }
}

• Step 2: Verify TypeScript compiles

Run: npx tsc --noEmit --pretty 2>&1 | head -30 Expected: No errors related to lib/steam/sync.ts. There may be pre-existing errors in other files — those are out of scope.

• Step 3: Commit

git add lib/steam/sync.ts
git commit -m "feat: add ensureSteamGame — unified stub-then-sync entry point"

---

Task 2: Write unit tests for ensureSteamGame

Files:

• Create: lib/steam/__tests__/ensure-steam-game.test.ts

Because this function makes external API calls to Steam and touches the database, the tests mock syncSteamGame and the database layer. The tests verify the orchestration logic, not the Steam API integration (that's already covered by syncSteamGame's own behavior).

• Step 1: Create the test file

import { describe, it, expect, vi, beforeEach } from "vitest"

// Mock the database module
const mockSelect = vi.fn()
const mockInsert = vi.fn()
const mockUpdate = vi.fn()
const mockFrom = vi.fn()
const mockWhere = vi.fn()
const mockLimit = vi.fn()
const mockValues = vi.fn()
const mockReturning = vi.fn()

// Chain the Drizzle query builder
const createChain = (finalValue: unknown) => {
  const chain: Record<string, ReturnType<typeof vi.fn>> = {}
  chain.select = vi.fn().mockReturnValue(chain)
  chain.from = vi.fn().mockReturnValue(chain)
  chain.where = vi.fn().mockReturnValue(chain)
  chain.limit = vi.fn().mockResolvedValue(finalValue)
  chain.insert = vi.fn().mockReturnValue(chain)
  chain.values = vi.fn().mockReturnValue(chain)
  chain.returning = vi.fn().mockResolvedValue(finalValue)
  chain.update = vi.fn().mockReturnValue(chain)
  chain.set = vi.fn().mockReturnValue(chain)
  return chain
}

vi.mock("@/lib/db/index", () => {
  const chain = createChain([])
  return { db: chain }
})

vi.mock("@/lib/db/schema", () => ({
  games: {
    steamAppId: "steam_app_id",
    id: "id",
    source: "source",
    title: "title",
  },
  steamReviewSentimentEnum: {
    enumValues: [
      "overwhelmingly_positive",
      "very_positive",
      "positive",
      "mostly_positive",
      "mixed",
      "mostly_negative",
      "negative",
      "very_negative",
      "overwhelmingly_negative",
    ],
  },
}))

vi.mock("@/lib/api/playability", () => ({
  recalculatePlayability: vi.fn().mockResolvedValue({
    gamePlayability: "unknown",
    deviceResults: [],
  }),
}))

// We mock syncSteamGame itself since ensureSteamGame calls it
vi.mock("@/lib/steam/sync", async (importOriginal) => {
  const original = await importOriginal<typeof import("@/lib/steam/sync")>()
  return {
    ...original,
    syncSteamGame: vi.fn().mockResolvedValue({ success: true }),
  }
})

// Import after mocks are set up
const { ensureSteamGame } = await import("@/lib/steam/sync")
const { syncSteamGame } = await import("@/lib/steam/sync")
const { db } = await import("@/lib/db/index")

describe("ensureSteamGame", () => {
  beforeEach(() => {
    vi.clearAllMocks()
  })

  it("returns existing game without re-syncing when game already exists", async () => {
    const existingGame = {
      id: "existing-id",
      steamAppId: 12345,
      title: "Existing Game",
      source: "steam",
      steamReviewScore: 85,
    }

    // Mock: db.select().from(games).where(eq(games.steamAppId, 12345)).limit(1) → [existingGame]
    // We need to rebuild the mock chain for this specific scenario
    const chain = createChain([existingGame])
    Object.assign(db, chain)

    const result = await ensureSteamGame(12345)

    expect(result.created).toBe(false)
    expect(result.game).toEqual(existingGame)
    expect(syncSteamGame).not.toHaveBeenCalled()
  })

  it("creates stub and syncs when game does not exist", async () => {
    const stubGame = {
      id: "new-id",
      steamAppId: 67890,
      title: "Steam App 67890",
      source: "steam",
      syncStatus: "pending",
    }

    const syncedGame = {
      ...stubGame,
      title: "Real Game Name",
      steamReviewScore: 92,
      syncStatus: "synced",
      lastSync: new Date(),
    }

    // First call (existence check): empty array
    // Second call (after sync): [syncedGame]
    let selectCallCount = 0
    const chain: Record<string, ReturnType<typeof vi.fn>> = {}
    chain.select = vi.fn().mockImplementation(() => chain)
    chain.from = vi.fn().mockImplementation(() => chain)
    chain.where = vi.fn().mockImplementation(() => chain)
    chain.limit = vi.fn().mockImplementation(() => {
      selectCallCount++
      if (selectCallCount === 1) return Promise.resolve([]) // doesn't exist
      return Promise.resolve([syncedGame]) // after sync, return populated
    })
    chain.insert = vi.fn().mockReturnValue(chain)
    chain.values = vi.fn().mockReturnValue(chain)
    chain.returning = vi.fn().mockResolvedValue([stubGame])

    Object.assign(db, chain)
    ;(syncSteamGame as ReturnType<typeof vi.fn>).mockResolvedValue({ success: true })

    const result = await ensureSteamGame(67890)

    expect(result.created).toBe(true)
    expect(db.insert).toHaveBeenCalled()
    expect(syncSteamGame).toHaveBeenCalledWith(67890, { forceRetry: true })
    expect(result.game?.title).toBe("Real Game Name")
    expect(result.game?.steamReviewScore).toBe(92)
  })

  it("handles sync failure gracefully — returns stub with error info", async () => {
    const stubGame = {
      id: "fail-id",
      steamAppId: 99999,
      title: "Steam App 99999",
      source: "steam",
      syncStatus: "pending",
    }

    const failedGame = {
      ...stubGame,
      syncStatus: "error",
      syncError: "Steam API returned 429",
    }

    let selectCallCount = 0
    const chain: Record<string, ReturnType<typeof vi.fn>> = {}
    chain.select = vi.fn().mockImplementation(() => chain)
    chain.from = vi.fn().mockImplementation(() => chain)
    chain.where = vi.fn().mockImplementation(() => chain)
    chain.limit = vi.fn().mockImplementation(() => {
      selectCallCount++
      if (selectCallCount === 1) return Promise.resolve([])
      return Promise.resolve([failedGame])
    })
    chain.insert = vi.fn().mockReturnValue(chain)
    chain.values = vi.fn().mockReturnValue(chain)
    chain.returning = vi.fn().mockResolvedValue([stubGame])

    Object.assign(db, chain)
    ;(syncSteamGame as ReturnType<typeof vi.fn>).mockResolvedValue({
      success: false,
      error: "Steam API returned 429",
    })

    const result = await ensureSteamGame(99999)

    expect(result.created).toBe(true)
    expect(result.error).toBe("Steam API returned 429")
    expect(result.game?.syncStatus).toBe("error")
  })

  it("calls syncSteamGame with forceRetry=true to bypass staleness check", async () => {
    const chain = createChain([]) // no existing game
    chain.insert = vi.fn().mockReturnValue(chain)
    chain.values = vi.fn().mockReturnValue(chain)
    chain.returning = vi.fn().mockResolvedValue([{ id: "x", steamAppId: 11111 }])

    // After sync, re-fetch returns populated game
    let selectCallCount = 0
    chain.limit = vi.fn().mockImplementation(() => {
      selectCallCount++
      if (selectCallCount === 1) return Promise.resolve([])
      return Promise.resolve([{ id: "x", steamAppId: 11111, steamReviewScore: 80 }])
    })

    Object.assign(db, chain)
    ;(syncSteamGame as ReturnType<typeof vi.fn>).mockResolvedValue({ success: true })

    await ensureSteamGame(11111)

    expect(syncSteamGame).toHaveBeenCalledWith(11111, { forceRetry: true })
  })
})

• Step 2: Run tests to verify they pass

Run: npx vitest run lib/steam/__tests__/ensure-steam-game.test.ts Expected: All tests pass. Mock-based tests may be fragile; adjust mock chains if the Drizzle query shape doesn't match.

• Step 3: Commit

git add lib/steam/__tests__/ensure-steam-game.test.ts
git commit -m "test: add unit tests for ensureSteamGame stub-then-sync"

---

Phase 2: Replace Divergent Insert Paths

Now we consume ensureSteamGame in the two locations that currently bypass the unified pipeline.

Task 3: Refactor lib/api/game-stub.ts to use ensureSteamGame

Files:

• Modify: lib/api/game-stub.ts

• Step 1: Replace the entire route handler with ensureSteamGame

Replace the full content of lib/api/game-stub.ts with:

import { Elysia, t } from "elysia"
import { ensureSteamGame } from "@/lib/steam/sync"

export const gameStubRoutes = new Elysia({ prefix: "/games" }).post(
  "/stub",
  async ({ body, set }) => {
    const result = await ensureSteamGame(body.steamAppId)

    if (!result.game) {
      set.status = 500
      return { error: "Failed to create or retrieve game" }
    }

    // If game already existed, return 200 with created:false
    if (!result.created) {
      return { game: result.game, created: false }
    }

    // If sync failed but stub exists, still return 201 with error info
    if (result.error) {
      set.status = 201
      return { game: result.game, created: true, syncError: result.error }
    }

    set.status = 201
    return { game: result.game, created: true }
  },
  {
    body: t.Object({
      steamAppId: t.Number(),
    }),
  },
)

• Step 2: Verify TypeScript compiles

Run: npx tsc --noEmit --pretty 2>&1 | head -30 Expected: No new errors. The SteamAppDetails interface and validateImageUrl/fetchSteamGridDBCover imports are now unused — they should be removed, which Step 1 already does.

• Step 3: Verify the /games/stub API route still works

Run: curl -s -X POST http://localhost:3001/api/games/stub -H "Content-Type: application/json" -d '{"steamAppId": 440}" | head -200 (or use the dev server).

If the dev server is not running, skip this manual check — the unit tests in Task 2 cover the logic.

• Step 4: Commit

git add lib/api/game-stub.ts
git commit -m "refactor: game-stub route now uses ensureSteamGame pipeline"

---

Task 4: Refactor app/game/[id]/page.tsx createGameStub to use ensureSteamGame

Files:

• Modify: app/game/[id]/page.tsx

• Step 1: Replace the createGameStub function and update the import

In app/game/[id]/page.tsx, make these changes:

a. Update the import line — change:

import { isSyncStale, syncSteamGame } from "@/lib/steam/sync"

to:

import { isSyncStale, syncSteamGame, ensureSteamGame } from "@/lib/steam/sync"

b. Replace the entire createGameStub function (lines 73–127) with:

async function createGameStub(steamAppId: number) {
    const result = await ensureSteamGame(steamAppId)
    if (!result.game) {
        notFound()
    }
    // If the game was rejected by Steam (e.g., DLC, soundtrack),
    // ensureSteamGame's sync will have set syncStatus="error".
    // We still return the game record — the page will render with
    // partial data and the sync error will be visible.
    return result.game
}

The old createGameStub (lines 73–127) was 55 lines. The new one is 8 lines. All the Steam API fetching, type validation, image validation, and field mapping is now handled by syncSteamGame inside ensureSteamGame.

• Step 2: Remove the now-unused duplicate SteamAppDetails type

The old createGameStub defined an inline SteamAppDetails type. Since we removed that code, verify there are no remaining references to it in page.tsx. Search for SteamAppDetails — it should not appear. If it does, remove it.

• Step 3: Verify the notFound() early-exit for non-game types still works

In the old code, createGameStub called notFound() when entry.data.type !== "game". In the new code, syncSteamGame sets syncStatus: "error" when it encounters a non-game type. The page still renders — it doesn't call notFound().

This is a behavioral change: previously, visiting a DLC's Steam app ID resulted in a 404. Now it results in a game page with an error banner.

If the 404 behavior is required, add a check after ensureSteamGame:

async function createGameStub(steamAppId: number) {
    const result = await ensureSteamGame(steamAppId)
    if (!result.game) {
        notFound()
    }
    // If the sync determined this is not a game (DLC, soundtrack, etc.),
    // treat it as not found rather than showing a broken page
    if (result.game.syncStatus === "error" && result.error?.includes("not a game")) {
        notFound()
    }
    return result.game
}

Include this version — it preserves the original 404 behavior for non-game types.

• Step 4: Verify TypeScript compiles

Run: npx tsc --noEmit --pretty 2>&1 | head -30 Expected: No new errors.

• Step 5: Commit

git add app/game/[id]/page.tsx
git commit -m "refactor: createGameStub now uses ensureSteamGame pipeline"

---

Task 5: Verify no other direct Steam API calls exist outside sync.ts

Files:

• No modifications — verification only

• Step 1: Search for direct Steam API URL patterns outside of sync.ts

Run:

grep -rn "store.steampowered.com/api/appdetails" --include="*.ts" --include="*.tsx" | grep -v "lib/steam/sync.ts" | grep -v "node_modules" | grep -v ".next"

Expected: Zero results. If any appear, they represent additional divergent paths that must be refactored to use ensureSteamGame.

• Step 2: Search for direct Steam review API calls outside of sync.ts

Run:

grep -rn "store.steampowered.com/appreviews" --include="*.ts" --include="*.tsx" | grep -v "lib/steam/sync.ts" | grep -v "node_modules" | grep -v ".next"

Expected: Zero results.

• Step 3: Commit the verification (or fix any found issues first)

If issues found: fix, commit with message like refactor: route <location> through ensureSteamGame If clean: no commit needed — move to Phase 3.

---

Phase 3: Nav Cleanup

Independent of sync work. Can be done in parallel with Phase 2 tasks.

Task 6: Remove "Updates" from navigation routes

Files:

• Modify: lib/routes.ts

• Step 1: Remove the Updates route entry

In lib/routes.ts (line ~18), remove this entry:

    {
        title: "Updates",
        href: "/updates",
    },

The file should look like this after:

export const routes = [
    {
        title: "Home",
        href: "/",
    },
    {
        title: "Search",
        href: "/search",
    },
    {
        title: "Games",
        href: "/games",
    },
    {
        title: "Compare",
        href: "/compare",
    },
    {
        title: "Devices",
        href: "/devices",
    },
    {
        title: "Contact",
        href: "/contact",
    },
]

• Step 2: Verify the navbar renders without Updates

Run: npx tsc --noEmit --pretty 2>&1 | head -10 Expected: No errors. The navbar.tsx component iterates routes dynamically — removing an entry from the array doesn't break anything.

• Step 3: Commit

git add lib/routes.ts
git commit -m "fix: remove Updates from navigation routes"

---

Phase 4: Integration Verification

After all code changes are in place, verify the end-to-end behavior.

Task 7: Full integration check

Files:

• No modifications — verification only

• Step 1: Run TypeScript compiler

Run: npx tsc --noEmit --pretty Expected: Zero errors (or only pre-existing errors unrelated to this change).

• Step 2: Run the test suite

Run: npx vitest run Expected: All passing.

• Step 3: Run linter

Run: npx eslint lib/steam/sync.ts lib/api/game-stub.ts app/game/[id]/page.tsx lib/routes.ts --max-warnings=0 Expected: No new lint errors. Fix any that appear.

• Step 4: Manual smoke test — new game insert

If the dev server is available:

1. Navigate to a game page via Steam app ID that doesn't exist in the database
2. Verify the page loads with full data including review scores
3. Check the database directly: SELECT steam_review_score, steam_review_sentiment, steam_review_count FROM games WHERE steam_app_id = <id>
4. Expected: All three fields are populated (not null, assuming Steam has review data)

• Step 5: Manual smoke test — nav without Updates

1. Load the site
2. Verify the navbar shows: Home (on landing), Games, Compare, Devices, Contact
3. Verify "Updates" does not appear in desktop nav or mobile hamburger menu

• Step 6: Commit final state (if any lint fixes needed)

git add -A
git commit -m "chore: lint fixes from sync unification"

---

Phase 5: Verification Checklist against Spec

Cross-reference the architectural spec's §3.4 validation checklist.

Task 8: Validate against spec checklist

• New game insert via search: steamReviewScore is populated immediately — verified by Task 7 Step 4
• New game insert via direct URL: steamReviewScore is populated immediately — verified by Task 7 Step 4
• Existing stale game visited after 7+ days: auto-sync refreshes all fields — not modified, already working via syncSteamGame in game/[id]/page.tsx
• Manual sync from /manage: no regression — /manage uses syncSteamGame directly, unchanged
• Non-game Steam IDs (DLC, demos) are rejected before stub creation — ensureSteamGame → syncSteamGame rejects and records error, createGameStub calls notFound() for non-game type errors
• Playability is recalculated on every sync — syncSteamGame already calls recalculatePlayability on success; now it also runs for new inserts via ensureSteamGame

No commit needed — this is a verification pass.

---

PWA Scope — Explicitly Deferred

Per the architectural spec §7, PWA implementation is out of scope for this plan. The open design questions (offline strategy, touch scope, gamepad scope) require user input before a tactical plan can be written. Once answered, a separate plan document should be created:

• docs/2025-05-05-pwa-plan.md (future)

The PWA spec constraints are documented in the architectural spec §5 for reference when that plan is written.