Ideal Tech price snapshots#

Status: Working reference
Last verified: 2026-09-28
Canonical for: capturing and querying append-only Ideal Tech PC-builder prices

Data contract#

Ideal Tech's PC builder is a Next.js application. Its product catalog is present in the React Server Components (Flight) response, so the capture path reads that structured payload rather than parsing rendered HTML labels.

Every capture is written to a new directory under idealtech-price-snapshots/:

idealtech-price-snapshots/<UTC timestamp>/
  metadata.json
  products.json
  response.flight.txt  # or response.html when the HTML fallback was used

The timestamp directory is append-only. The extractor creates it with overwrite protection and fails if the same directory already exists. Never edit a prior response, products.json, or metadata file to represent a new price. Capture a new timestamp instead.

products.json is a flat JSON array suitable for DuckDB. Capture time, source URL, transport, and the site's own last-updated label are repeated on every row. The following retailer fields are then copied without normalization:

category_id
category_name
product_id
product_name
ori_price
dis_price
is_label
is_discounted

In particular, do not trim, rewrite, merge, or canonicalize product_name or category_name in this layer. GPU/spec identity and BOM-family mappings belong in separate attributed join data, so a mapping change cannot rewrite the market evidence.

The source array also contains grouping rows with is_label = true. They are retained for source fidelity. Use WHERE NOT is_label for purchasable products. Metadata reports source rows, products, and label rows separately.

Capture the live list#

From this directory:

bun run prices:idealtech

This is the canonical catalogue capture and does not include saved-quote state. The root Flight endpoint was fetched successfully without a browser cookie on 2026-09-28.

To reproduce the browser's Edit Spec navigation from /quote/<id>, pass the saved-quote ID:

bun run prices:idealtech -- --edit-id '<current quote UUID>'

That form targets /?edit=<id>&_rsc=<token>, while Next-Url, Referer, and the URL-encoded router-state tree describe the source /quote/<id> route exactly as the browser does. It is useful for diagnosing the request path, but the root capture above is preferred for catalogue history because it contains no quote configuration.

If the Flight request is not accepted, the extractor downloads /?edit=<id> as HTML and reconstructs the embedded self.__next_f.push(...) Flight chunks. Both paths validate the expected product fields and require at least 2,000 non-label products before writing anything.

When Cloudflare requires the browser's clearance cookie, supply it only for the process invocation. It is not logged or written to snapshot metadata. A clearance cookie can be tied to the browser user agent, so pass both through the environment when needed:

IDEALTECH_COOKIE='<current Cookie request header>' \
IDEALTECH_USER_AGENT='<the same browser User-Agent>' \
bun run prices:idealtech -- --edit-id '<current quote UUID>'

Never commit either value. Treat a copied clearance cookie as a short-lived secret.

The deterministic request/parser tests cover the observed Edit Spec request, direct Flight extraction, and the HTML embedded-Flight fallback:

bun run test:prices:idealtech

Live verification on 2026-09-28#

The canonical command above wrote idealtech-price-snapshots/2026-09-28T065601Z/ from a real text/x-component response. It retained 3,376 source rows: 2,906 purchasable products and 470 grouping rows across 29 categories. The raw response SHA-256 is b00a33045d0bec39257bd92a6120edbafc05dc8c16e59cef4b445368ea1a2888.

The response says Price list last updated: 26 Sept 2026, 1:46 pm. The September 22 capture remains immutable history. An exact-name comparison found 2,895 purchasable products present in both captures, 11 new exact names, seven missing exact names, and three changed prices. This describes only the retailer payload; it does not supersede separately sourced founder observations or prove that a listed item is in stock.

To test a saved response or import an older immutable observation:

bun run prices:idealtech -- \
  --input ideatech-prices.json \
  --captured-at 2026-09-22T14:03:13.591Z

The misspelled ideatech-prices.json filename is the original saved response and is intentionally not renamed or overwritten. idealtech-price-list-from-partial-2026-09-22.json is the earlier manually flattened/nested derivative and is also immutable.

The imported September 22 source has:

Measure	Count
Source rows retained	3,372
Purchasable products	2,902
Grouping/label rows	470
Categories	29

Query with DuckDB#

DuckDB can scan the snapshots directly:

SELECT
    snapshot_captured_at,
    category_name,
    product_id,
    product_name,
    try_cast(ori_price AS DECIMAL(12, 2)) AS original_price_myr,
    try_cast(dis_price AS DECIMAL(12, 2)) AS current_price_myr,
    is_discounted
FROM read_json_auto(
    'idealtech-price-snapshots/*/products.json',
    format = 'array',
    union_by_name = true,
    filename = true,
    maximum_object_size = 10000000
)
WHERE NOT is_label
ORDER BY snapshot_captured_at DESC, category_id, product_id;

The benchmark service materializes the same rows at startup and on its existing refresh endpoint as idealtech_price_observations. That table preserves the source price columns as source_original_price/source_discounted_price and adds typed original_price_myr and current_price_myr columns for analysis. Its health counters are exposed under idealtech_prices in /healthz.

The source contains 38 literal $NaN values in ori_price. They are preserved in the JSON/source column; try_cast intentionally produces NULL only in the typed original_price_myr derivative. dis_price, the current selectable price, is numeric in the September 22 snapshot.

Joining specifications, prices, and performance#

Do not join on a rewritten extraction-layer product name. Create an explicit mapping table with, at minimum, the snapshot source, category/product IDs, exact product name, canonical GPU key, mapping method, author, and mapping timestamp. The eventual analytical relation is:

idealtech_price_observations
  -> attributed exact-SKU-to-GPU mapping
  -> gpu_device_specs
  -> normalized_runs / workload aggregates

Keep exact card prices and complete-node BOM prices separate. The former compares silicon offers; the latter is the purchase Pareto input because common CPU, RAM, storage, board, PSU, case, and assembly costs change the frontier.