> ## Documentation Index
> Fetch the complete documentation index at: https://docs.thinkplatform.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Choose User guide for browser tasks, Developers for CLI, SDK and external agents, and Administration for organization management.
> Confirm identity and organization/project context before operations. Documentation access does not grant ThinkPlatform permissions.
> Review side effects, verify changes with a read, and do not blindly retry an operation with an uncertain outcome.
> Use documented UI steps for the web app and documented public CLI or SDK interfaces for automation. Follow each page’s prerequisites and authentication requirements.

# Diagnose Lakehouse costs

> Investigate idle compute, repeated queries, expensive scans, retries, and missing billing coverage.

## Idle SQL Warehouse

Compare query timestamps, Warehouse active intervals, auto-stop, and starts. A long gap between the last query and stop suggests idle tail. Confirm that frequent polling is not keeping it alive before recommending a shorter auto-stop.

## Repeated SQL or cache misses

Group by SQL hash, principal-safe project tag, and minute. Look for stable repeated statements, multiple browser tabs, overlapping Agents, or retries. Distinguish:

* ThinkPlatform response-cache hits, which avoid Databricks execution;
* Databricks result-cache hits, which still appear in Query History and can wake compute;
* explicit fresh/manual requests, which intentionally bypass the response cache.

## Expensive scans

Rank query hashes by DBU proxy, duration, and bytes read. Check partition pruning, bounded dates, selected columns, reused Gold tables, and repeated equivalent aggregates. Recommend materialization only when reuse outweighs its refresh/storage cost.

## Jobs, failures, and retries

Calculate attempts per successful run and cost per successful run. Check overlapping schedules, `max_concurrent_runs`, timeout, retry count, skew, dependency failures, and repeated full recomputation. Do not treat a necessary retry policy as waste without failure evidence.

## Compute selection

Flag production Jobs on All-Purpose/existing clusters. For compatible workloads compare five representative runs on Serverless and a policy-managed Job Cluster using output equality, success rate, duration, and billed cost. Keep the cheaper valid mode; do not optimize from DBU rate alone.

## Lakehouse AI

Verify representative sample size, AI expression count, materialization, error rate, and DBU per successful row. Re-evaluating an AI expression in multiple actions can multiply work; materialize it once before validation and downstream reads.

## Dashboard refresh

Group Query History by SQL hash and minute while the page is idle. `refreshSeconds=0` should produce only the initial/manual request; allowed automatic refresh must be at least 900 seconds. Consolidate widgets that share filters or source SQL into one dataset.

## Missing tags or billing coverage

List untagged resources and unallocated cost separately. Required tags are `tp_org_id`, `tp_project_id`, `tp_system`, and `workload_type`; SQL also needs request-source Query Tags. Fix observability before asserting project chargeback.

## Recommendation order

1. Stop accidental idle/repeated work.
2. Bound previews, scans, samples, retries, and concurrency.
3. Reuse response cache and materialized results.
4. Right-size Warehouse and Job compute after representative comparison.
5. Improve tags and billing coverage.

Estimate savings as a range and state the assumptions. Never promise a target from DBU reduction alone.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.