Unify operational and Lakehouse data with direct Iceberg access in AlloyDB for PostgreSQL

To make good business decisions, you need two kinds of data: the live state of the business from operational databases, and the historical context stored in an analytical system such as a lakehouse or a data warehouse. Traditionally, those systems have been separate, but they are now converging rapidly. That’s why today, we're announcing direct Iceberg access in AlloyDB for PostgreSQL, in preview. Now, developers can query Apache Iceberg tables stored in a cloud storage bucket directly from AlloyDB and join them with live operational data, all in a single SQL query. In addition, we’re announcing the general availability of AlloyDB sync and Lakehouse federation. Together, they give AlloyDB three ways to work with the borderless Lakehouse, which keeps one open copy of your data available to any engine. Etsy, for example, unified three petabytes with the borderless Lakehouse, joining data in place to accelerate pipeline queries by 60%. With AlloyDB, which approach to use depends on where the data lives and how your application reads it: Direct Iceberg access: Read Iceberg tables in place, accelerated by the AlloyDB columnar engine. AlloyDB sync: Serve lakehouse data from AlloyDB for your most latency-sensitive workloads, refreshed on a schedule you set. Lakehouse federation: Query native BigQuery tables in place, with BigQuery doing the processing. With this launch, apps and  agents can act on operational and lakehouse data together without building or managing custom pipelines. AlloyDB and the borderless Lakehouse: Query Apache Iceberg and BigQuery data in place or sync it to AlloyDB, giving real-time applications and AI agents governed access to operational and lakehouse data together. Decisions at the moment of action With these integrations between AlloyDB and the borderless Lakehouse, historical context sits next to live transactions, so decisions can happen at the moment of action, without building or managing custom pipelines. For example: Fraud and risk: Before approving a payment, a fraud agent checks it against five years of customer transaction history in Iceberg, in addition to the last 30 days in the database. Personalization: A shopping agent joins the live cart with lifetime value and return history in the lakehouse to choose the right offer while the shopper is still on the page. Dynamic pricing: A pricing agent combines live inventory in AlloyDB with years of demand and weather history in the lakehouse to adjust prices as conditions change. Data products: A data team publishes curated tables once to Iceberg, and then dozens of applications read them in place instead of each maintaining its own copy. In each case, the history stays in the lakehouse and the decision happens in the database. Agents act on your behalf using only the tables and permissions you grant them in AlloyDB. Let’s take a deeper look at each of the three integrations. Direct Iceberg access in AlloyDB (Preview) With direct Iceberg access, AlloyDB reads Apache Iceberg tables directly from Cloud Storage, using either a bucket path or the Lakehouse runtime catalog. You register an Iceberg table once and query it like any other PostgreSQL table, including joins with live operational tables. code_block <ListValue: [StructValue([('code', "-- Register five years of transaction history stored as Iceberg\r\nCREATE TABLE transaction_history (\r\n customer_id BIGINT, category TEXT, amount NUMERIC, ts TIMESTAMPTZ)\r\nUSING iceberg\r\nWITH (uri = 'gs://my-bucket/warehouse/payments/transaction_history');\r\n\r\n\r\n-- Compare the payment in flight with the customer's history\r\nSELECT t.amount,\r\n AVG(h.amount) AS usual_spend,\r\n MAX(h.amount) AS largest_past_purchase\r\nFROM recent_transactions t -- live, in AlloyDB\r\nJOIN transaction_history h -- five years, in Iceberg\r\n ON h.customer_id = t.customer_id AND h.category = t.category\r\nWHERE t.transaction_id = 'TX-1001'\r\nGROUP BY t.amount;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f1080db0690>)])]> AlloyDB reads Iceberg tables in place and joins them with live operational data, using the same copy as the rest of your lakehouse. Iceberg support is built into AlloyDB and is accelerated by the AlloyDB columnar engine, so queries over lakehouse data are fast. They run on the same database instance as your application, without a separate reader instance to manage, and they read the latest data your team has published to the lakehouse. In our testing on a 1 TB dataset in Cloud Storage, AlloyDB direct Iceberg access is up to 7x faster on aggregation queries than Amazon's comparable PostgreSQL-compatible service, and up to 3.5x faster on aggregation queries than the open-source PostgreSQL extension. AlloyDB sync for the lowest-latency reads (GA) Not every table needs to be read in place. A retail merchant risk score, for example, is checked before every payment but

Extras afișat conform termenilor RSS ai publicației · 125 cuvinte · textul integral și figurile sunt pe site-ul publicației