---
title: "Time Travel: Snapshots, Timestamps and Rollbacks"
author: "Youzhi Yu<br><span style='font-size:85%;'>University of Chicago</span>"
bibliography: vignette_reference.bib
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Time Travel: Snapshots, Timestamps and Rollbacks}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
library(icebergr)
```

Every Iceberg commit leaves the previous state of the table intact and readable
[@icebergspec]. Nothing is overwritten, so a snapshot from before a bad load is
still there, and reading it needs no backup, no restore and no cooperation from
whoever wrote it.

```{r}
tbl <- icebergr_example_table(rows = 200)
```

## Snapshot history

```{r}
history <- icebergr_snapshots(tbl)
history[, c(
  "snapshot_id", "parent_snapshot_id", "operation", "added_records",
  "total_records"
)]
```

The example table was built with two appends, so there are two snapshots and the
second's parent is the first. `total_records` is cumulative; `added_records` is
what that commit contributed.

**Snapshot ids are `character`, not numeric.** Iceberg assigns them as random
64-bit integers and an R double carries only 53 bits of mantissa, so a numeric
round trip would silently select a *different, existing* snapshot rather than
fail. This is the one type decision in the package that exists purely to prevent
a wrong answer:

```{r}
class(history$snapshot_id)
```

## Reading a snapshot by id

```{r}
first <- history$snapshot_id[[1]]

nrow(icebergr_collect(icebergr_scan(tbl, snapshot_id = first)))
nrow(icebergr_collect(icebergr_scan(tbl)))
```

The first snapshot holds one append; the current table holds both. Pushdown works
against a historical snapshot exactly as it does against the current one, because
the manifests of that snapshot are still on disk with their statistics intact:

```{r}
nrow(icebergr_scan_plan(icebergr_scan(tbl, snapshot_id = first)))
nrow(icebergr_scan_plan(icebergr_scan(tbl)))
```

## Reading as of a time

`as_of` takes a time and resolves it to a snapshot:

```{r}
icebergr_collect(
  icebergr_scan(tbl, as_of = history$timestamp[[1]], select = "id", limit = 3)
)
```

Two things about that argument are worth being explicit about, because both are
easy to get wrong.

**It is a commit time, not a time in your data.** `as_of` asks "what did this
table look like then", not "which rows are from then". A `day` or `recorded_at`
column is data; filter on it with `filter`.

**It resolves against the snapshot log, not the snapshot list.** Iceberg keeps
both: the list of every snapshot that still exists, and the *log* of which
snapshot was current at which time. A rollback adds a log entry pointing back at
the earlier snapshot, while the snapshot it abandoned stays in the list with its
own, later, timestamp. Resolving against the list would find that abandoned
snapshot and read a state the table was rolled back *out of*; `icebergr`
resolves against the log, so it does not.

The same applies to a snapshot that only ever existed on another branch: present
in the list, never in this branch's log, and so never selected by `as_of`. It
is still reachable by id, which is the right split: an explicit id is a request
for a specific state, while a time is a request for the state this table was in.

## The schema travels too

Iceberg records a schema per snapshot, so a column that has since been renamed or
dropped is still nameable as of the snapshot that had it:

```{r}
identical(
  icebergr_schema(tbl, snapshot_id = first)$name,
  icebergr_schema(tbl)$name
)
```

Nothing was renamed here, so the two agree. Where they would not, `filter` and
`select` are resolved against the schema of the snapshot actually being read,
not the current one. That is what makes a historical read reproducible: the names
that worked then still work, and you do not have to know what happened to the
schema in between.

A read of the current state is a read of the current snapshot, and follows its
schema in the same way. That is the table's current schema too, except straight
after another engine changes the schema without writing anything, as an
`ALTER TABLE ... ADD COLUMN` does: until the next commit, a read still has the
columns the current snapshot was written with, while `icebergr_schema()` already
shows the new ones.

## Isolation, and why a handle looks stale

A table handle is bound to a snapshot. Another session's commit does not change
what your handle reads:

```{r}
current <- icebergr_snapshots(tbl)
tbl2 <- icebergr_reload(tbl)
identical(nrow(icebergr_snapshots(tbl2)), nrow(current))
```

`icebergr_reload()` is how you opt in to seeing new commits. Until you call it,
every scan off that handle reads one consistent state: snapshot isolation in the
sense of @berenson1995critique, obtained from immutability rather than from
locking. All three lakehouse formats get it the same way [@jain2023lakehouse].

The practical consequence: two scans off the same handle are guaranteed to agree,
which is what you want when a report runs several queries and has to add up.

## What is not here

Snapshot expiry, which deletes old snapshots and the files only they reference,
is a maintenance operation this version does not expose, and neither is rollback
itself. `iceberg-rust` 0.10 has a transaction action for expiry and none for
rollback, so the first is a gap in this package and the second is one upstream:

```{r}
features <- icebergr_spec_support()$features
features[
  grepl("snapshot|maintenance", features$feature, ignore.case = TRUE),
  c("feature", "supported", "reason")
]
```

Reading a table that another engine has rolled back or expired works normally:
time travel is a read, and reading by snapshot id or by time is all here.

# References