Totem Export Format v1

This page documents the Totem export ZIP format, field by field. It is version-pinned: v1 is frozen. Any breaking changes will ship as v2 at a separate URL.

ZIP layout

A Totem export is a standard ZIP archive with this structure:

totem-export-YYYY-MM-DD.zip
├── manifest.json
├── readme.md
├── bookmarks.csv
├── bookmarks/
│   ├── 01-bookmark-title.md
│   └── 02-another-bookmark-title.md
└── data/
    ├── bookmarks-2024.jsonl
    ├── bookmarks-2025.jsonl
    ├── details.jsonl
    ├── highlights.jsonl
    └── reading-progress.jsonl

File	Role
`manifest.json`	Export metadata, version info, SHA-256 checksums
`readme.md`	Index of every exported bookmark and plain-English guide
`bookmarks.csv`	Spreadsheet-friendly view (RFC 4180, UTF-8 + BOM)
`bookmarks/*.md`	Human-readable Markdown, one file per bookmark
`data/*.jsonl`	Canonical data — the only source the importer reads

CSV and Markdown are derived views of the JSONL data. They exist for convenience and are ignored on re-import.

manifest.json

{
  "totem": {
    "export_version": 1,
    "schema_version": 8
  },
  "generated_at": "2026-05-12T14:23:00.000Z",
  "generated_by": {
    "app": "totem",
    "version": "1.2.2",
    "platform": "chrome-extension"
  },
  "account": {
    "id_hash": "sha256:9c8d…",
    "handle_redacted": "@y***ndle"
  },
  "kind": "library",
  "counts": {
    "bookmarks": 3000,
    "details": 2816,
    "highlights": 47,
    "reading_progress": 312
  },
  "shards": {
    "bookmarks": [
      "data/bookmarks-2024.jsonl",
      "data/bookmarks-2025.jsonl"
    ],
    "details": ["data/details.jsonl"],
    "highlights": ["data/highlights.jsonl"],
    "reading_progress": ["data/reading-progress.jsonl"]
  },
  "derived": {
    "csv": "bookmarks.csv",
    "markdown_index": "readme.md",
    "markdown_files": [
      "bookmarks/01-first-bookmark.md",
      "bookmarks/02-second-bookmark.md"
    ]
  },
  "checksums": {
    "data/bookmarks-2024.jsonl": "sha256:ab12…",
    "data/bookmarks-2025.jsonl": "sha256:cd34…",
    "data/details.jsonl": "sha256:ef56…",
    "data/highlights.jsonl": "sha256:78ab…",
    "data/reading-progress.jsonl": "sha256:cd90…",
    "bookmarks.csv": "sha256:11aa…",
    "readme.md": "sha256:22bb…",
    "bookmarks/01-first-bookmark.md": "sha256:33cc…",
    "bookmarks/02-second-bookmark.md": "sha256:44dd…"
  }
}

Field reference

Field	Type	Description
`totem.export_version`	`number`	Format version. Always `1` for this spec.
`totem.schema_version`	`number`	IDB schema version at export time. Currently `8`. Import refuses if this exceeds the importer's version.
`generated_at`	`string`	ISO 8601 UTC timestamp of export.
`generated_by.app`	`"totem"`	Always `"totem"`.
`generated_by.version`	`string`	Totem app version (semver).
`generated_by.platform`	`"chrome-extension"`	Runtime platform.
`account.id_hash`	`string`	SHA-256 of the X user ID, prefixed `sha256:`. Used for account-match check on import.
`account.handle_redacted`	`string`	Redacted handle: first char + `*` + last 4 chars. E.g. `@y*ndle`.
`kind`	`"library"`	Export scope. Always `"library"` in v1.
`counts.*`	`number`	Row counts per store: `bookmarks`, `details`, `highlights`, `reading_progress`.
`shards.*`	`string[]`	Paths to JSONL files for each store. Bookmarks are year-sharded (`data/bookmarks-YYYY.jsonl`).
`derived.csv`	`string`	Path to the CSV file. Always `"bookmarks.csv"`.
`derived.markdown_index`	`string`	Path to the Markdown index. Always `"readme.md"`.
`derived.markdown_files`	`string[]`	Paths to per-bookmark Markdown files under `bookmarks/`.
`checksums.*`	`string`	SHA-256 hex digest prefixed `sha256:` for each data and derived file. Verified on import.

JSONL stores

Each .jsonl file contains one JSON object per line (no trailing comma, newline-delimited). Field names mirror src/types/index.ts exactly — no translation layer. Unknown fields are ignored on import for forward compatibility.

bookmarks-YYYY.jsonl

Year-sharded by UTC year of createdAt. Each line is one Bookmark object. Primary key: id.

Field	Type	Nullable	Example
`id`	`string`	no	`"bm_abc123"`
`tweetId`	`string`	no	`"1792345678901234567"`
`text`	`string`	no	`"Just shipped the new feature…"`
`createdAt`	`number`	no	`1715500800000` (Unix ms)
`sortIndex`	`string`	no	`"1792345678901234567"`
`bookmarked`	`boolean`	no	`true`
`author`	`Author`	no	See Author
`metrics`	`Metrics`	no	See Metrics
`media`	`Media[]`	no	See Media
`urls`	`TweetUrl[]`	no	See TweetUrl
`isThread`	`boolean`	no	`false`
`hasImage`	`boolean`	no	`true`
`hasVideo`	`boolean`	no	`false`
`hasLink`	`boolean`	no	`true`
`quotedTweet`	`QuotedTweet`	yes	See QuotedTweet
`retweetedTweet`	`QuotedTweet`	yes	Same shape as `quotedTweet`
`article`	`ArticleContent`	yes	See ArticleContent
`tweetKind`	`string`	yes	`"tweet"` \| `"reply"` \| `"quote"` \| `"repost"` \| `"thread"` \| `"article"`
`tweetDisplayType`	`string`	yes	Display hint from X API
`inReplyToTweetId`	`string`	yes	`"1792345678901234560"`
`inReplyToScreenName`	`string`	yes	`"someuser"`

details.jsonl

Thread and detail information for hydrated bookmarks. Sparse — only bookmarks with fetched details or a terminal unavailable status have entries. Primary key: tweetId.

Field	Type	Nullable	Description
`tweetId`	`string`	no	References a bookmark's `tweetId`
`fetchedAt`	`number`	no	Unix ms when detail was fetched
`focalTweet`	`Bookmark`	yes	Full bookmark object for the focal tweet (same shape as bookmarks JSONL)
`thread`	`ThreadTweet[]`	no	Array of tweets in the thread. See ThreadTweet
`detailsStatus`	`"ok" \| "unavailable"`	yes	Hydration result. Missing on legacy rows and treated as `"ok"`.
`unavailableReason`	`"deleted" \| "protected" \| "parse_failed" \| "unknown"`	yes	Present when `detailsStatus` is `"unavailable"`.

highlights.jsonl

User-created highlights and annotations. Append-only by nature. Primary key: id.

Field	Type	Nullable	Example
`id`	`string`	no	`"hl_xyz789"`
`tweetId`	`string`	no	`"1792345678901234567"`
`sectionId`	`string`	no	Section within article/tweet
`startOffset`	`number`	no	`42`
`endOffset`	`number`	no	`108`
`selectedText`	`string`	no	`"the key insight is…"`
`note`	`string`	yes	`"Follow up on this"`
`color`	`string`	no	`"yellow"`
`createdAt`	`number`	no	`1715500800000` (Unix ms)
`type`	`string`	yes	`"highlight"` \| `"note"`

reading-progress.jsonl

Reading state and scroll position tracking. Primary key: tweetId.

Field	Type	Nullable	Example
`tweetId`	`string`	no	`"1792345678901234567"`
`openedAt`	`number`	no	`1715500800000` (Unix ms)
`lastReadAt`	`number`	no	`1715501400000` (Unix ms)
`scrollY`	`number`	no	`1420` (pixels)
`scrollHeight`	`number`	no	`3200` (pixels)
`completed`	`boolean`	no	`true`
`reopenCount`	`number`	yes	`2` (defaults to 0 for legacy rows)

Nested types

Author

Field	Type	Nullable
`name`	`string`	no
`screenName`	`string`	no
`profileImageUrl`	`string`	no
`verified`	`boolean`	no
`bio`	`string`	yes
`followersCount`	`number`	yes
`followingCount`	`number`	yes
`website`	`string`	yes
`createdAt`	`string`	yes
`bannerUrl`	`string`	yes
`affiliate`	`{ name, badgeUrl?, url? }`	yes

 Metrics
   Field Type Nullable
 
  likes number no
 retweets number no
 replies number no
 views number no
 bookmarks number no
 
 
 Media
   Field Type Nullable
 
  type "photo" | "video" | "animated_gif" no
 url string no
 videoUrl string yes
 width number no
 height number no
 altText string yes
 
 
 TweetUrl
   Field Type Nullable
 
  url string no
 displayUrl string no
 expandedUrl string no
 card LinkCard yes
 
 
 LinkCard (nested in TweetUrl.card):
   Field Type Nullable
 
  title string yes
 description string yes
 imageUrl string yes
 imageAlt string yes
 domain string yes
 cardType string yes
 
 
 QuotedTweet
   Field Type Nullable
 
  tweetId string no
 text string no
 createdAt number no
 author Author no
 media Media[] no
 urls TweetUrl[] yes
 article ArticleContent yes
 
 
 ThreadTweet
 Same as QuotedTweet plus:
   Field Type Nullable
 
  quotedTweet QuotedTweet yes
 retweetedTweet QuotedTweet yes
 tweetKind string yes
 tweetDisplayType string yes
 inReplyToTweetId string yes
 inReplyToScreenName string yes
 isThread boolean yes
 
 
 ArticleContent
   Field Type Nullable
 
  title string yes
 plainText string no
 coverImageUrl string yes
 contentBlocks ArticleContentBlock[] yes
 entityMap Record<string, ArticleContentEntity> yes
 
 
 CSV format
  bookmarks.csv is an RFC 4180-compliant
        CSV file encoded as UTF-8 with a byte-order mark (BOM) so Excel renders emoji correctly.
        Line endings are \r\n. One header row, one data row per bookmark.
 Columns (in order)
   # Column Type Description
 
  1 tweet_id string X tweet ID
 2 tweet_url string https://x.com/{handle}/status/{id}
 3 author_handle string Author's X handle (no @ prefix)
 4 author_name string Author's display name
 5 text string Tweet body with card URLs stripped. Newlines preserved inside quoted field.
 6 created_at string ISO 8601 UTC (e.g. 2026-05-12T14:23:00.000Z)
 7 bookmarked_at string ISO 8601 UTC. Same as created_at (X does not expose a separate bookmark timestamp).
 8 media_urls string Pipe-separated (|) URLs. Photos first, then videos. Empty string if none.
 9 quoted_tweet_url string URL of the quoted tweet, or empty string.
 10 is_thread string "true" or "false"
 11 has_full_thread string "true" if a tweet_details row exists for this bookmark. Lets you filter to fully-hydrated rows in Sheets.
 
 
 Quoting rules
  Fields containing commas, double quotes, or line breaks are enclosed in double quotes.
 Double quotes within a field are escaped as "" (two double-quote characters).
 The UTF-8 BOM (U+FEFF) precedes the header row so Excel auto-detects encoding.
 
 Markdown format
  readme.md is a Markdown index linking to one file per bookmark under
bookmarks/. Bookmark files are ordered by bookmarkedAt descending
        (newest first), with zero-padded numeric prefixes to keep filenames sortable.
 Structure
 # Totem export — 3,000 bookmarks

Generated 2026-05-12 from @yourhandle.

## Bookmarks

- [First bookmark](bookmarks/0001-first-bookmark.md) · [Open on X](https://x.com/…)
- [Second bookmark](bookmarks/0002-second-bookmark.md) · [Open on X](https://x.com/…)

---

# First bookmark

By Author Name (@author)

> tweet body, threads inline if hydrated

Exported 2026-05-12 · [Open on X](https://x.com/…)

---
 Formatting rules
  readme.md uses a localized bookmark count (e.g. 3,000) and links to every bookmark file.
 Per-bookmark filenames use a zero-padded ordinal and a slug generated from the article title or first line of tweet text.
 Bookmark Markdown is rendered with the same structure as Totem's reader Copy Markdown export.
 Image Markdown includes source alt text when available and a descriptive fallback when X did not provide alt text.
 
 Re-import contract
 The Totem importer reads data/*.jsonl only. CSV and Markdown files inside the ZIP are derived views and are ignored on import.
 Import rules
  Additive only. Existing rows are never overwritten. If a row's primary key already exists in the local database, it is counted as already_had and skipped.
 Account check. The importer computes the SHA-256 of the active user's X ID and compares it to manifest.account.id_hash. Mismatch triggers a confirmation prompt (not a block).
 Schema check. Import refuses if manifest.totem.schema_version exceeds the local DB_VERSION. Update Totem first.
 Checksum verification. Every JSONL shard's SHA-256 is verified against manifest.checksums before any writes. Mismatch refuses the import.
 Per-store sequential. Stores are imported in order: bookmarks, details, highlights, reading progress. Failure on one store stops the chain but does not roll back successful stores.
 Batched writes. Inserts are batched at ~500 rows per IDB transaction commit.
 
 Primary keys per store
   Store Primary key field
 
  bookmarks id
 details tweetId
 highlights id
 reading_progress tweetId
 
 
 Versioning policy
  v1 is frozen. The field set, file layout, and CSV columns documented on this page will not change. New fields may be added to JSONL rows (importers ignore unknown fields), but no existing fields will be removed or have their types changed.
 v2 will live at a separate URL (/export-format/v2) if and when breaking changes are needed.
 export_version in manifest.json identifies the ZIP format version (this page documents version 1).
 schema_version in manifest.json tracks the IDB schema version and controls import compatibility. Field-additive changes do not bump export_version; only structural breaks do.
 
 
 
This documentation corresponds to Totem export format v1, generated by Totem ≥ 1.2.2.
        For questions or tooling support, see the project repository.

Field	Type	Nullable
`likes`	`number`	no
`retweets`	`number`	no
`replies`	`number`	no
`views`	`number`	no
`bookmarks`	`number`	no

Field	Type	Nullable
`type`	`"photo" \| "video" \| "animated_gif"`	no
`url`	`string`	no
`videoUrl`	`string`	yes
`width`	`number`	no
`height`	`number`	no
`altText`	`string`	yes

Field	Type	Nullable
`url`	`string`	no
`displayUrl`	`string`	no
`expandedUrl`	`string`	no
`card`	`LinkCard`	yes

Field	Type	Nullable
`title`	`string`	yes
`description`	`string`	yes
`imageUrl`	`string`	yes
`imageAlt`	`string`	yes
`domain`	`string`	yes
`cardType`	`string`	yes

Field	Type	Nullable
`title`	`string`	yes
`plainText`	`string`	no
`coverImageUrl`	`string`	yes
`contentBlocks`	`ArticleContentBlock[]`	yes
`entityMap`	`Record<string, ArticleContentEntity>`	yes

#	Column	Type	Description
1	`tweet_id`	string	X tweet ID
2	`tweet_url`	string	`https://x.com/{handle}/status/{id}`
3	`author_handle`	string	Author's X handle (no `@` prefix)
4	`author_name`	string	Author's display name
5	`text`	string	Tweet body with card URLs stripped. Newlines preserved inside quoted field.
6	`created_at`	string	ISO 8601 UTC (e.g. `2026-05-12T14:23:00.000Z`)
7	`bookmarked_at`	string	ISO 8601 UTC. Same as `created_at` (X does not expose a separate bookmark timestamp).
8	`media_urls`	string	Pipe-separated (`\|`) URLs. Photos first, then videos. Empty string if none.
9	`quoted_tweet_url`	string	URL of the quoted tweet, or empty string.
10	`is_thread`	string	`"true"` or `"false"`
11	`has_full_thread`	string	`"true"` if a `tweet_details` row exists for this bookmark. Lets you filter to fully-hydrated rows in Sheets.

Store	Primary key field
bookmarks	`id`
details	`tweetId`
highlights	`id`
reading_progress	`tweetId`