Fix x-cli me bookmarks by adding oauth2 login flow

.gitignore

@@ -5,3 +5,5 @@ __pycache__/
 dist/
 build/
 .pytest_cache/
+AGENTS.md
+tasks/

LLMs.md

@@ -6,9 +6,14 @@ You are an AI agent working with the x-cli codebase. This file tells you where e
 
 ## What This Is
 
-x-cli is a Python CLI that talks directly to the Twitter/X API v2. It uses OAuth 1.0a for write operations and Bearer token auth for read operations. No third-party auth libraries -- signing is done with stdlib `hmac`/`hashlib`.
+x-cli is a Python CLI that talks directly to the Twitter/X API v2. It uses:
+- OAuth 1.0a for user-context write/engagement endpoints (tweet post/delete, like, retweet, mentions).
+- App Bearer token for public read endpoints.
+- OAuth 2.0 User Context (PKCE) for bookmarks endpoints.
 
-It shares the same credentials as x-mcp (the MCP server counterpart). If a user already has x-mcp configured, they can symlink its `.env` to `~/.config/x-cli/.env`.
+No third-party auth frameworks are used; OAuth signing/challenge logic is implemented in project code.
+
+It shares static credentials with x-mcp via `~/.config/x-cli/.env`. Mutable OAuth2 token keys are stored in `~/.config/x-cli/.env.auth2`.
 
 ---
 
@@ -17,11 +22,15 @@ It shares the same credentials as x-mcp (the MCP server counterpart). If a user
 ```
 src/x_cli/
     cli.py          -- Click command groups and entry point
-    api.py          -- XApiClient: one method per Twitter API v2 endpoint
-    auth.py         -- Credential loading and OAuth 1.0a HMAC-SHA1 signing
+    api.py          -- XApiClient: endpoint methods + auth routing
+    auth.py         -- Env loading + OAuth 1.0a HMAC-SHA1 signing
+    oauth2.py       -- OAuth2 PKCE helpers, token exchange/refresh, token persistence
     formatters.py   -- Human (rich), JSON, and TSV output modes
     utils.py        -- Tweet ID parsing from URLs, username stripping
 tests/
+    test_api.py
+    test_cli_auth.py
+    test_oauth2.py
     test_utils.py
     test_formatters.py
     test_auth.py
@@ -33,7 +42,7 @@ tests/
 
 ### `cli.py` -- Start here
 
-The entry point. Defines Click command groups: `tweet`, `user`, `me`, plus top-level `like` and `retweet`. Every command follows the same pattern: parse args, call the API client, pass the response to a formatter.
+The entry point. Defines Click command groups: `auth`, `tweet`, `user`, `me`, plus top-level `like` and `retweet`. Most commands follow the same pattern: parse args, call the API client, pass the response to a formatter.
 
 The `State` object holds the output mode (`human`/`json`/`plain`/`markdown`) and verbose flag, and lazily initializes the API client. It's passed via Click's context system (`@pass_state`).
 
@@ -43,22 +52,33 @@ Global flags: `-j`/`--json`, `-p`/`--plain`, `-md`/`--markdown` control output m
 
 `XApiClient` wraps all Twitter API v2 endpoints. Key patterns:
 
-- **Read-only endpoints** (get_tweet, search, get_user, get_timeline, get_followers, get_following) use Bearer token auth via `_bearer_get()` or direct `httpx` calls with Bearer header.
-- **Write endpoints** (post_tweet, delete_tweet, like, retweet, bookmark) use OAuth 1.0a via `_oauth_request()`.
-- **Authenticated read endpoints** (get_mentions, get_bookmarks) use OAuth 1.0a because they access the authenticated user's data.
-- `get_authenticated_user_id()` resolves and caches the current user's numeric ID (needed for like/retweet/bookmark/mentions endpoints).
+- **Read-only endpoints** (get_tweet, search, get_user, get_timeline, get_followers, get_following) use app Bearer token auth.
+- **OAuth 1.0a endpoints** (post_tweet, delete_tweet, like, retweet, mentions) use `_oauth_request()`.
+- **OAuth 2.0 user-context endpoints** (bookmarks) use `_oauth2_user_request()`.
+- `get_authenticated_user_id()` caches OAuth1 user id; `get_authenticated_user_id_oauth2()` caches OAuth2 user id.
+- OAuth2 refresh is automatic using stored refresh token and expiry metadata.
+- OAuth2 token exchange/refresh optionally uses `X_OAUTH2_CLIENT_SECRET` for app types that require client authentication.
 
 All methods return raw `dict` parsed from the API JSON response. Error handling is in `_handle()` -- raises `RuntimeError` on non-2xx or rate limit responses.
 
-### `auth.py` -- OAuth signing
+### `auth.py` -- Env + OAuth1 signing
 
 Two responsibilities:
 
-1. **`load_credentials()`** -- Loads 5 env vars (`X_API_KEY`, `X_API_SECRET`, `X_ACCESS_TOKEN`, `X_ACCESS_TOKEN_SECRET`, `X_BEARER_TOKEN`) with `.env` fallback from `~/.config/x-cli/.env` and current directory.
+1. **`load_credentials()`** -- Loads static vars from `~/.config/x-cli/.env` and current directory `.env`, then overlays mutable OAuth2 token vars from `~/.config/x-cli/.env.auth2`.
 2. **`generate_oauth_header()`** -- Builds an OAuth 1.0a `Authorization` header using HMAC-SHA1. Follows the standard OAuth signature base string construction: percent-encode params, sort, concatenate with `&`, sign with consumer secret + token secret.
 
 Query string parameters from the URL are included in the signature base string (required by OAuth spec).
 
+### `oauth2.py` -- OAuth2 PKCE + token management
+
+- Generates PKCE verifier/challenge/state.
+- Builds browser authorization URL.
+- Parses redirected URL for code/state validation (user must paste full browser address-bar URL).
+- Exchanges authorization code for tokens and refreshes access token.
+- Persists/removes OAuth2 token env vars in `~/.config/x-cli/.env.auth2`.
+- Auto-migrates legacy token keys from `~/.config/x-cli/.env` to `.env.auth2`.
+
 ### `formatters.py` -- Output
 
 Four modes routed by `format_output(data, mode, title, verbose)`:
@@ -105,9 +125,17 @@ Note: `timeline`, `followers`, `following` resolve username to numeric ID automa
 | Command | Args | Flags | API method |
 |---------|------|-------|------------|
 | `mentions` | | `--max N` | `get_mentions()` |
-| `bookmarks` | | `--max N` | `get_bookmarks()` |
-| `bookmark` | `ID_OR_URL` | | `bookmark_tweet()` |
-| `unbookmark` | `ID_OR_URL` | | `unbookmark_tweet()` |
+| `bookmarks` | | `--max N` | `get_bookmarks()` (OAuth2 login required) |
+| `bookmark` | `ID_OR_URL` | | `bookmark_tweet()` (OAuth2 login required) |
+| `unbookmark` | `ID_OR_URL` | | `unbookmark_tweet()` (OAuth2 login required) |
+
+### Auth commands (`x-cli auth <action>`)
+
+| Command | Purpose |
+|---------|---------|
+| `login` | Run OAuth2 PKCE browser flow; store tokens |
+| `status` | Show OAuth2 token presence/expiry |
+| `logout` | Remove stored OAuth2 tokens |
 
 ### Top-level commands
 
@@ -139,7 +167,7 @@ The Twitter API v2 requires numeric user IDs for timeline/followers/following en
 uv run pytest tests/ -v
 ```
 
-Tests cover utils (tweet ID parsing), formatters (JSON/TSV output), and auth (OAuth header generation). No live API calls in tests.
+Tests cover utils, formatters, OAuth1 signing, OAuth2 helpers, API auth routing, and auth CLI command behavior. No live API calls in tests.
 
 ---
 
@@ -148,7 +176,12 @@ Tests cover utils (tweet ID parsing), formatters (JSON/TSV output), and auth (OA
 | Error | Cause | Fix |
 |-------|-------|-----|
 | 403 "oauth1-permissions" | Access Token is Read-only | Enable "Read and write" in app settings, regenerate Access Token |
+| "Missing OAuth2 user token for bookmarks" | OAuth2 login not completed | Set `X_OAUTH2_CLIENT_ID` and run `x-cli auth login` |
+| X consent page says "Something went wrong" | Callback URL mismatch | Configure callback exactly as `https://example.com/oauth/callback` (or set matching `X_OAUTH2_REDIRECT_URI`) |
+| "OAuth2 token request failed (HTTP 401): Missing valid authorization header" | App requires OAuth2 client authentication for token exchange | Set `X_OAUTH2_CLIENT_SECRET` and retry `x-cli auth login` |
+| "X_OAUTH2_ACCESS_TOKEN is not a user-context token" | Token is OAuth2 app-only bearer token | Run `x-cli auth login` and use returned user-context token |
+| 401 on bookmarks after login | OAuth2 token expired/revoked and refresh failed | Re-run `x-cli auth login` |
 | 401 Unauthorized | Bad credentials | Verify all 5 values in `.env` |
 | 429 Rate Limited | Too many requests | Error includes reset timestamp |
-| "Missing env var" | `.env` not found or incomplete | Check `~/.config/x-cli/.env` or set env vars directly |
+| "Missing env var" | Static `.env` missing required keys | Check `~/.config/x-cli/.env` (and optional cwd `.env`) |
 | `RuntimeError: API error` | Twitter API returned an error | Check the error message for details (usually permissions or invalid IDs) |

README.md

@@ -1,6 +1,6 @@
 # x-cli
 
-A CLI for X/Twitter that talks directly to the API v2. Post tweets, search, read timelines, manage bookmarks -- all from your terminal.
+A CLI for X/Twitter that talks directly to the API v2. Post tweets, search, read timelines, and manage engagement from your terminal.
 
 Uses the same auth credentials as [x-mcp](https://github.com/INFATOSHI/x-mcp). If you already have x-mcp set up, x-cli works with zero additional config.
 
@@ -16,7 +16,7 @@ Uses the same auth credentials as [x-mcp](https://github.com/INFATOSHI/x-mcp). I
 | **Read** | `tweet get`, `tweet search`, `user timeline`, `me mentions` | `x-cli tweet search "from:elonmusk"` |
 | **Users** | `user get`, `user followers`, `user following` | `x-cli user get openai` |
 | **Engage** | `like`, `retweet` | `x-cli like <tweet-url>` |
-| **Bookmarks** | `me bookmarks`, `me bookmark`, `me unbookmark` | `x-cli me bookmarks --max 20` |
+| **Bookmarks** | `me bookmarks`, `me bookmark`, `me unbookmark` | `x-cli auth login && x-cli me bookmarks --max 20` |
 | **Analytics** | `tweet metrics` | `x-cli tweet metrics <tweet-id>` |
 
 Accepts tweet URLs or IDs interchangeably -- paste `https://x.com/user/status/123` or just `123`.
@@ -39,7 +39,7 @@ uv tool install x-cli
 
 ## Auth
 
-You need 5 credentials from the [X Developer Portal](https://developer.x.com/en/portal/dashboard).
+You need OAuth1 credentials for general commands and OAuth2 client settings for bookmarks login.
 
 ### If you already use x-mcp
 
@@ -57,19 +57,63 @@ ln -s /path/to/x-mcp/.env ~/.config/x-cli/.env
 3. Save your **Consumer Key** (API Key), **Secret Key** (API Secret), and **Bearer Token**
 4. Under **User authentication settings**, set permissions to **Read and write**
 5. Generate (or regenerate) **Access Token** and **Access Token Secret**
+6. In OAuth2 app settings, add callback URL: `https://example.com/oauth/callback`
 
-Put all 5 values in `~/.config/x-cli/.env`:
+Put these values in `~/.config/x-cli/.env`:
 
 ```
 X_API_KEY=your_consumer_key
 X_API_SECRET=your_secret_key
 X_BEARER_TOKEN=your_bearer_token
 X_ACCESS_TOKEN=your_access_token
 X_ACCESS_TOKEN_SECRET=your_access_token_secret
+X_OAUTH2_CLIENT_ID=your_oauth2_client_id
+X_OAUTH2_CLIENT_SECRET=your_oauth2_client_secret  # optional, required by some X app types
+# Optional: override callback URL if your app uses a different one.
+# Must exactly match your app callback setting.
+# X_OAUTH2_REDIRECT_URI=https://example.com/oauth/callback
 ```
 
 x-cli also checks for a `.env` in the current directory.
 
+Mutable OAuth2 token values are stored separately in:
+
+```bash
+~/.config/x-cli/.env.auth2
+```
+
+Managed keys:
+- `X_OAUTH2_ACCESS_TOKEN`
+- `X_OAUTH2_REFRESH_TOKEN`
+- `X_OAUTH2_EXPIRES_AT`
+
+If these keys already exist in `~/.config/x-cli/.env`, x-cli auto-migrates them to `.env.auth2`.
+
+### OAuth2 login for bookmarks
+
+Bookmarks endpoints require OAuth 2.0 User Context. Run:
+
+```bash
+x-cli auth login
+```
+
+`auth login` opens a PKCE browser flow:
+1. It prints an authorize URL.
+2. You approve access in the browser.
+3. Copy the full redirected URL from your browser address bar and paste it into the CLI.
+4. x-cli stores:
+   - `X_OAUTH2_ACCESS_TOKEN`
+   - `X_OAUTH2_REFRESH_TOKEN`
+   - `X_OAUTH2_EXPIRES_AT`
+   in `~/.config/x-cli/.env.auth2`
+
+You can check or clear saved OAuth2 tokens:
+
+```bash
+x-cli auth status
+x-cli auth logout
+```
+
 ---
 
 ## Usage
@@ -146,11 +190,31 @@ Your Access Token was generated before you enabled write permissions. Go to the
 ### 401 Unauthorized
 Double-check all 5 credentials in your `.env`. No extra spaces or newlines.
 
+### Bookmarks fail with "Missing OAuth2 user token"
+Run `x-cli auth login`. You must set `X_OAUTH2_CLIENT_ID` first.
+
+### Login page says "Something went wrong"
+Most common cause is callback mismatch. Ensure your X app has callback URL exactly:
+`https://example.com/oauth/callback`
+If you set `X_OAUTH2_REDIRECT_URI`, it must exactly match the callback in X app settings.
+
+### Bookmarks fail saying token is not user-context
+Your `X_OAUTH2_ACCESS_TOKEN` is likely an OAuth2 app-only token. Run `x-cli auth login` to mint a user-context token.
+
+### `auth login` fails with "Missing valid authorization header"
+Set `X_OAUTH2_CLIENT_SECRET` in your env and retry `x-cli auth login`. Some X app configurations require client authentication at token exchange time.
+
+### What URL should I paste into the CLI prompt?
+Paste the full redirected URL from your browser address bar (the one containing `code=` and `state=`), not just the page contents.
+
+### Bookmarks fail with 401 after login
+Your refresh token may be expired/revoked. Run `x-cli auth login` again to refresh OAuth2 credentials.
+
 ### 429 Rate Limited
 The error includes the reset timestamp. Wait until then.
 
 ### "Missing env var" on startup
-x-cli looks for credentials in `~/.config/x-cli/.env`, then the current directory's `.env`, then environment variables. Make sure at least one source has all 5 values.
+x-cli loads static credentials from `~/.config/x-cli/.env` and current `.env`, then overlays mutable OAuth2 token keys from `~/.config/x-cli/.env.auth2`.
 
 ---