Skip to content

feat: Add Signet Orders Indexer for cross-chain order tracking#2

Open
init4samwise wants to merge 10 commits intomasterfrom
feat/signet-orders-indexer
Open

feat: Add Signet Orders Indexer for cross-chain order tracking#2
init4samwise wants to merge 10 commits intomasterfrom
feat/signet-orders-indexer

Conversation

@init4samwise
Copy link

@init4samwise init4samwise commented Feb 16, 2026

Summary

Implements the Signet Orders Fetcher for ENG-1894 as part of the broader Signet order tracking project (ENG-1876).

Changes

New Database Tables

  • signet_orders: Stores Order events with inputs, outputs, deadline, and optional Sweep data
  • signet_fills: Stores Filled events from both rollup (L2) and host (L1) chains

Indexer Implementation

  • OrdersFetcher: Main fetcher module using BufferedTask pattern (same as Arbitrum fetchers)
  • EventParser: ABI decoding for Order, Filled, Sweep events using @signet-sh/sdk ABIs
  • ReorgHandler: Graceful handling of chain reorganizations
  • Import Runners: Bulk import support for orders and fills

Key Features

  1. ✅ Parse Order events from rollup chain
  2. ✅ Parse Filled events from both chains
  3. ✅ Index all events by block number (block-level coordination)
  4. ✅ Index by transaction hash
  5. ✅ Handle chain reorgs gracefully
  6. ✅ Add metrics/logging for indexer health

Note: Orders and fills are indexed independently. Only block-level coordination is possible — direct order-to-fill correlation is not supported.

Configuration

config :indexer, Indexer.Fetcher.Signet.OrdersFetcher,
  enabled: true,
  rollup_orders_address: "0x...",
  host_orders_address: "0x...",
  l1_rpc: "https://...",
  l1_rpc_block_range: 1000,
  recheck_interval: 15_000,
  start_block: 0

Files Added

  • apps/explorer/lib/explorer/chain/signet/order.ex
  • apps/explorer/lib/explorer/chain/signet/fill.ex
  • apps/explorer/lib/explorer/chain/import/runner/signet/orders.ex
  • apps/explorer/lib/explorer/chain/import/runner/signet/fills.ex
  • apps/explorer/priv/signet/migrations/20260216040000_create_signet_tables.exs
  • apps/indexer/lib/indexer/fetcher/signet/orders_fetcher.ex
  • apps/indexer/lib/indexer/fetcher/signet/event_parser.ex
  • apps/indexer/lib/indexer/fetcher/signet/reorg_handler.ex
  • apps/indexer/lib/indexer/fetcher/signet/utils/db.ex
  • apps/indexer/lib/indexer/fetcher/signet/orders_fetcher/supervisor.ex
  • apps/indexer/lib/indexer/fetcher/signet/README.md

Testing

TODO: Add integration tests in signet-test-utils

Closes ENG-1894

Implements indexer fetcher for Signet order tracking:

## New Files

Database Schema:
- apps/explorer/lib/explorer/chain/signet/order.ex
- apps/explorer/lib/explorer/chain/signet/fill.ex

Migration:
- apps/explorer/priv/signet/migrations/20260216040000_create_signet_tables.exs

Import Runners:
- apps/explorer/lib/explorer/chain/import/runner/signet/orders.ex
- apps/explorer/lib/explorer/chain/import/runner/signet/fills.ex

Fetcher:
- apps/indexer/lib/indexer/fetcher/signet/orders_fetcher.ex
- apps/indexer/lib/indexer/fetcher/signet/event_parser.ex
- apps/indexer/lib/indexer/fetcher/signet/reorg_handler.ex
- apps/indexer/lib/indexer/fetcher/signet/utils/db.ex
- apps/indexer/lib/indexer/fetcher/signet/orders_fetcher/supervisor.ex

## Features

- Parse Order, Filled, Sweep events from RollupOrders contract
- Parse Filled events from HostOrders contract
- Compute outputs_witness_hash for cross-chain correlation
- Insert into signet_orders / signet_fills tables
- Handle chain reorgs gracefully
- Add metrics/logging for indexer health

## Modified Files

- apps/explorer/lib/explorer/chain/import/stage/chain_type_specific.ex
  - Added signet runners to chain type specific import stage
- apps/indexer/lib/indexer/supervisor.ex
  - Added SignetOrdersFetcher to supervisor

Closes ENG-1894
Copy link
Member

@prestwich prestwich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@init4samwise does this PR account for the fact that Output chain id semantics are different in orders and fills?

in an order the chain_id is the TARGET chain, i.e. where assets should be delivered
in a fill the chain_id is the ORIGINATING chain

@init4samwise
Copy link
Author

@prestwich Good question - I need to clarify the chain ID semantics.

Looking at the current implementation:

  1. Order events are indexed from the rollup chain. The outputs in the order specify where assets should be delivered (target chain). However, the Output struct in the event is (recipient, token, amount) - I don't see a chain_id field in the event data itself.

  2. Filled events are indexed from BOTH rollup and host chains (via parse_rollup_logs and parse_host_filled_logs). The fill happens on the originating chain.

  3. Cross-chain correlation uses outputs_witness_hash - which should match between orders and fills regardless of chain.

Questions:

  • Does the Output struct in the actual contract events include a chain_id? Looking at Output = (address recipient, address token, uint256 amount) in my parser, I don't see it.
  • Should the indexer track: (a) which chain the order targets, and (b) which chain the fill originated from?
  • If chain_id is in the calldata but not the event, should I be decoding the original transaction input instead?

Want to make sure I understand the contract semantics correctly before adjusting.

@prestwich
Copy link
Member

    /// @notice Tokens sent by the swapper as inputs to the order
    /// @dev From ERC-7683
    struct Input {
        /// @dev The address of the ERC20 token on the origin chain
        address token;
        /// @dev The amount of the token to be sent
        uint256 amount;
    }

    /// @notice Tokens that must be receive for a valid order fulfillment
    /// @dev From ERC-7683
    struct Output {
        /// @dev The address of the ERC20 token on the destination chain
        /// @dev address(0) used as a sentinel for the native token
        address token;
        /// @dev The amount of the token to be sent
        uint256 amount;
        /// @dev The address to receive the output tokens
        address recipient;
        /// @dev When emitted on the origin chain, the destination chain for the Output.
        ///      When emitted on the destination chain, the origin chain for the Order containing the Output.
        uint32 chainId;
    }
    ```

@init4samwise
Copy link
Author

Thanks for catching this @prestwich! You're right - I reviewed against the signet-sdk ABI and found:

  1. Output struct is wrong - missing chainId (uint32) and field order is incorrect
  2. Witness hash computation will be incorrect as a result
  3. chainId semantic difference needs handling - target chain in orders vs originating chain in fills

I've posted a detailed breakdown to #eng. Will need to fix:

  • ABI decoding in decode_output_array/2
  • compute_outputs_witness_hash/1 to include chainId
  • JSON format to include chainId
  • Documentation for the semantic difference

Want me to push a fix, or would you prefer to take this?

- Add tools/signet-sdk to extract ABIs from npm package
- Store extracted ABIs in apps/explorer/priv/contracts_abi/signet/
- Create Indexer.Fetcher.Signet.Abi module for event topic computation
- Update EventParser to use correct SDK-defined event signatures
- Fix Output struct decoding: (token, amount, recipient, chainId)
- Add chainId to formatted JSON outputs for cross-chain tracking

Key changes:
- Event signatures now match @signet-sh/sdk v0.3.0
- Output struct properly decoded with all 4 fields
- Sweep event correctly parses indexed topics for recipient/token

To update ABIs when SDK changes:
  cd tools/signet-sdk && npm install && npm run extract
- Fix Output struct documentation to match @signet-sh/sdk:
  (token, amount, recipient, chainId) not (recipient, token, amount)
- Add comprehensive tests for Abi module:
  - Event topic hash computation and consistency
  - Event signatures format validation
  - ABI file loading from priv directory
- Add comprehensive tests for EventParser module:
  - outputs_witness_hash determinism and correctness
  - Output field order verification (critical for cross-chain correlation)
  - Log parsing edge cases
  - Event topic matching with Abi module

Part of Phase 1 completion for ENG-1876
Per James's architecture update: orders and fills cannot be correlated
directly - only block-level coordination is possible.

Changes:
- Remove compute_outputs_witness_hash from EventParser
- Change Order primary key to (transaction_hash, log_index)
- Change Fill primary key to (chain_type, transaction_hash, log_index)
- Update migration to use new composite primary keys
- Update import runners for new primary keys
- Remove correlation tests from EventParserTest
- Update module docs to reflect independent indexing
Per James's request - use exact version 0.4.4 instead of ^0.3.0
…semantics

- Remove outdated outputs_witness_hash correlation documentation
- Update database tables to show composite primary keys
- Add chainId semantics explanation (destination in orders, origin in fills)
- Clarify that orders and fills are indexed independently

Per architecture update from James
@init4samwise
Copy link
Author

@prestwich Update on your review comments:

I've addressed the Output struct and chainId issues:

  1. Output struct fixed (commit 3bc412e) - Now correctly decodes (token, amount, recipient, chainId) per SDK spec

  2. Correlation logic removed (commit dff1fa1) - Per your architecture guidance, orders and fills are now indexed independently with composite primary keys:

    • Orders: (transaction_hash, log_index)
    • Fills: (chain_type, transaction_hash, log_index)
  3. SDK pinned to v0.4.4 (commit 5996d5a) - Using exact version as requested

  4. Documentation updated (just pushed) - README now explains:

    • New primary key structure (no outputs_witness_hash)
    • chainId semantic difference (destination in orders, origin in fills)

The outputs_json field stores the full Output struct including chainId, so the semantic difference is preserved for downstream consumers to interpret correctly.

Is there anything else you'd like me to address before this can be merged?

@init4samwise
Copy link
Author

@prestwich Friendly follow-up - let me know if there's anything else you'd like addressed on this PR. Happy to make any additional changes.

In the meantime, I'll start working on Phase 2 (integration testing) in parallel so we're ready to move quickly once this is approved.

Phase 2 progress for ENG-1876: Added factory entries for test data generation.
- signet_order_factory: generates Order structs with proper JSON fields
- signet_fill_factory: generates Fill structs with chain_type and outputs

These factories enable integration testing of the signet indexer.
@prestwich
Copy link
Member

[Claude Code]

PR Review: feat: Add Signet Orders Indexer

Overall this is a solid foundation for cross-chain order/fill indexing. The structure follows Blockscout's patterns well (BufferedTask, import runners, Ecto schemas). However, there are several issues that need to be addressed before merge — including a couple that will prevent compilation.


Critical (won't compile/run)

1. Missing import Ecto.Query in orders_fetcher.ex

get_last_processed_block/2 uses from(o in Order, ...) and from(f in Fill, ...) but the module never imports Ecto.Query. Every other module in this PR that uses from/2 (Runner.Signet.Fills, Runner.Signet.Orders, reorg_handler.ex, utils/db.ex) correctly includes the import. This will fail at compile time.

Fix: Add import Ecto.Query, only: [from: 2] to the module header.

2. utils/db.ex references non-existent outputs_witness_hash column

Every query function in this module (get_order_by_witness_hash/1, get_fills_for_order/1, order_filled_on_chain?/2, get_unfilled_orders/1, get_order_fill_counts/0) references outputs_witness_hash — a field that doesn't exist in either the Order or Fill schema, nor in the migration. This looks like residual code from an earlier design iteration. All of these will raise Ecto.QueryError at runtime.

Fix: Either remove utils/db.ex entirely (it's unused — nothing calls it), or update the queries to use the actual schema fields (transaction_hash, log_index).


High (incorrect behavior at runtime)

3. Unbounded eth_getLogs range on rollup chain

In fetch_and_process_rollup_events/1, the rollup path queries from start_block to latest_block with no range cap. The host path correctly caps at l1_rpc_block_range. If the fetcher falls behind or starts fresh, this will request an enormous block range that most RPC providers reject (typical limits are 2000-10000 blocks).

Fix: Apply a similar min(start_block + rollup_block_range, latest_block) cap and iterate in chunks.

4. handle_reorg is dead code

ReorgHandler provides handle_reorg/2 and block_still_valid?/3, but nothing ever calls them. The fetcher doesn't subscribe to Blockscout's reorg detection pipeline and doesn't proactively check for reorgs. The start_block cursor only moves forward.

Fix: Either integrate with Blockscout's existing reorg detection (subscribe to reorg events) or remove the reorg handler to avoid giving a false sense of safety.

5. with chains have no else clauses + import_orders/import_fills crash on error

The with chains in fetch_and_process_rollup_events/1 and fetch_and_process_host_events/1 have no else clause. Meanwhile, import_orders/1 does {:ok, _} = Chain.import(...) which will raise MatchError on failure. Some errors propagate silently while others crash — inconsistent behavior.

Fix: Add else clauses to with blocks, and wrap Chain.import in case returning {:error, reason}.

6. get_latest_block hex parsing is fragile

{block, ""} = Integer.parse(String.trim_leading(hex_block, "0x"), 16)

If the response isn't a clean hex string (e.g. "0x0" may work, but edge cases around response format vary by RPC provider), the pattern match on "" will crash. Consider using EthereumJSONRPC.quantity_to_integer/1 which Blockscout already provides for this exact purpose.


Medium (design/correctness concerns)

7. Config key mismatch — fetcher may never start

Indexer.Supervisor.configure/2 checks Application.get_env(:indexer, process)[:enabled] where process is Indexer.Fetcher.Signet.OrdersFetcher.Supervisor. But the supervisor's disabled?/0 checks Application.get_env(:indexer, OrdersFetcher, []) (the fetcher module, not the supervisor module). These are different config keys. The configure check will never find :enabled under the supervisor key.

Fix: Ensure the config key matches what configure/2 looks up, or bypass configure and use a direct {Module, opts} tuple (like ArbitrumMessagesToL2Matcher.Supervisor does).

8. get_last_processed_block(:rollup, ...) only checks Orders, not rollup Fills

The rollup chain can emit both Order and Filled events. But get_last_processed_block(:rollup, ...) only queries the Order table for max block. If a block only contained Filled events, on restart the fetcher could re-process or skip that block range.

Fix: Query both Order and Fill (where chain_type = :rollup) and take the max of the two.

9. No transaction wrapping in reorg cleanup

handle_rollup_reorg/1 deletes orders and fills in two separate Repo.delete_all calls with no transaction. If the process crashes between them, the database is left inconsistent.

Fix: Wrap in Repo.transaction/1.

10. Sweep-to-Order association uses only the first sweep per transaction

case Map.get(sweeps_by_tx, order.transaction_hash) do
  [sweep | _] -> ...

If a transaction emits multiple Order + Sweep events, the first sweep is applied to all orders from that transaction. The other sweeps are silently dropped.

11. ABI offset arithmetic in decode_order_data/1 is fragile

The manual binary_part(rest, inputs_offset - 96, ...) calculation assumes offsets are always exactly 96 for the 3 header words. No bounds checking — an out-of-range offset will ArgumentError with a generic rescue message that doesn't indicate the real issue.


Low

12. parse_block_number and parse_log_index silently return 0 on failure — This can create records with block_number = 0 or log_index = 0 that collide with legitimate data. Prefer returning an error.

13. Multiple files missing trailing newline — The diff shows \ No newline at end of file on ~10 files (ABI JSON, some Elixir files).

14. PR description says "TODO: Add integration tests" — Worth tracking this as a follow-up.


What looks good

  • Schema design for signet_orders and signet_fills is clean and well-structured
  • Import runners follow the existing Blockscout pattern correctly (conflict handling, timestamps, Instrumenter integration)
  • Migration uses appropriate types, composite primary keys, and indexes
  • The @signet-sh/sdk ABI extraction tooling is a nice approach for keeping ABIs in sync
  • EventParser unit tests are thorough with good edge case coverage
  • Factory entries follow existing patterns

@prestwich
Copy link
Member

[Claude Code]

Follow-up: Test Quality Gaps

A few additional observations on the test coverage in this PR.


1. No tests for actual ABI-encoded event decoding

The core functionality of EventParser — decoding ABI-encoded binary data from Order, Filled, and Sweep event logs — has zero test coverage. The binary decoding logic in decode_order_data/1, decode_filled_data/1, decode_output_array/1, and decode_input_array/1 is the most complex and error-prone code in the PR, and none of it is exercised by any test.

At minimum, there should be a happy-path test for each event type that constructs a valid ABI-encoded log and verifies the parser produces the correct output. These can be generated using the @signet-sh/sdk or by encoding test data with ABI.TypeEncoder in Elixir.


2. Several tests are tautologies

event_parser_test.exs — "output struct field order" test (~line 5200):

This test creates an Elixir tuple, destructures it, and asserts the destructured values match the originals. It tests Elixir's tuple semantics, not EventParser behavior. A meaningful version would ABI-encode an Output struct and verify the parser extracts fields in the correct order.

event_parser_test.exs — "parsed event structure" tests (~line 5281):

These tests create a list of atoms and assert atoms are in the list. They never invoke EventParser. They provide false coverage — if someone looks at test counts, these suggest the parser's output structure is validated when it isn't.


3. ABI topic hash tests don't assert known-good values

abi_test.exs:

The tests verify that topic hashes are 66 characters long and hex-formatted, but never assert the actual expected keccak256 values. If the event signature string had a typo (e.g., wrong parameter order or type), all tests would still pass. At least one test per event should assert the exact expected topic hash, cross-referenced with the SDK or computed independently:

# Example:
assert Abi.order_event_topic() == "0x<expected_keccak256_hash>"

4. Sweep-to-order association is untested

associate_sweeps_with_orders/2 pairs sweeps with orders by transaction hash, but this logic has no tests. Key untested edge cases:

  • Multiple sweeps in a single transaction (currently only the first is used)
  • Orders with no corresponding sweep
  • Multiple orders sharing a transaction hash

5. No tests for ReorgHandler, OrdersFetcher, or Supervisor

  • ReorgHandler.handle_reorg/2 — the database deletion logic is untested
  • OrdersFetcher — no integration or unit tests for the main fetcher loop, task scheduling, or error recovery
  • OrdersFetcher.Supervisor.disabled?/0 — no tests for the enable/disable config logic

6. Factory log_index uses Enum.random instead of sequence

Both signet_order_factory/0 and signet_fill_factory/0 use log_index: Enum.random(0..100). Since (transaction_hash, log_index) is part of the composite primary key, random values can cause non-deterministic test failures from key collisions. Consider using sequence(:signet_log_index, & &1) instead.

Per PR review feedback from @prestwich:
- In Order events: chainId = destination chain (where assets should be delivered)
- In Filled events: chainId = origin chain (where the order was created)

Added inline documentation to the @TypeDoc in both schema files to make
this semantic difference explicit for future developers.
@init4samwise init4samwise requested a review from a team February 17, 2026 18:36
@init4samwise
Copy link
Author

@prestwich Addressed your question about chainId semantics (commit b1a0ef6):

I've added explicit documentation to the schema files clarifying the semantic difference:

order.ex - outputs_json now documents:

In Order events, the chainId field represents the DESTINATION chain (where assets should be delivered), not the chain where the order was created.

fill.ex - outputs_json now documents:

In Filled events, the chainId field represents the ORIGIN chain (where the order was created), not the chain where the fill occurred.

This supplements the existing "chainId Semantics" section in the README.md which already explained this at a higher level.

The semantic difference is now documented at three levels:

  1. README.md - high-level explanation for consumers
  2. Schema @TypeDoc - inline docs for developers working with the code
  3. The outputs_json field preserves the raw chainId so downstream consumers can interpret it correctly based on context

Phase 2 - Integration Tests:

## New Test Files

Explorer Tests:
- apps/explorer/test/explorer/chain/import/runner/signet/orders_test.exs
  - Tests for Order import runner (insert, upsert, batch operations)
  - Tests composite primary key (transaction_hash, log_index)
  - Tests sweep data handling

- apps/explorer/test/explorer/chain/import/runner/signet/fills_test.exs
  - Tests for Fill import runner (rollup/host fills)
  - Tests composite primary key (chain_type, transaction_hash, log_index)
  - Tests same transaction on different chains

Indexer Tests:
- apps/indexer/test/indexer/fetcher/signet/orders_fetcher_test.exs
  - Full pipeline tests via Chain.import
  - ReorgHandler tests (rollup and host reorgs)
  - Db utility function tests
  - Factory integration tests

## Bug Fix

- apps/indexer/lib/indexer/fetcher/signet/utils/db.ex
  - Updated Db utility functions to use correct primary key fields
  - Replaced outputs_witness_hash references with transaction_hash
  - Added get_order_by_tx_and_log, get_orders_for_transaction
  - Added get_fill, get_fills_for_transaction

Closes ENG-1876 Phase 2
- Add missing 'import Ecto.Query' in orders_fetcher.ex (compilation fix)
- Add bounded l2_rpc_block_range for rollup eth_getLogs calls
- Wrap reorg cleanup operations in transaction for atomicity
- Add runtime.exs configuration for Signet OrdersFetcher
- Update module documentation with all config options

Addresses code review feedback from James on ENG-1876
@init4samwise
Copy link
Author

Code Review Feedback Addressed

Pushed commit dc3c60c with the following fixes:

Critical (compilation blockers)

  1. ✅ Added missing import Ecto.Query in orders_fetcher.ex (lines 403, 413 used from macro without import)
  2. ✅ Added bounded l2_rpc_block_range for rollup eth_getLogs calls (prevents unbounded queries)

High (runtime issues)

  1. ✅ Wrapped reorg cleanup operations in Repo.transaction for atomicity
  2. ✅ Added runtime.exs configuration for Signet OrdersFetcher with proper enabled key

Environment Variables Added

INDEXER_SIGNET_ORDERS_ENABLED
INDEXER_SIGNET_ROLLUP_ORDERS_ADDRESS
INDEXER_SIGNET_HOST_ORDERS_ADDRESS
INDEXER_SIGNET_L1_RPC
INDEXER_SIGNET_L1_RPC_BLOCK_RANGE
INDEXER_SIGNET_L2_RPC_BLOCK_RANGE
INDEXER_SIGNET_RECHECK_INTERVAL
INDEXER_SIGNET_START_BLOCK
INDEXER_SIGNET_FAILURE_THRESHOLD

Ready for re-review @prestwich

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants