feat: Add Signet Orders Indexer for cross-chain order tracking#2
feat: Add Signet Orders Indexer for cross-chain order tracking#2init4samwise wants to merge 10 commits intomasterfrom
Conversation
Implements indexer fetcher for Signet order tracking: ## New Files Database Schema: - apps/explorer/lib/explorer/chain/signet/order.ex - apps/explorer/lib/explorer/chain/signet/fill.ex Migration: - apps/explorer/priv/signet/migrations/20260216040000_create_signet_tables.exs Import Runners: - apps/explorer/lib/explorer/chain/import/runner/signet/orders.ex - apps/explorer/lib/explorer/chain/import/runner/signet/fills.ex Fetcher: - apps/indexer/lib/indexer/fetcher/signet/orders_fetcher.ex - apps/indexer/lib/indexer/fetcher/signet/event_parser.ex - apps/indexer/lib/indexer/fetcher/signet/reorg_handler.ex - apps/indexer/lib/indexer/fetcher/signet/utils/db.ex - apps/indexer/lib/indexer/fetcher/signet/orders_fetcher/supervisor.ex ## Features - Parse Order, Filled, Sweep events from RollupOrders contract - Parse Filled events from HostOrders contract - Compute outputs_witness_hash for cross-chain correlation - Insert into signet_orders / signet_fills tables - Handle chain reorgs gracefully - Add metrics/logging for indexer health ## Modified Files - apps/explorer/lib/explorer/chain/import/stage/chain_type_specific.ex - Added signet runners to chain type specific import stage - apps/indexer/lib/indexer/supervisor.ex - Added SignetOrdersFetcher to supervisor Closes ENG-1894
prestwich
left a comment
There was a problem hiding this comment.
@init4samwise does this PR account for the fact that Output chain id semantics are different in orders and fills?
in an order the chain_id is the TARGET chain, i.e. where assets should be delivered
in a fill the chain_id is the ORIGINATING chain
|
@prestwich Good question - I need to clarify the chain ID semantics. Looking at the current implementation:
Questions:
Want to make sure I understand the contract semantics correctly before adjusting. |
|
|
Thanks for catching this @prestwich! You're right - I reviewed against the signet-sdk ABI and found:
I've posted a detailed breakdown to #eng. Will need to fix:
Want me to push a fix, or would you prefer to take this? |
- Add tools/signet-sdk to extract ABIs from npm package - Store extracted ABIs in apps/explorer/priv/contracts_abi/signet/ - Create Indexer.Fetcher.Signet.Abi module for event topic computation - Update EventParser to use correct SDK-defined event signatures - Fix Output struct decoding: (token, amount, recipient, chainId) - Add chainId to formatted JSON outputs for cross-chain tracking Key changes: - Event signatures now match @signet-sh/sdk v0.3.0 - Output struct properly decoded with all 4 fields - Sweep event correctly parses indexed topics for recipient/token To update ABIs when SDK changes: cd tools/signet-sdk && npm install && npm run extract
- Fix Output struct documentation to match @signet-sh/sdk: (token, amount, recipient, chainId) not (recipient, token, amount) - Add comprehensive tests for Abi module: - Event topic hash computation and consistency - Event signatures format validation - ABI file loading from priv directory - Add comprehensive tests for EventParser module: - outputs_witness_hash determinism and correctness - Output field order verification (critical for cross-chain correlation) - Log parsing edge cases - Event topic matching with Abi module Part of Phase 1 completion for ENG-1876
Per James's architecture update: orders and fills cannot be correlated directly - only block-level coordination is possible. Changes: - Remove compute_outputs_witness_hash from EventParser - Change Order primary key to (transaction_hash, log_index) - Change Fill primary key to (chain_type, transaction_hash, log_index) - Update migration to use new composite primary keys - Update import runners for new primary keys - Remove correlation tests from EventParserTest - Update module docs to reflect independent indexing
Per James's request - use exact version 0.4.4 instead of ^0.3.0
…semantics - Remove outdated outputs_witness_hash correlation documentation - Update database tables to show composite primary keys - Add chainId semantics explanation (destination in orders, origin in fills) - Clarify that orders and fills are indexed independently Per architecture update from James
|
@prestwich Update on your review comments: I've addressed the Output struct and chainId issues:
The Is there anything else you'd like me to address before this can be merged? |
|
@prestwich Friendly follow-up - let me know if there's anything else you'd like addressed on this PR. Happy to make any additional changes. In the meantime, I'll start working on Phase 2 (integration testing) in parallel so we're ready to move quickly once this is approved. |
Phase 2 progress for ENG-1876: Added factory entries for test data generation. - signet_order_factory: generates Order structs with proper JSON fields - signet_fill_factory: generates Fill structs with chain_type and outputs These factories enable integration testing of the signet indexer.
|
[Claude Code] PR Review: feat: Add Signet Orders IndexerOverall this is a solid foundation for cross-chain order/fill indexing. The structure follows Blockscout's patterns well (BufferedTask, import runners, Ecto schemas). However, there are several issues that need to be addressed before merge — including a couple that will prevent compilation. Critical (won't compile/run)1. Missing
Fix: Add 2. Every query function in this module ( Fix: Either remove High (incorrect behavior at runtime)3. Unbounded In Fix: Apply a similar 4.
Fix: Either integrate with Blockscout's existing reorg detection (subscribe to reorg events) or remove the reorg handler to avoid giving a false sense of safety. 5. The Fix: Add 6. {block, ""} = Integer.parse(String.trim_leading(hex_block, "0x"), 16)If the response isn't a clean hex string (e.g. Medium (design/correctness concerns)7. Config key mismatch — fetcher may never start
Fix: Ensure the config key matches what 8. The rollup chain can emit both Fix: Query both 9. No transaction wrapping in reorg cleanup
Fix: Wrap in 10. Sweep-to-Order association uses only the first sweep per transaction case Map.get(sweeps_by_tx, order.transaction_hash) do
[sweep | _] -> ...
If a transaction emits multiple Order + Sweep events, the first sweep is applied to all orders from that transaction. The other sweeps are silently dropped. 11. ABI offset arithmetic in The manual Low12. 13. Multiple files missing trailing newline — The diff shows 14. PR description says "TODO: Add integration tests" — Worth tracking this as a follow-up. What looks good
|
|
[Claude Code] Follow-up: Test Quality GapsA few additional observations on the test coverage in this PR. 1. No tests for actual ABI-encoded event decodingThe core functionality of At minimum, there should be a happy-path test for each event type that constructs a valid ABI-encoded log and verifies the parser produces the correct output. These can be generated using the 2. Several tests are tautologies
This test creates an Elixir tuple, destructures it, and asserts the destructured values match the originals. It tests Elixir's tuple semantics, not
These tests create a list of atoms and assert atoms are in the list. They never invoke 3. ABI topic hash tests don't assert known-good values
The tests verify that topic hashes are 66 characters long and hex-formatted, but never assert the actual expected keccak256 values. If the event signature string had a typo (e.g., wrong parameter order or type), all tests would still pass. At least one test per event should assert the exact expected topic hash, cross-referenced with the SDK or computed independently: # Example:
assert Abi.order_event_topic() == "0x<expected_keccak256_hash>"4. Sweep-to-order association is untested
5. No tests for
|
Per PR review feedback from @prestwich: - In Order events: chainId = destination chain (where assets should be delivered) - In Filled events: chainId = origin chain (where the order was created) Added inline documentation to the @TypeDoc in both schema files to make this semantic difference explicit for future developers.
|
@prestwich Addressed your question about chainId semantics (commit b1a0ef6): I've added explicit documentation to the schema files clarifying the semantic difference: order.ex -
fill.ex -
This supplements the existing "chainId Semantics" section in the README.md which already explained this at a higher level. The semantic difference is now documented at three levels:
|
Phase 2 - Integration Tests: ## New Test Files Explorer Tests: - apps/explorer/test/explorer/chain/import/runner/signet/orders_test.exs - Tests for Order import runner (insert, upsert, batch operations) - Tests composite primary key (transaction_hash, log_index) - Tests sweep data handling - apps/explorer/test/explorer/chain/import/runner/signet/fills_test.exs - Tests for Fill import runner (rollup/host fills) - Tests composite primary key (chain_type, transaction_hash, log_index) - Tests same transaction on different chains Indexer Tests: - apps/indexer/test/indexer/fetcher/signet/orders_fetcher_test.exs - Full pipeline tests via Chain.import - ReorgHandler tests (rollup and host reorgs) - Db utility function tests - Factory integration tests ## Bug Fix - apps/indexer/lib/indexer/fetcher/signet/utils/db.ex - Updated Db utility functions to use correct primary key fields - Replaced outputs_witness_hash references with transaction_hash - Added get_order_by_tx_and_log, get_orders_for_transaction - Added get_fill, get_fills_for_transaction Closes ENG-1876 Phase 2
- Add missing 'import Ecto.Query' in orders_fetcher.ex (compilation fix) - Add bounded l2_rpc_block_range for rollup eth_getLogs calls - Wrap reorg cleanup operations in transaction for atomicity - Add runtime.exs configuration for Signet OrdersFetcher - Update module documentation with all config options Addresses code review feedback from James on ENG-1876
Code Review Feedback AddressedPushed commit dc3c60c with the following fixes: Critical (compilation blockers)
High (runtime issues)
Environment Variables AddedReady for re-review @prestwich |
Summary
Implements the Signet Orders Fetcher for ENG-1894 as part of the broader Signet order tracking project (ENG-1876).
Changes
New Database Tables
Indexer Implementation
Key Features
Note: Orders and fills are indexed independently. Only block-level coordination is possible — direct order-to-fill correlation is not supported.
Configuration
Files Added
apps/explorer/lib/explorer/chain/signet/order.exapps/explorer/lib/explorer/chain/signet/fill.exapps/explorer/lib/explorer/chain/import/runner/signet/orders.exapps/explorer/lib/explorer/chain/import/runner/signet/fills.exapps/explorer/priv/signet/migrations/20260216040000_create_signet_tables.exsapps/indexer/lib/indexer/fetcher/signet/orders_fetcher.exapps/indexer/lib/indexer/fetcher/signet/event_parser.exapps/indexer/lib/indexer/fetcher/signet/reorg_handler.exapps/indexer/lib/indexer/fetcher/signet/utils/db.exapps/indexer/lib/indexer/fetcher/signet/orders_fetcher/supervisor.exapps/indexer/lib/indexer/fetcher/signet/README.mdTesting
TODO: Add integration tests in signet-test-utils
Closes ENG-1894