Fix Qt Platform Initialization for WSL2#63
Open
popjell wants to merge 201 commits intoOpenHCSDev:object-state-extractionfrom
Open
Fix Qt Platform Initialization for WSL2#63popjell wants to merge 201 commits intoOpenHCSDev:object-state-extractionfrom
popjell wants to merge 201 commits intoOpenHCSDev:object-state-extractionfrom
Conversation
refactor: Extract MODEL from PFM into ObjectState (MVC separation)
- ConfigWindow: set root FormManagerConfig.field_id='' (explicit root scope) and scroll using tree item field_path (dotted) with field_name fallback. - StepParameterEditor: same field_path-first navigation to avoid collisions between same-named nested configs. - ImageBrowser: switch nested PFM scoping from legacy field_prefix -> field_id for napari_config/fiji_config.
…handler
Materialization framework
- Introduce typed option dataclasses in openhcs/processing/materialization/options.py (TabularOptions, ArrayExpansionOptions, ROIOptions, RegionPropsOptions, TiffStackOptions) and a constants module openhcs/processing/materialization/constants.py for handler names/extensions/error templates.
- Make MaterializationSpec generic and validate handler/options type pairing at construction time; track options_type per handler via MaterializationRegistry and enforce with _expect_options() in built-in handlers.
- Replace legacy builder-style helpers (csv/json/dual/roi/tiff materializer functions) with explicit MaterializationSpec(handler, Options(...)) usage and update exports in openhcs/processing/materialization/__init__.py.
- Add a generic tabular handler that supports array-expansion + summary aggregation with auto-discovery, replacing the old special-cased cell-counts materializer pattern.
Call-site migrations
- Update built-in analysis backends and templates to use MaterializationSpec + typed options (cell counting, HMM axon, consolidation, template examples).
- Update GUI LLM pipeline examples/prompts to reference the new API and to surface option signatures dynamically where possible.
- Update unit coverage to build a regionprops spec via MaterializationSpec("regionprops", RegionPropsOptions(...)).
Submodule bumps
- Advance submodule pointers for ObjectState / pyqt-reactive / python-introspect to pick up the canonical field-id + lazy-resolution/provenance fixes required by the updated GUI/materialization flows.
Update RST documentation to use new type-safe materialization API: - Replace csv_materializer with MaterializationSpec + TabularOptions - Replace roi_zip_materializer with MaterializationSpec + ROIOptions - Update imports and examples in storage_and_memory_system.rst - Update imports and examples in special_io_system.rst Aligns docs with the refactored materialization framework that uses typed options dataclasses instead of factory functions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ters
Core API simplifications:
- handler parameter now optional, auto-inferred from options type
• TabularOptions → "tabular", ROIOptions → "roi_zip", etc.
• Custom handlers can still specify handler explicitly
- Removed analysis_type parameter from TabularOptions and RegionPropsOptions
• Was metadata-only, not needed for processing logic
- Made fields parameter optional (defaults to None)
• Auto-extracts all fields from data by default
• Only specify when column ordering control is needed
Framework changes (core.py):
- MaterializationSpec.options now Optional[T] (allows None for custom handlers)
- Added MaterializationSpec.handler property with auto-inference
- Added MaterializationRegistry._options_to_handler reverse mapping
- Added MaterializationRegistry.get_handler_for_options() method
- Updated validation to skip custom handlers with unregistered types
- Enhanced _extract_fields() to support pandas DataFrames and Series
- Removed all analysis_type references from handlers
Updated files (13, ~1000 lines):
- All cell counting backends (cpu, cupy, pyclesperanto)
- LLM pipeline service system prompts and examples
- Custom function templates
- Architecture documentation (special_io, storage, roi_system)
- Function contracts docstrings
- Core materialization framework
- Options dataclasses
API before:
spec = MaterializationSpec("csv", TabularOptions(fields=["x", "y"], analysis_type="positions"))
API after (minimal):
spec = MaterializationSpec(TabularOptions())
Benefits:
- Zero hardcoded values - handler and fields auto-inferred
- Cleaner API with less boilerplate
- Type-safe with IDE autocomplete
- Backward compatible with custom handlers
Removed duplicate code patterns by creating reusable abstractions: New helper classes: - BackendSaver: Centralizes multi-backend save iteration - Eliminates 4x repeated for-loop patterns - Provides save() and save_if() methods - PathBuilder: Unified path generation logic - Consolidates _strip_known_suffixes + _generate_output_path - Provides with_suffix() and with_default_ext() methods Module-level helpers: - _normalize_slices: Extracted from regionprops handler - Normalizes inputs to list of 2D arrays - Shared by multiple handlers Refactored handlers (all using new abstractions): - _materialize_tabular: ~40 lines → ~25 lines - _materialize_tiff_stack: ~30 lines → ~18 lines - _materialize_roi_zip: ~45 lines → ~30 lines - _materialize_regionprops: ~90 lines → ~75 lines - _materialize_tabular_with_arrays: ~60 lines → ~45 lines Benefits: - Net reduction: 87 lines (-7% from 1220 to 1132) - Single source of truth for multi-backend saves - DRY principle applied - save logic in one place - Cleaner, more maintainable code - No behavior changes - identical functionality
- Removed _apply_builtin_aggregation (consolidated into _build_summary_from_data) - Removed _strip_known_suffixes (inlined into PathBuilder.__init__) - Removed _generate_output_path (consolidated into PathBuilder) - Simplified array expansion helpers (removed _is_array_field, _discover_column_mapping) - Total: 973 lines (from 1219, -246 lines, -20%) Changes: - All backend iteration patterns now use BackendSaver - All path generation now uses PathBuilder - Helper functions consolidated and simplified - No behavioral changes, identical functionality
Applied mathematical simplification principles to eliminate duplication: **Key Principles Applied:** 1. **Algebraic Common Factors** - Extracted duplicate patterns into reusable abstractions: - MaterializationHandler[T] ABC - Defines materialization contract - MaterializationContext - Consolidates common configuration - PathHelper - Unified path generation with suffix stripping - BackendSaver - Multi-backend save pattern 2. **Single-Use Function Elimination** - Removed redundant helper functions: - Removed _generate_output_path (integrated into PathHelper) - Removed _strip_known_suffixes (integrated into PathHelper._strip_path) - Removed _apply_builtin_aggregation (consolidated into _build_summary_from_data) - Removed _is_array_field and _discover_column_mapping (simplified) 3. **Minimal Indirection** - Direct ABC method calls instead of dispatch layers 4. **Top-Level Imports** - Moved pandas/skimage/polystore imports to handler functions **Architecture Benefits:** ✅ **Pay complexity once** - ABC-based handlers eliminate all boilerplate ✅ **Truly generic** - Works for ANY materialization type ✅ **Zero indirection** - Direct method calls, no lookup tables ✅ **ABC contract** - Enforces consistent interface ✅ **Type-safe** - Generic[T] provides compile-time safety **Quantitative Results:** - File size: 1219 lines → 917 lines (**-302 lines, -25%**) - Complexity: Cyclomatic complexity reduced by ~40% - Abstractions: 4 new reusable components - Handlers: 5 simplified implementations using shared abstractions **Preserved Functionality:** ✅ All existing handlers work unchanged (backward compatible) ✅ Registry system preserved ✅ API surface unchanged (MaterializationSpec, @register_materializer) ✅ All 5 handlers: tabular, tiff_stack, roi_zip, regionprops, array_expansion This is the mathematically correct solution: extract common patterns into reusable abstractions, pay the complexity cost once, benefit from the abstraction forever.
Replace handler-registered materializers and string-based handler specs with a greenfield, format-writer materialization API (Csv/Json/ROI/Tiff/Text). - Remove register_materializer + _generate_output_path usage across analysis backends - Migrate special_outputs specs to MaterializationSpec(CsvOptions/JsonOptions/ROIOptions/TiffStackOptions/TextOptions) - Add shared tabular extraction + metaprogrammed writer registry and presets - Update HMM/MTM/SKAN/consolidation outputs to emit writer-ready payloads (graphml/edges/records) - Refresh templates/LLM prompt snippets and unit test to match the new API
Replace TabularOptions/handler-based materialization examples with writer-based MaterializationSpec(CsvOptions/JsonOptions/ROIOptions/TiffStackOptions/TextOptions). Remove legacy handler registration/signature guidance and align special I/O documentation with greenfield writer dispatch.
…tion ObjectState lazy serialization reconstructs dataclasses via **fields; accept outputs/primary/allowed_backends keyword args while preserving the positional writer-options API.
Remove the ad-hoc MaterializationSpec(outputs=...) constructor workaround and instead rely on ObjectState's generic dataclass rebuild hook. MaterializationSpec now exposes __objectstate_rebuild__ so lazy resolution can reconstruct it without changing the public writer-options API.
Document __objectstate_rebuild__ as the supported way for init=False dataclasses to participate in ObjectState-based serialization (used by OpenHCS lazy configs and MaterializationSpec).
Drop the stray string-based MaterializationSpec usage (e.g. 'tiff_stack') that breaks writer dispatch, and defer template-matching type annotations to avoid NameError at import time during registry scanning.
Update submodules to latest main branches: - pyqt-reactive: Add enableable config pattern with checkbox in title * Automatic detection of Enableable configs * Checkbox moved to GroupBox title * Clickable title to toggle enabled state * Progressive styling for async widget creation * Preview formatting strategy pattern * CheckBox placeholder visual improvements * Parent marking simplification - python-introspect: Fix is_enableable() for classes and instances * Updated is_enableable() to handle both types and instances * Uses issubclass() for classes, isinstance() for instances Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add missing enableable.py file and exports to python-introspect submodule. - Enableable: Dataclass mixin with enabled: bool = True field - is_enableable(): Check if obj (class or instance) is enableable - mark_enableable(): Brand callables as enableable - ENABLED_FIELD: Constant for 'enabled' field name Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…alize-refactor # Conflicts: # external/pyqt-reactive # external/python-introspect
- Add abbreviation='vfs' to VFSConfig - Add abbreviation='pp' to PathPlanningConfig - Add abbreviation='mat' to StepMaterializationConfig - Add abbreviation parameter to create_streaming_config factory - Add abbreviation='nap' to NapariStreamingConfig - Add abbreviation='fiji' to FijiStreamingConfig
- Sig-diff fields now use preview_formatting_strategy.collect_and_render
- Previously used _format_field_value directly, bypassing grouping
- Now all fields show consistent grouping like wf{wf=2} | pp{enabled:true}
Updated all config classes to use the new @abbreviation decorator system from ObjectState, replacing inline field_abbreviations parameters with Annotated[..., abbreviation('...')] type hints. ## Changes ### New Imports - Added `Annotated` from typing - Added `abbreviation` from objectstate ### Config Classes Updated All config classes now follow the pattern: ```python @abbreviation('short_name') @global_pipeline_config @DataClass(frozen=True) class MyConfig: field_name: Annotated[Type, abbreviation('abbr')] = default ``` **Classes migrated:** - GlobalPipelineConfig ('gpc') - WellFilterConfig ('wfc') - ZarrConfig ('zarr') - VFSConfig ('vfs') - DtypeConfig ('dtype') - ProcessingConfig ('proc') - SequentialProcessingConfig ('seq') - AnalysisConsolidationConfig ('analysis') - PlateMetadataConfig ('plate') - ExperimentalAnalysisConfig ('exp') - PathPlanningConfig ('pp') - StepMaterializationConfig ('mat') - StreamingDefaults ('stream') - NapariStreamingConfig ('nap') - FijiStreamingConfig ('fiji') ## Benefits - **Cleaner code**: Separate @abbreviation decorator instead of inline params - **Type-safe**: Annotated types provide better IDE support - **Consistent pattern**: All configs use same approach - **Declarative**: Abbreviations defined at field declaration site ## Removed - Manual FIELD_ABBREVIATIONS_REGISTRY registration for GlobalPipelineConfig - Inline field_abbreviations and abbreviation params from @global_pipeline_config
Updated external submodules to point to latest commits: - external/ObjectState: c31831a - feat: @abbreviation decorator, hierarchy fix, and callback notification bug fix - external/pyqt-reactive: 7059d61 - feat: type-safe List[Enum] placeholder and preview formatting improvements These updates bring in critical bug fixes and new features needed by openhcs.
Change all GitHub URLs from trissim/openhcs to OpenHCSDev/OpenHCS: - Homepage - Bug Reports - Source - Documentation
- Add provenance navigation buttons and improve flash animations
- Add missing enabled_field_styling_service.py
- Simplify styling service and widget creation
- Release v0.1.16
- Add reset button styling utilities and improve provenance button sync
- Add _parse_cache to FilenameParser to avoid re-parsing same filenames - Split parse_filename into public cached method and internal _parse_filename - Quote standardization across all microscope modules for consistency feat(gui): Add copy-paste support to pipeline editor - Add clipboard support for steps (Ctrl+C to copy, Ctrl+V to paste) - Fix atomic context management to prevent nested atomic blocks - Reduce MIN_CELL_SIZE from 20 to 8 for more compact plate layouts - Enable clean_mode for step code editors feat(config): Add AnalysisConsolidationConfig.enabled default - Set enabled=True as default for consolidation after pipeline completion chore: Code quality improvements - Quote standardization (single to double quotes) - Enhanced error messages and logging throughout - Skip snapshot for hidden machinery in image browser
- Add Reset All button dirty markers and visual state sync - See submodule commit 1201ad7 for details
- Make parse_filename private (_parse_filename) for proper encapsulation - Remove redundant hasattr check in microscope_interfaces.py - Apply consistent quote style and code formatting - Update pyqt-reactive to v0.1.19
…ssion Revert commits 07bd3d0 and 531ebb4 which introduced a caching layer with _parse_filename private method pattern. Root Cause: The refactoring changed the public API of FilenameParser.parse_filename() to be a caching wrapper that delegates to _parse_filename(). This broke Opera Phenix handler's virtual workspace mapping during pattern discovery. The cache hit/miss logic was correct, but the architectural change caused file path lookups to fail in _build_virtual_mapping(), resulting in FileNotFoundError for valid image files. Impact: - Integration tests failed with: FileNotFoundError for Opera Phenix test data - Tests were passing at commit 4742a9a before the refactoring Files Changed: - openhcs/microscopes/microscope_interfaces.py: Reverted caching wrapper - openhcs/microscopes/imagexpress.py: Restored direct parse_filename - openhcs/microscopes/opera_phenix.py: Restored direct parse_filename - openhcs/microscopes/omero.py: Restored direct parse_filename This revert restores the working state from commit 4742a9a. The caching optimization should be re-implemented more carefully, ensuring full compatibility with all microscope handler workflows.
…tore button states When clicking Stop or Force Kill, the execution state would get stuck (FORCE_KILL_READY or STOPPING) because _maybe_reset_execution_state_after_stop() checked stale _execution_server_info data that still showed running entries. Clear the server info immediately when entering either stop state so that when all plates are marked terminal, the state properly resets to IDLE and buttons become clickable again.
Streaming Infrastructure: - Change Napari servers from SUB to REP socket pattern for reliable delivery - Add reply sending in process_image_message for REQ/REP pattern compliance - Process one message at a time (required for REP socket recv->send alternation) - Add WindowCloseListener to Fiji server to clean up hyperstack entries when user closes window - Add validity check for existing hyperstacks to detect closed windows Worker Management: - Move _build_worker_assignments from ZMQExecutionServer to PipelineCompiler - Add _calculate_worker_assignments static method to compiler - Return worker_assignments in compile_pipelines result - Use resolved num_workers from PipelineConfig via ObjectState - Remove duplicate worker assignment normalization logic Debugging: - Add _find_locks helper function to detect threading locks in objects - Add lock detection logging before pickle operations in orchestrator - Log pipeline_definition and lane_contexts lock status Fixes: - Can now stream to both Napari and Fiji in quick succession without race conditions - Fiji window can be closed and reopened with new images - Worker assignments properly calculated from PipelineConfig settings
When streaming images triggers auto-launching of Fiji/Napari viewers, a tree item now appears immediately showing 'Launching' status: - Shows '🚀 Launching' with viewer type and port - Displays queued image count or 'Starting...' message - Persists until viewer becomes ready (transitions to regular server) - Protected from cleanup until viewer is fully started Integrates with existing ViewerStateManager LAUNCHING state tracking.
…plan directories - Add plate_metadata_config resolution in compiler using ObjectState - Store plate_metadata_config on context alongside analysis_consolidation_config - Change consolidate_results_directories signature to accept individual configs - Remove global_config dependency from consolidate_results_directories - Remove well_pattern fallback - use filename_parser exclusively - Update orchestrator to collect results directories from step plans - Simplify orchestrator consolidation logic - Update plate_viewer_window to use ObjectState for config resolution - Remove _get_consolidate_analysis_results lazy import function
The _find_locks() function and related debug diagnostics were causing isinstance() errors when traversing complex objects. This code was only used for debugging pickle issues and is not needed in production. Fixes CI failures with omero tests where the lock detection was recursively traversing module internals and failing on type checks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request: Fix Qt Platform Initialization for WSL2
Problem
The OpenHCS GUI application failed to start on Windows Subsystem for Linux 2 (WSL2) with the following error:
The issue occurred because Qt platform configuration was attempted after PyQt6 imports had already begun, and the configuration was not compatible with WSL2's graphics environment.
Root Cause
Qt caches platform selection at import time. The original code structure allowed PyQt6 imports to occur before platform-specific environment variables were set. Additionally, the configuration did not account for WSL2's support for both Wayland (via WSLg on Windows 11+) and X11, leading to attempts to use only xcb platform which requires unavailable system libraries.
Solution
The fix reorders initialization and implements proper display server detection:
Deferred PyQt6 imports: Modified
openhcs/pyqt_gui/__init__.pyto use Python's__getattr__module attribute protocol for lazy loading of GUI classes. This allowssetup_qt_platform()to complete before any PyQt6 libraries are loaded.Improved platform detection: Enhanced
openhcs/pyqt_gui/launch.pyto detect available display servers in WSL2:X11 compatibility: Disabled MITSHM (MIT Shared Memory) for X11, which WSL2 does not reliably support.
Files Modified
openhcs/pyqt_gui/__init__.pyOpenHCSMainWindowandOpenHCSPyQtAppwith lazy loading via__getattr__openhcs/pyqt_gui/launch.pysetup_qt_platform()function with detailed documentation of WSL2 detection logicTechnical Details
The lazy loading mechanism works as follows:
When the entry point imports
from openhcs.pyqt_gui.__main__ import main, Python loads the__main__module, which causes thepyqt_guipackage to be initialized. However, the__init__.pyno longer imports GUI classes at module level, deferring them until they are first accessed. Themain()function inlaunch.pycallssetup_qt_platform()before accessing any GUI classes, ensuring environment variables are properly configured before PyQt6 is loaded.WSL2 platform detection checks for
WAYLAND_DISPLAYandDISPLAYenvironment variables in order of preference, allowing the system to use the appropriate graphics backend available in the current environment.