Revise Serialized Model Structure for Standardization

[Gnpd]

Review and refine the structure of the serialized model to align with possible existing serialization standards. The current format includes fields such as estimator_class, attributes, params,... and metadata like producer_version. Explore established standards (e.g., ONNX, PMML) to ensure compatibility and extensibility. Propose a revised structure that balances clarity, completeness, and interoperability.

Current format

serialized_estimator = {
    "estimator_class": model.__class__.__name__,  #str
    "attributes": serializable_attributes, #dict  loadings: [1,2,3]
    "attribute_types": attribute_types, #dict ndarray
    "attribute_dtypes": attribute_dtypes, #dict float64
    "params": self.convert_to_serializable(params), #dict
    "param_types": param_types, #dict
    "param_dtypes": param_dtypes, #dict
    "producer_version": getattr(model, "_sklearn_version", None),
    "producer_name": model.__module__.split(".")[0],
    "domain": "sklearn",
}

[paucablop]

@Gnpd I have been playing with it over the last few days, and I think the format is very nice. I am a bit uncertain about the difference between the producer_name and the domain. Is the domain the general framework (e.g., sklearn) and the producer_name any library that implements the same API as the domain (e.g., could be sklearn itself but also chemotools?

[Gnpd]

You're right, the idea is that producer_name refers to the specific library or package the model comes from (for example, sklearn for pure scikit-learn estimators, or chemotools or any other compatible library).

domain always refers to the general framework or API that the model follows, here it’s always sklearn, since we’re using the SklearnSerializer.
From the latest update, we have:

"producer_version": sklearn.__version__,  # instead of getattr(model, "_sklearn_version", None)

but not sure what is best...

I also realized that we’re not currently covering the version of the producer_name, which might be something to include later.

But this is still under discussion, so it’s good to have different perspectives before making a final decision.