Skip to content

allow passing pytables options on load#40

Open
m4ce wants to merge 4 commits intouchicago-cs:masterfrom
m4ce:feat/tables_opts
Open

allow passing pytables options on load#40
m4ce wants to merge 4 commits intouchicago-cs:masterfrom
m4ce:feat/tables_opts

Conversation

@m4ce
Copy link

@m4ce m4ce commented Nov 19, 2019

This is useful for those cases where one wants to load things directly from memory (e.g. BytesIO) without having to write it out to a file first and load it from there.

bytes_io = io.BytesIO()
dd.io.load("in-memory-sample.h5", driver="H5FD_CORE", driver_core_image=bytes_io.read(), driver_core_backing_store=0)

@coveralls
Copy link

coveralls commented Nov 19, 2019

Coverage Status

Coverage decreased (-0.6%) to 72.143% when pulling d45d90e on m4ce:feat/tables_opts into 01af936 on uchicago-cs:master.

@m4ce
Copy link
Author

m4ce commented Nov 19, 2019

Alternatively, load can take a path or a buffer. The signature could be changed to something like:

def load(path_or_buf, group=None, sel=None, unpack=False):

and then internally check whether we are dealing with a string path or a buffer and call tables.open_file with the right params?

@gustavla
Copy link
Member

Yes, this would be useful! I think I like the path_or_buf option better. If not, perhaps an option that's called pytables_kwargs. By adding *args, **kwargs, it can swallow any typo without complaining (well, pytables might complain, but that's a bit out of our hands).

@m4ce
Copy link
Author

m4ce commented Nov 20, 2019

@gustavla, I made the changes and updated the docstring. I opted for pytables_kwargs for simplicity. I am not really sure why the tests fail?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants