Upgrading from v2.0.9, confused by changes

[BittermanAndy]

Thank you for the fantastic work you have done with this library.

I have just upgraded from version 2.0.9 to latest (3.0.4) and some of the breaking changes between versions are confusing me.

First and most importantly is that I had some code similar to this example in the documentation, but GetTextureData no longer exists. My old code looked like this:

var textureAsset = _instance.table.GetAssetInfo(texturePathId);
var assetTypeValueField = _manager.GetTypeInstance(_instance.file, textureAsset).GetBaseField();
var textureFile = TextureFile.ReadTextureFile(assetTypeValueField);
var textureData = textureFile?.GetTextureData(_instance);
// ...use textureData to create an image to show in my application

As near as I can figure out, the equivalent in the new version would be this (FillPictureData being the main change):

var textureAsset = _instance.file.GetAssetInfo(texturePathId);
var textureBase = _manager.GetBaseField(_instance, textureAsset);
var textureFile = TextureFile.ReadTextureFile(textureBase);
var textureData = textureFile?.FillPictureData(_instance);
// ...use textureData to create an image to show in my application

...but the textureData that comes back (and is filled in as textureFile.PictureData) is too small. For a 1024x1024 ARGB image, I am only given a 1048576 byte array - which is too small, there should be 1048576 pixels, but 4194304 bytes (4 bytes per pixel). I tried "hacking" the size by manually multiplying m_streamData.size *= 4 before calling FillPictureData, but then the contents of the returned array don't look right anyway (apparently random data); perhaps m_streamData.offset is wrong too?

Am I using FillPictureData correctly, as a replacement for the old GetPictureData? What else might I be doing wrong here? Or is this a bug?

Secondly (let me know if you want me to break this into a separate discussion), and less importantly: in the old version I could do this to find a sprite asset with the given idName (of type string):

var asset = _instance.table.GetAssetInfo(idName, AssetTypeIdSprite)

In the new version that overload is gone. Instead I am having to do this:

var spritesList = _instance.file.GetAssetsOfType(AssetTypeIdSprite);
var sprites = spritesList.Select(s => (_manager.GetBaseField(_instance, s)["m_Name"].AsString, s)).ToArray();

...and then manually search through sprites myself, looking for one with the matching idName. It seems to work, but it is much clunkier, and I'm not sure whether I'm missing something. Is there a better way?

[nesrak1]

GetTextureData no longer exists

I forgot to update the docs, sorry about that. The purpose of this refactor (which, as a user of it myself, I'm not super happy about the design either) was to move all of the texture reading and writing code away from UABEA and into AssetsTools.NET. In particular, I wanted support for both decompression in C# and compression from C# (for limited formats) and the rest from C++. I wanted as much code to be C# because that's more code that works cross platform without any changes. The C++ was included because many texture encoding libraries only exist in C++. To support all of that, to allow for more control over the process, and to more easily support png import/export, the texture library got a bunch of changes.

FillPictureData

Texture2D assets in Unity can store data in two ways. One is through a field called pictureData and the other is by pointing at a resS file and using a range of bytes. When pictureData is set, the data is encoded in the asset data, so there's nothing special we need to do. When it's in the resS file, FillPictureData copies the data from the resS file and into pictureData. It also returns the data it just read. If pictureData is already set, I think this method is a no-op.

However, the data this returns is still the compressed data. Depending on how you are "showing the image in the application", there are different options. DecodeTextureRaw returns raw RGBA32 data like before. DecodeTextureImage exports the image as one of the ImageExportType types. You can pass in textureFile.pictureData for the textureData. In retrospect, neither of those functions should take textureData as a parameter and should always default to using the pictureData field, but that's how it is right now.

---

in the old version I could [use one of GetAssetInfo's overloads] to find a sprite asset with the given idName

The explanation to this one is a bit complex, but the TLDR is that the old way was unreliable and hacky. It may have worked for some use cases, but it was not guaranteed to work in all use cases. I think UABE's AssetsTools version of this function was also not very reliable.

The old way this method worked was by checking if an asset matched a list of hardcoded class IDs and then reading the string at the start of the asset if it matched. There were two exceptions: MonoBehaviours don't store their name in their own asset most of the time, and GameObjects don't store their name at the beginning of the asset. These two assets had hardcoded handlers, but they only worked if their formats never changed (editor versions, custom version, etc. would break this method).

To make this more resilient to changes while still being performant, a few new features had to be added.

In the new version that overload is gone. Instead I am having to [use GetBaseField]

This is something I was trying to avoid. Technically this works, but it's slow because it has to deserialize all assets you want to look through. Yes, it's only sprites in this case, but in other cases you may want to look through assets of all types. You're wasting time by reading whole assets to get the name at the very start.

Recently, a new class was added for partial reading, AssetTypeValueIterator. It doesn't allocate memory for each field it reads, it just passes you field names and you can read it if you want or skip it. This is useful for name reading because you can keeping requesting fields until you reach m_Name, read the string, then stop reading. Of course, this is still a little slower than reading the name field at the beginning, but better than nothing.

In UABEANext, I recently added support for using the iterator to read names. The code is here. It supports multithreaded reading, checking if GameObjects or MonoBehaviours meet the format we're expecting and using a more optimal version instead, reading names through replaced assets, and length limiting in case the file is corrupt or custom. Then plan is for this code to move to AssetsTools.NET, but that hasn't happened yet.

If you need it right now, you could adapt this code to plain AssetsTools.NET by replacing Workspace with a regular AssetsManager (the code only uses _workspace.Manager anyways) and AssetInst with AssetsFileInfo and the AssetsFileInstance that info belongs to.

If you only care about reading sprite assets and you know the format isn't changing (or you're targeting a specific game), you can use the old code which goes something like this (untested):

lock (fileInst.LockReader)
{
    var reader = fileInst.file.Reader;
    reader.Position = info.GetAbsoluteByteOffset(fileInst.file);
    return reader.ReadCountStringInt32();
}

[BittermanAndy]

Superb, thanks so much for such a rapid and thorough response. I'll try out these suggestions this weekend and let you know how I get on.

[BittermanAndy]

Good news! For the first difficulty, DecodeTextureRaw gave me exactly what I was looking for and is now working (I can see that DecodeTextureImage would be cleaner, but it would be a bigger change in my code than I wanted to make right now).

For the second part, thanks for the explanation. I've decided to leave my code as it is for now because it is "working", just clunky; I might come back and make it prettier / more efficient later... in my scenario it might mean my application load time is fractionally slower than it might otherwise have been, but it's "good enough", so again I will minimise changes.

Thank you again for the super fast and helpful response, I really appreciate your efforts.