KMS client encrypt with plaintext blob as Uint8Array causes SerializationException

[GWellerGMSL]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

The typings for the SDK v3 now only accept Uint8Array | undefined for blobs, but these params aren't being serialized to a base64 string as expected.

Specifically, I'm trying to call encrypt on KMS. The encrypt command accepts a parameter Plaintext which is of type blob...

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/kms/command/EncryptCommand/

The SDK client code accepts a Uint8Array for the Plaintext param and seems to sending it unaltered as a JSON object to AWS, looking like this...

 {'type': 'Buffer', 'data': [100, 71, 104, 112, 99, 121, 66, 112, 99, 121, 66, 104, 98, 105, 66, 49, 98, 109, 86, 117, 89, 50, 57, 107, 9, ... ] }

AWS rejects this, of course, as its expecting a string.

I don't honestly know if this is a problem with the typings and the documentation, if the base64 marshalling isn't working internally, or if the Encrypt endpoint in AWS is supposed to accept a buffer as a JSON object and just isn't working. I get the same error running against localstack FWIW.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/client-kms@3.966.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v22.20.0

Reproduction Steps

In a node project with the latest client-kms package...

  "dependencies": {
    "@aws-sdk/client-kms": "^3.966.0",
    ...

Call encrypt as per the documentation

  const client = new KMS({region: "eu-central-1"});
  await client.encrypt({
    KeyId: keyIdAlias,
    Plaintext: Buffer.from(data),
    // or Plaintext: new TextEncoder().encode(data),
  });

Observed Behavior

An HTTP error response and the following error...

SerializationException: Start of structure or map found where not expected.
    at ProtocolLib.getErrorSchemaOrThrowBaseException ((...)\node_modules\@aws-sdk\client-kms\node_modules\@aws-sdk\core\dist-cjs\submodules\protocols\index.js:69:67)
    at AwsJson1_1Protocol.handleError ((...)\node_modules\@aws-sdk\core\dist-cjs\submodules\protocols\index.js:774:65)
    at AwsJson1_1Protocol.deserializeResponse ((...)\node_modules\@aws-sdk\client-kms\node_modules\@smithy\core\dist-cjs\submodules\protocols\index.js:476:24)
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async (...)\node_modules\@aws-sdk\client-kms\node_modules\@smithy\core\dist-cjs\submodules\schema\index.js:26:24
    at async (...)\node_modules\@aws-sdk\client-kms\node_modules\@smithy\core\dist-cjs\index.js:121:20
    at async (...)\node_modules\@aws-sdk\client-kms\node_modules\@smithy\middleware-retry\dist-cjs\index.js:254:46
    at async (...)\src\lib\secrets.ts:11:20
    at async (...)\node_modules\@aws-sdk\client-kms\node_modules\@aws-sdk\middleware-logger\dist-cjs\index.js:5:26
    at async encryptData ((...)\src\lib\secrets.ts:29:30)
    at async Object.<anonymous> ((...)\src\lib\kms.test.ts:10:24)

The documentation for the encrypt command does even list SerializationException so I'm guessing the error comes from some initial deserialization step.

Expected Behavior

For the Plaintext param to be accepted.

Possible Solution

Ignoring the typing and explicitly encoding the blob param as a base 64 string works...

Using Uint8Array.from() seems to be compatible with the actual endpoint in AWS...

  const client = new KMS({region: "eu-central-1"});
  await client.encrypt({
    KeyId: keyIdAlias,
    Plaintext: Uint8Array.from(data, (v) => v.charCodeAt(0))),
  });

It will also accept a base64 encoded string but this seems to result in the data being encoded twice.

The owned members of the result of both Uint8Array.from(data, (v) => v.charCodeAt(0))) and new TextEncoder().encode(token) appear to be identical, but the latter is rejected.

Additional Information/Context

#7078 seems to be related

The following change may also be related

https://github.com/aws/aws-sdk-js-v3/pull/4836/files#diff-caa976711e9724e7c04af786a57ef4f06c096e0529027cfaa35b6cf0382b1ba5

[GWellerGMSL]

Updated to mention Uint8Array.from(data, (v) => v.charCodeAt(0))).

I wonder if its something to do with the way that Buffer works in Node v22.

[kuhe]

I'm not seeing this behavior. I ran this test script.
Plain strings, Buffers, Uint8Arrays all converted to base64 string in JSON in the request.

import { KMS } from "@aws-sdk/client-kms";

const kms = new KMS({
  requestHandler: {
    async handle(req) {
      console.log(req.body);
      process.exit(0);
    },
  },
});

const r = await kms.encrypt({
  Plaintext: Buffer.from("test"),
});

{"Plaintext":"dGVzdA=="}

Did anything change the type of Buffer in your runtime environment to be something that isn't an instanceof Uint8Array?

[GWellerGMSL]

Ah, yes the runtime environment is a very good point. The context is a NextJS application, and I was running that code from a Jest test. When using the requestHandler() you demonstrated above I get the following in test...

{"Plaintext":{"type":"Buffer","data":[116,101,115,116]}}

However when I call the exact same code inside the mainline code and run next dev I get the expected base64 encoded string you showed. 🤦

For reference, the Jest config I have is largely as described in the NextJS documentation...

https://nextjs.org/docs/app/guides/testing/jest

Except with testEnvironment: "jest-fixed-jsdom". I don't know why this would cause the AWS SDK to behave this way, but I suppose thats a whole can of worms thats beyond your remit. Oh well, maybe this might help someone else in future.

[kuhe]

Possibly related: jestjs/jest#13547

If your test is simulating a browser environment, the application code should refrain from using Buffer when combined with the AWS SDK unless the polyfill extends Uint8Array such as the implementation provided by https://www.npmjs.com/package/buffer.