Conversation
# Context AWS has added support for multi-attribute indexes in DynamoDB. Support for this new functionality would be immensely helpful for users and is likely possible without breaking changes.
✅ Deploy Preview for electrodb-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@tywalch just wanted to mention that the "condition" option should not be available for multi-attribute indexes |
|
Hey @tywalch! Fortunately, AWS released an update yesterday to DynamoDB Local 3.3.0 that adds support for multi-attribute GSIs, so we no longer need to use an AWS-hosted DynamoDB table to test the new GSI 🎉 |
…ly identify an Entity within a service. These values are stored on DynamoDB items using "identifier" attributes. When building filter expressions for these identifier attributes we must also make the ExpressionAttributeNames and ExpressionAttributeValues unique within the context of the query operation. There are also character restrictions on these values imposed by DynamoDB. We use the name/alias of the Entity to help create these unique values. Howevery, if the Entity name/alias contains special characters that are not allowed in DynamoDB ExpressionAttributeNames/Values we must remove them. This can lead to potential collisions between different Entity names/aliases that when stripped of special characters become identical. These tests ensure that at a minimum queries can be performed successfully even when special characters are present in the Entity name/alias.
…ature/composite-attribute-indexes # Conflicts: # src/entity.js # src/errors.js # test/init.js # www/src/pages/en/reference/errors.mdx
…ature/composite-attribute-indexes
DynamoDB-Local now supports multi-attribute keys so actually hitting a live dynamodb api is unnecessary.
|
@anatolzak I believe this feature may be finished. Would you mind taking a once-over look to see if anything jumps out to you? |
@tywalch of course! I will take a look tomorrow (it's night time where I am) |
There was a problem hiding this comment.
- The
casingattribute should not be supported on composite indexes. We should throw an error if it's provided. - The
castattribute should not be supported on composite indexes. We should throw an error if it's provided. - When querying a composite index and specifying an attribute from the middle of the sort key, it gets silently swallowed
index: 'gsi1',
type: 'composite',
pk: { composite: ['a'] }
sk: {
composite: ['b', 'c']
}
// Electro ignores C completely and sends a query as if only `a` was provided. We should throw an error in this case
entity.query.gsi({ a: '', c: '' }).go()- you should only be able to pass one attribute to all the range methods like
lt,begins,between. I tested this manually, and DynamoDB will throw an error if you provide multiple attributes with the range operator.
Also
entity.query.gsi({ a: '' }).lt({ b: '', c: '' }).go()
// Electro will send params like a = ... and b = ... and c < ...
// even though the user may have intended for a = ... and b < ...
// we should throw an error if multiple attributes are provided to a range operation|
|
||
| function expectSubset(label: string, received: any, expected: any) { | ||
| try { | ||
| if (!received === undefined) { |
There was a problem hiding this comment.
!received === undefined will always evaluate to false
| } | ||
|
|
||
| function expectNotSubset(label: string, received: any, expected: any) { | ||
| if (!received === undefined) { |
There was a problem hiding this comment.
!received === undefined will always evaluate to false
| InvalidLoggerProvided: { | ||
| code: 1022, | ||
| section: "inconsistent-index-definition", | ||
| name: "Inconsistent Index Definition", | ||
| section: "invalid-listener-provided", | ||
| name: "InvalidListenerProvided", |
There was a problem hiding this comment.
InvalidLoggerProvided is mapped to listener metadata
|
@tywalch I briefly reviewed the PR, but primarily conducted manual testing and found a few issues. Let me know when you'd like me to take another look. I will be happy to do so! |
2 participants
Context
AWS has added support for multi-attribute indexes in DynamoDB. Support for this new functionality would be immensely helpful for users and is likely possible without breaking changes.
Approach
In this PR, I will be chipping away at support for this feature. My priority is to just-get-things-working™ with tests. Ideally, this uncovers edge cases and tough judgment calls.
Only once things are working and behavior is locked down, I'll revisit ways to improve code organization, error handling, documentation, etc.
Questions
[ ] Should collection support for "composite" attributes ultimately just be that they enable processing multiple entity types
[ ] Should we still try to achieve index-level entity/service isolation with "composite" indexes or should we just use filters?
[ ] Should we add a "meta" object as the third argument to
getandsetthat includes theentity,version, andservicevalues from the schema? This might help the user build their own isolation fields.Decisions
Learnings