Dynamoose Query Bug: KeyCondition Issue With GSI

by Alex Johnson 49 views

Introduction

In this article, we delve into a peculiar issue encountered while using Dynamoose, an Object-Document Mapper (ODM) for DynamoDB, where queries against Global Secondary Indexes (GSIs) fail with a ValidationException. The error message indicates that either KeyConditions or KeyConditionExpression must be specified, despite the code explicitly defining the key conditions. This discrepancy between Dynamoose's behavior and the expected behavior based on DynamoDB console queries suggests a potential bug or inconsistency in how Dynamoose constructs key conditions for GSIs. We'll explore the details of the problem, the code setup, and the expected versus actual outcomes.

Summary of the Issue

The core of the problem lies in Dynamoose's inability to correctly translate query requests for GSIs into valid DynamoDB queries. When a query is executed using Dynamoose with .query("accntId").eq(), which should target a GSI, Dynamoose generates a query request that lacks the necessary KeyConditionExpression. This omission leads to a ValidationException from DynamoDB, halting the query execution. The perplexing part is that the same key values and GSI configuration work flawlessly when queried directly through the DynamoDB console, pointing to an issue within Dynamoose's query building process.

Detailed Explanation

To understand the issue better, let's break down the components involved:

1. DynamoDB Schema Definition

The DynamoDB schema defines the structure of the data, including the primary key, attributes, and any GSIs. In this case, the schema includes an index on accntId. Multiple GSIs are defined, including:

  • accntId-index
  • accntId-updatedAt-index
  • accntId-status-index

These indexes allow efficient querying based on accntId and, in some cases, combined with updatedAt or status. The schema definition is crucial as it dictates how Dynamoose interacts with the DynamoDB table.

const { uuid, timeStamp } = require("../../utils/util");
const { Schema } = require("dynamoose");
const statusVals = ["SCHEDULED", "ACTIVE", "STOPPED", "PAUSED", "COMPLETED", "DELETE","IN_PROGRESS","CANCELLED"];
const enrolmntVals = ["ALL_COHORT_USER", "SELECTED_COHORT_USER", "SELECTED_COMPANY_USER"];
const isNewUserAllowedOptions = ["NEW_USERS_NO", "NEW_USERS_STRT_FIRST", "NEW_USERS_STRT_CURRENT"]

const scheduleSchema = new Schema({
    "entityId": {
        "type": String,
        "hashKey": true
    },
    "recType": {
        "type": String,
        "rangeKey": true
    },
    "accntId": {
        "type": String,
        "index":[ {
            "name": "accntId-index",
            "global": true
        },{
            "name": "accntId-updatedAt-index",
            "global": true,
            hashKey:"accntId",
            rangeKey:"updatedAt"
        },{
            "name": "accntId-status-index",
            "global": true,
            hashKey:"accntId",
            rangeKey:"status"
        }]
    },
    "cohortId": {
        "type": String,
        "index": {

            "name": "cohortId-index",
            "global": true
        }
    },
    "cohortName": {
        "type": String
    },
    "cohortuserCnt": {
        "type": Number
    },
    "courseId": {
        "type": String,
        "index": {
            "name": "courseId-index",
            "global": true
        }
    },
    "courseName": {
        "type": String
    },
    "chapterCnt": {
        "type": Number
    },
    "cardCnt": {
        "type": Number
    },
    "scheduleId": {
        "type": String
    },
    "description": {
        "type": String
    },
    "scheduleName": {
        "type": String
    },
    "scheduleInfo": {
        "type": Object
    },
    "nextScheduledTime": {
        "type": String
    },
    "status": {
        "type": String,
        "index":{
            "name":"status-index",
            "global":true
        },
        "default": "SCHEDULED",
        "enum": statusVals
    },
    "userRespForCourse": {
        "type": Object
    },
    "isUserRespReqrd": {
        "type": Boolean
    },
    // "isReminderReqrd": {
    //      "type":Boolean,
    //      "index": {
    //         // "name": "isReminderReqrd-status-index",
    //         "name":"isReminderReqrd-index",
    //         "global": true
    //     }
    //  },
     "isReminderReqrd": {
         "type":Boolean
     },
    "enrolmntType": {
        "type": String,
        "default": "SELECTED_COHORT_USER",
        "enum": enrolmntVals
    },
    "isNewUserAllowed": {
        "type": String,
        "default": "NEW_USERS_NO",
        "enum": isNewUserAllowedOptions
    },
    "timezone": {
        "type": String
    },
    "on": {
        "type": Array
    },
    "isSchdlReqrd": {
        "type": Boolean
    },
    "date": {
        "type": String
    },
    "entryType": {
        "type": String
    },
    "createdBy": {
        "type": String
    },
    "updatedBy": {
        "type": String
    },
    "cardsPerSchedule": {
        "type": Number
    },
    "assessmentFlag": {
        "type": Boolean
    },
    "assessFBReq": {
        "type": Boolean
    },
    "enableRetake": {
        "type": Boolean
    },
    "benchmarkScore": {
        "type": Number
    },
    "assessQuesCnt": {
        "type": Number
    },
    "assessmentPlacement": {
        "type": String
    },
    "languageCode" :{
        "type":String
    },
    "certificateReq":{
        "type":Boolean
    }
}, {
    "saveUnknown": true,
    "timestamps": true
});

module.exports = {
    scheduleSchema,
    statusVals
};

2. Dynamoose Query

The query in question is constructed using Dynamoose's query builder. The code attempts to filter data based on the accntId attribute, which is part of a GSI. However, Dynamoose fails to correctly translate this into a valid DynamoDB query.

3. ValidationException

The ValidationException indicates that the DynamoDB service received a query request that is missing the necessary key conditions. This typically happens when querying an index without specifying the index's hash key in the query. The error message, "Either the KeyConditions or KeyConditionExpression parameter must be specified in the request," is a direct result of this omission.

4. DynamoDB Console Success

The fact that the same query works correctly in the DynamoDB console is a critical piece of information. It confirms that the GSI is properly configured and that the data exists as expected. This eliminates potential issues with the index setup or data integrity, focusing the problem squarely on Dynamoose's query generation.

Expected vs. Current Behavior

The expected behavior is that Dynamoose should generate a valid KeyConditionExpression for the query, ensuring that the DynamoDB service can correctly process the request. This expression should include the partition key (accntId) and any optional sort keys (e.g., status) defined in the GSI. The current behavior, however, is that Dynamoose sends a query request without this essential KeyConditionExpression, leading to the ValidationException.

Environment Details

The environment in which this issue occurs is as follows:

  • Operating System: Windows 10
  • Operating System Version: 22H2
  • Node.js Version: v20.x
  • NPM Version: 10.x
  • Dynamoose Version: Latest (as of November 2025)

These details are important for reproducibility and debugging purposes.

Potential Causes and Solutions

Several factors might be contributing to this issue:

  1. Dynamoose Bug: The most likely cause is a bug in Dynamoose's query builder, specifically related to how it handles GSIs. This could be due to incorrect logic in constructing the KeyConditionExpression or a failure to recognize the GSI properly.
  2. Incorrect Index Configuration: Although the DynamoDB console query works, it's worth double-checking the index configuration in Dynamoose to ensure it matches the actual GSI definition in DynamoDB.
  3. Version Incompatibility: While the user is running the latest version of Dynamoose, there might be compatibility issues with other libraries or the Node.js version being used.

To address this issue, the following steps can be taken:

  1. Verify Index Configuration: Ensure that the index names and key names in the Dynamoose schema match exactly with the GSI definition in DynamoDB. Any discrepancies can lead to incorrect query generation.
  2. Examine Dynamoose Query Output: Use debugging tools or logging to inspect the actual query request generated by Dynamoose. This can help identify whether the KeyConditionExpression is missing or malformed.
  3. Test with AWS SDK Directly: Bypass Dynamoose and use the AWS SDK directly to construct and execute the query. This can help isolate whether the issue is with Dynamoose or with the underlying AWS SDK.
  4. Update Dynamoose: Make sure you are using the latest version of Dynamoose, as updates often include bug fixes and improvements.
  5. Contact Dynamoose Community: Report the issue to the Dynamoose community, providing all relevant details, including the code samples, environment information, and the steps taken to reproduce the issue.

Conclusion

The ValidationException encountered while querying a GSI with Dynamoose, despite the same query working in the DynamoDB console, indicates a potential issue with Dynamoose's query building process. By carefully examining the schema definition, query construction, and environment details, it may be possible to identify the root cause and implement a solution. If all else fails, reaching out to the Dynamoose community for assistance is a viable option. For more information on DynamoDB and best practices, refer to the AWS DynamoDB Documentation.