Strings are encoded incorrectly in Atlas Triggered function

[Timmy_Pandahouse]

Inside an Atlas Triggered function, when I use JSON.stringify or console.log a property containing the curly quote character, e.g. ’, the output is changed to â.

What I did

Configure a Trigger function on a collection and log a property from a document, for example:

exports = async (changeEvent) => {
    const property = changeEvent && changeEvent.fullDocument && changeEvent.fullDocument.name
    console.log('the string', property)
    console.log('the json', JSON.stringify({ name: property }))
}

Create a document in the collection with a curly brace in the name property, e.g. { "name": "Test’s" }

View the log entry for the create.

What I expect to happen

The log entries should show the text using the curly quote, e.g. the output should be:

the string Test’s
the json {"name":"Test’s"}

What actually happens

The log entries show some other encoding mechanism:

the string Testâs
the json {"name":"Testâs"}

Additional details

This is not just a problem of the function log itself being encoded incorrectly: our trigger function pushes the JSON.stringify of the document to SQS, and the encoding shows the incorrect Testâs as well for the message on the SQS queue.

[Timmy_Pandahouse]

As of now, this behavior has been corrected, e.g. curly quotes now are left as curly quotes and so on.

I initially posted this to the MongoDB Jira board, before it was suggested by support that I post here. https://jira.mongodb.org/browse/SERVER-55450

Since I was able to isolate this behavior to the MongoDB Atlas triggered function, I would still like follow-up on what occurred on the MongoDB side, e.g. an incident report, and will comment here if I receive it.

[Timmy_Pandahouse]

This issue has come back again, and I have yet to hear any reply from a MongoDB representative.

[Timmy_Pandahouse]

For reference, the exact encoding transform is from ’ to â\u0080\u0099

[Timmy_Pandahouse]

@Timmy_Pandahouse

Create a document in the collection with a curly brace in the name property, e.g. { "name": "Test’s" }

View the log entry for the create.

To be more clear here, the exact steps are:

1. Create a document in the collection with a curly brace in the name property, e.g. { "name": "Test’s" }
2. Look in the Atlas Trigger logs for the Function execution, and view the log entry from the console.log outputs.

[Timmy_Pandahouse]

This appears to be an intermittent problem. The two times it happened so far are:

• We spotted the problem March 23 at about 09:00 CDT, but by around 14:00 CDT it started working correctly.
• Today (March 26) the problem started happening again, from a different Atlas Trigger, roughly around 10:00 CDT, but stopped happening at about 11:30 CDT.

I’ve been able to make a stopgap fix which takes care of the most visible+ugly character encoding problem, using this (not in the Realm function, it’s in an AWS post-process function):

const { get, set } = require('lodash')
const stringifyKeys = require('stringify-keys')

function fixMongoDbMangling(document) {
    stringifyKeys(document)
        .forEach(key => {
            if (typeof get(document, key) === 'string') {
                set(document, key, get(document, key).replace(/â\u0080\u0099/g, '’'))
            }
        })
}

[Timmy_Pandahouse]

Update on 2021-03-26 13:01:45

It has been confirmed that this is an issue on our end and we are actively working on a fix.

I have passed this information along to our Realm Engineering Team to help in their efforts in resolving this issue.

Then a later update 2021-03-29 00:47:21

My colleagues on the Realm Engineering Team believe that they fixed the issue