Storing strings in MongoDB larger than 16 MB

Working with Data
1

Hello,

I would like to store strings larger than 16 MB into MongoDB. From the documentation I learned that there is GridFS that I can use to store files into the database.

Here’s how my software works:

  1. Users fills out a form (for example: username, text and file)
  2. Users clicks on submit button
  3. The data from the form will be transfered to the servers as strings (also the file the user can select from the form)
  4. The serverside PHP script will store all information in a MongoDB document

So the file (Point 3) that the user can select will be converted to a string and then transfered to the server.

In the documentation I couldn’t find a class method to store strings to GridFS. Is there a way I can store them into GridFS? Or is there a way I can store it normally in a collection? For example, can I set the size limit for data in MongoDB? Is that possible?

2

Hello, @Cryptnote!

You can check MongoDB driver’s tutorials for GridFS usage examples

1 Like
3

Well yes, I know the documentation. There’s no method for GridFS which you can use to store a large string.

4

I am a Java developer (am not familiar with PHP; I think you are using MongoDB PHP driver). I tried your issue of storing a string into GridFS collections. It works both ways, writing a string and retrieving it. Ofcourse, I used Java APIs. I can relate to what I did (code) in PHP, and I think a PHP developer can figure the rest.


Writing to GridFS:

Start from this topic: Uploading Files with Writable Streams

See the bucket class function: MongoDB\GridFS\Bucket::uploadFromStream

The function definition:

uploadFromStream($filename, $source, array $options = [])

  • $filename: This can be any name (a string value) for identifying - this is stored as filename field in the resulting fs.files collection.
  • $source: Readable stream, from which the new GridFS file’s contents will be read.

The Readable stream is an input stream which is created from the input string; and this needs to be created in PHP code ($streamToUploadFrom). With Java, the stream is created as:
InputStream streamToUploadFrom = new ByteArrayInputStream(inputString.getBytes("UTF-8"));

Write the string to the MongoDB GridFS collections:

$bucket = (new MongoDB\Client)->test->selectGridFSBucket();
$bucket->uploadFromStream('my-file', $streamToUploadFrom);

The function returns: The _id field of the metadata document associated with the newly created GridFS file.

After the successful write, MongoDB creates two collections in the specified database: fs.chunks and fs.files. These collections can be queried from the mongo shell or from the driver APIs.


Reading from GridFS:

To read from the GridFS collection, use MongoDB\GridFS\Bucket::downloadToStream function. In this case, an output steam is to be created and the downloadToStream function reads from the stored string to the stream. Then, extract the string from the stream.

5

Before you answered I already figured this out, thank you anyway for posting this. I’m pretty sure that it will help someone if he searches Google. After solving this problem I wrote a short “tutorial” on how to store and retrieve files from the database, you can find it here.

6

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.