Ticket: Migration for java developer Broken pipe

I got errors when I run my code, I guess it’s because the dateStringFilter returned too many items, but I couldn’t figure out how to change it. below is my code
Caused by: javax.net.ssl.SSLProtocolException: Broken pipe (Write failed)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:126)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:988)
at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:99)
at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:426)
… 20 more

public static void main(String args) {

    System.out.println("Dataset cleanup migration");

    // set your MongoDB Cluster connection string
    String mongoUri = "mongodb+srv://m220student:121121212121@cluster0-qubxy.gcp.mongodb.net/sample_mflix";
    // instantiate database and collection objects
    MongoDatabase mflix = MongoClients.create(mongoUri).getDatabase("sample_mflix");
    MongoCollection<Document> movies = mflix.getCollection("movies");
    Bson dateStringFilter = and( exists("lastupdated"), type("lastupdated", "string") );
    String datePattern = "yyyy-MM-dd HH:mm:ss";

    // documents that are required to be updated and the correct date
    // format pattern
    Document queryFilter = new Document();
    SimpleDateFormat dateFormat = new SimpleDateFormat(datePattern);

    // create list of bulkWrites to be applied.
    List<WriteModel<Document>> bulkWrites = new ArrayList<>();
    List<Document> list = new ArrayList<>();
    movies
        .find(dateStringFilter)
        .iterator()
        .forEachRemaining(list::add);


    for (Document doc : list) {

        // Apply lastupdate string to date conversion
        WriteModel<Document> updateDate = transformDates(doc, dateFormat);
        if (updateDate != null) {
            bulkWrites.add(updateDate);
        }
        System.out.println(bulkWrites.size());
    }


    // iterate over the documents and apply the transformations.
    for (Document doc : movies.find(dateStringFilter)) {

        // Apply lastupdate string to date conversion
        WriteModel<Document> updateDate = transformDates(doc, dateFormat);
        if (updateDate != null) {
            bulkWrites.add(updateDate);
        }
        if(doc.size() > 20)
            break;
    }

    // TODO> Ticket: Migration - create a query filter that finds
    // documents where `imdb.rating` is of type string
    Bson ratingStringFilter = and( exists("imdb.rating"), type("imdb.rating", "string") );
    for (Document doc : movies.find(ratingStringFilter)) {
        // Apply "imdb.rating" string to number conversion
        WriteModel<Document> updateRating = transformRating(doc);
        if (updateRating != null) {
            bulkWrites.add(updateRating);
        }
    }

    // execute the bulk update
    BulkWriteOptions bulkWriteOptions = new BulkWriteOptions().ordered(false);
    if (bulkWrites.isEmpty()) {
        System.out.println("Nothing to update!");
        System.exit(0);
    }

    BulkWriteResult bulkResult = movies.bulkWrite(bulkWrites, bulkWriteOptions);
    // output the number of updated documents
    System.out.println(
        MessageFormat.format("Updated {0} documents", bulkResult.getModifiedCount()));
}

for now I can use for (Document doc : list.subList(0,100)) to bypass the test, but I don’t think this is the beautiful solution to that

I see repetition of code. You are bulk adding to the list two times.
And did you see the detailed explanation for the correct implementation? I can see we can improve the code for the filter you have written.

Kanika

yes, I deleted the duplicate code but still the same, could you please shed some light on the improvement of filter? thanks very much~