Java – mongodb is obviously too big for a 16MB hat

I have a mongodb series Simply put, it has two columns: user and URL It has 39274590 lines The key to this table is {user, URL}

Using Java, I try to list different URLs:

MongoDBManager db = new MongoDBManager( "Website","UserLog" );
  return db.getDistinct("url");

But I received an exception:

Exception in thread "main" com.mongodb.CommandResult$CommandFailure: command Failed [distinct]: 
{ "serverUsed" : "localhost/127.0.0.1:27017","errmsg" : "exception: distinct too big,16mb cap","code" : 10044,"ok" : 0.0}

How can I solve this problem? Is there a plan B that can avoid this problem?

thank you.

Solution

In version 2.6, you can generate separate collections using the aggregate command:

For most queries, this will limit mongodb to 16 MB You can read more about using the aggregation framework for large datasets in mongodb 2.6 here: http://vladmihalcea.com/mongodb-2-6-is-out/

To perform "different" queries using the aggregation framework, group by field

db.userlog.aggregate([{$group: {_id: '$url'} }]);

Note: I don't know what this is for Java drivers. Good luck