I'm back to the persons dataset we used earlier in the persons collection.
Now sometimes you want to get a feeling for the distribution of the data you have
and there is a useful command that can help you with that,
the bucket stage. So let's prepare our pipeline for this
and let's add the
$bucket stage,
now what does bucket do? The bucket stage allows you to output your data in, well in buckets for which
you can calculate certain summary statistics. Buckets takes a group by parameter here, a group by field
where you define by which field do you want to put your data into buckets
and here, I will go for the age,
so dob.age,
so here I'll refer to $dob.age. This tells bucket
ok
where is my input data essentially which I want to put into buckets.
Then you define some boundaries and these are essentially your categories,
so you could say I'm interested in my ages 0 to 18 to 30 to 50 to 80 to 120, something like this,
this would now create your bucket, so the different categories you have,
you want to categorize your data into.
Now the question is what do you want to output for these buckets
and here you define the structure of what you get back.
So I could say that in each bucket, I want to have an array of the names,
this can be done like in the group phase with the push operator and I simply push name in there.
Now name is an object here, it's this object
so maybe we just take the first names to keep it a bit shorter.
You could push objects too,
I just want to keep it shorter.
So now each bucket will have a document where I see the names of the people who are in the bucket,
I also want to see the average age, let's say. We can do that with the average operator you saw before
and here, I simply point at dob.age
and I also want to find out how many persons are in the bucket.
So for this we can use some one, just in the group stage, one will be added for every element in the bucket.
Now let's give this a try, for that
I'll move pretty and aggregate into their positions,
copy that
and execute it and now well we have a lot of names in there,
it probably was not my best idea to put all the names into the buckets.
Here we have a bucket border, here we see basically for which category this bucket is, we see there are
2300 people in there,
we see the average age,
now let me get rid of the names here because it's really hard to read that otherwise so I'll just use
the summary statistics now
and this is now easier to read.
So now we see we got three buckets, three categories essentially
and the reason for us having only three buckets is probably that we seem to have no persons younger
than 18 or older than 80,
so these buckets and this starting point seems to be redundant.
Let's quickly check this by running a query on persons and finding all persons who are younger than
18, so we can of course do that by pointing at dob.age and then using the lower than operator to see
who's lower than 18 and we got none
and now let's tweak this for greater than 80 and we got none there too.
So this is correct
and if we sum this up, we would also get our 5000 records by the way
and this is the bucket command and how we can use it to get an idea of the distribution.
Now we can of course fine tune this, since we know that we got no one who's younger than 18, we can get
rid of that
and we want to keep our end bound so that everyone fits in there but we could add more levels in between
for 40 and 60.
If I now run this, you see now we got more buckets for more granularity, this is 62 open and basically
and this gives us an idea for our distribution and the average age in every bucket and so on.
Now there also is an alternative to this,
you can also run,
let me quickly create an aggregate pipeline,
you can also run another stage which is called bucketAuto.
Now as the name suggests, bucketAuto does the bucketing algorithm for you. What you do here is you simply
define the group by key because of course you need to tell by which field to bucket,
so dob.age,
you then also define the number of buckets you want to have
and they will then be derived automatically,
so mongodb will look at your data and see where it should draw the boundaries,
so we could say we want five buckets. And then you also define the output of course because you still
define what you want to see
and I'll just copy the output from above.
So now if I turn this into a usable format, I can copy that and if I execute this now, you see now I get
this output, mongodb tells me which boundaries it created for me
so I see that the youngest person seems to be 21 years old
and then I get these buckets with my summary statistics.
Now each bucket holds almost the same amount of values because mongodb tried to derive an equal
distribution and bucketAuto can be an even quicker way for getting a feeling for your data.